2023-10-11T05:54:15.929 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-11T05:54:15.930 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719 --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/orig.config.yaml 2023-10-11T05:54:15.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-11T05:54:16.078 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719 branch: wip-vshankar-testing-reef-20231009.131610 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} email: vshankar@redhat.com first_in_suite: false job_id: '7418719' kernel: flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-reef-20231009.131610 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-reef-20231009.131610 sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1116 sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af sleep_before_teardown: 0 subset: 593/1024 suite: fs suite_branch: wip-vshankar-testing-reef-20231009.131610 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIz07+vwo6enfYjYT3bgN9oAW/+WdhG0ul6F9y0AySoW1NTZ85wpDnIsN/oxucbA12mY5FlIxagz7oq58PLVr2s= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO4Fkjn9MgszQq+O3mVjZjzL214/drNn9oJ1GIkUSVYEf036TMD8K0z7blUuOud3nbFZcubNP/Ios9UYGazh49c= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-09_16:48:27 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-11T05:54:16.078 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa; will attempt to use it 2023-10-11T05:54:16.079 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks 2023-10-11T05:54:16.079 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-11T05:54:16.081 INFO:teuthology.task.internal:Checking packages... 2023-10-11T05:54:16.103 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a774796f0ecdd11a4b771de396e627aa1a187af' 2023-10-11T05:54:16.104 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-11T05:54:16.104 INFO:teuthology.packaging:ref: None 2023-10-11T05:54:16.105 INFO:teuthology.packaging:tag: None 2023-10-11T05:54:16.105 INFO:teuthology.packaging:branch: wip-vshankar-testing-reef-20231009.131610 2023-10-11T05:54:16.105 INFO:teuthology.packaging:sha1: 9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T05:54:16.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-reef-20231009.131610 2023-10-11T05:54:16.252 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-662.g9a774796 2023-10-11T05:54:16.252 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-11T05:54:16.259 INFO:teuthology.task.internal:no buildpackages task found 2023-10-11T05:54:16.259 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-11T05:54:16.265 INFO:teuthology.task.internal:Saving configuration 2023-10-11T05:54:16.284 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-11T05:54:16.328 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-11T05:54:16.352 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 05:49:11.621701', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIz07+vwo6enfYjYT3bgN9oAW/+WdhG0ul6F9y0AySoW1NTZ85wpDnIsN/oxucbA12mY5FlIxagz7oq58PLVr2s='} 2023-10-11T05:54:16.376 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 05:49:11.619716', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO4Fkjn9MgszQq+O3mVjZjzL214/drNn9oJ1GIkUSVYEf036TMD8K0z7blUuOud3nbFZcubNP/Ios9UYGazh49c='} 2023-10-11T05:54:16.376 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-11T05:54:16.382 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-11T05:54:16.382 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-11T05:54:16.382 INFO:teuthology.run_tasks:Running task console_log... 2023-10-11T05:54:16.536 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8e9ed88a60>, signals=[15]) 2023-10-11T05:54:16.537 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-11T05:54:16.544 INFO:teuthology.task.internal:Opening connections... 2023-10-11T05:54:16.544 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T05:54:16.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:54:16.634 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T05:54:16.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:54:16.717 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-11T05:54:16.723 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2023-10-11T05:54:16.741 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2023-10-11T05:54:16.741 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2023-10-11T05:54:16.798 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2023-10-11T05:54:16.798 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2023-10-11T05:54:16.798 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2023-10-11T05:54:16.799 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2023-10-11T05:54:16.799 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2023-10-11T05:54:16.799 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2023-10-11T05:54:16.799 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-11T05:54:16.799 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2023-10-11T05:54:16.800 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2023-10-11T05:54:16.800 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-11T05:54:16.800 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2023-10-11T05:54:16.800 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-11T05:54:16.800 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-11T05:54:16.801 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-11T05:54:16.801 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2023-10-11T05:54:16.836 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-10-11T05:54:16.854 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-10-11T05:54:16.854 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-10-11T05:54:16.910 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2023-10-11T05:54:16.911 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2023-10-11T05:54:16.911 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2023-10-11T05:54:16.911 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2023-10-11T05:54:16.911 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2023-10-11T05:54:16.911 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2023-10-11T05:54:16.912 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-11T05:54:16.912 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-10-11T05:54:16.912 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2023-10-11T05:54:16.912 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-11T05:54:16.912 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2023-10-11T05:54:16.912 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-11T05:54:16.913 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-11T05:54:16.913 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-11T05:54:16.913 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-10-11T05:54:16.936 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-11T05:54:16.945 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-11T05:54:16.950 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-11T05:54:16.951 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2023-10-11T05:54:16.953 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2023-10-11T05:54:16.969 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-11T05:54:16.988 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-11T05:54:16.988 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2023-10-11T05:54:17.009 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2023-10-11T05:54:17.114 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-11T05:54:17.166 INFO:teuthology.run_tasks:Running task kernel... 2023-10-11T05:54:17.180 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-10-11T05:54:17.181 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, timeout 300 2023-10-11T05:54:17.181 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-11T05:54:17.182 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-11T05:54:17.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T05:54:17.198 DEBUG:teuthology.task.kernel:sha1 for mon.a is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:54:17.199 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-11T05:54:17.199 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2023-10-11T05:54:17.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T05:54:17.200 DEBUG:teuthology.task.kernel:sha1 for mon.b is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:54:17.200 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-11T05:54:17.201 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-10-11T05:54:17.256 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-361.el9.x86_64 2023-10-11T05:54:17.256 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:54:17.257 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-11T05:54:17.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:54:17.262 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-361.el9.x86_64 2023-10-11T05:54:17.262 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:54:17.262 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-11T05:54:17.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:54:17.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum upgrade -y linux-firmware 2023-10-11T05:54:17.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum upgrade -y linux-firmware 2023-10-11T05:54:19.473 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 16 MB/s | 6.3 MB 00:00 2023-10-11T05:54:19.616 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-10-11T05:54:20.765 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 8.5 MB/s | 2.0 MB 00:00 2023-10-11T05:54:20.950 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2023-10-11T05:54:21.756 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 7.9 MB 00:00 2023-10-11T05:54:21.799 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 7.9 MB 00:00 2023-10-11T05:54:24.357 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 12 MB/s | 18 MB 00:01 2023-10-11T05:54:24.491 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-10-11T05:54:28.632 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 5.7 MB 00:00 2023-10-11T05:54:28.758 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 5.7 MB 00:00 2023-10-11T05:54:30.066 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 14 kB 00:00 2023-10-11T05:54:30.234 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 14 kB 00:00 2023-10-11T05:54:31.302 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 20 MB/s | 19 MB 00:00 2023-10-11T05:54:34.767 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 19 MB 00:04 2023-10-11T05:54:35.819 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-10-11T05:54:37.959 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-10-11T05:54:37.967 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-10-11T05:54:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-10-11T05:54:37.967 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-10-11T05:54:37.968 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-10-11T05:54:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-11T05:54:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T05:54:37.969 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-10-11T05:54:37.969 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2023-10-11T05:54:37.969 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 1 Package 2023-10-11T05:54:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T05:54:37.970 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 326 M 2023-10-11T05:54:37.970 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-10-11T05:54:39.191 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2023-10-11T05:54:41.345 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2023-10-11T05:54:41.355 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2023-10-11T05:54:41.355 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2023-10-11T05:54:41.356 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2023-10-11T05:54:41.356 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2023-10-11T05:54:41.356 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-11T05:54:41.356 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T05:54:41.356 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2023-10-11T05:54:41.357 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2023-10-11T05:54:41.357 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 1 Package 2023-10-11T05:54:41.357 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T05:54:41.357 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 326 M 2023-10-11T05:54:41.357 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2023-10-11T05:54:47.727 INFO:teuthology.orchestra.run.smithi099.stdout:linux-firmware-20230926-140.el9.noarch.rpm 33 MB/s | 326 MB 00:09 2023-10-11T05:54:47.728 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-10-11T05:54:47.728 INFO:teuthology.orchestra.run.smithi099.stdout:Total 33 MB/s | 326 MB 00:09 2023-10-11T05:54:47.728 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-10-11T05:54:47.828 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-10-11T05:54:47.828 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-10-11T05:54:49.841 INFO:teuthology.orchestra.run.smithi084.stdout:linux-firmware-20230926-140.el9.noarch.rpm 38 MB/s | 326 MB 00:08 2023-10-11T05:54:49.842 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2023-10-11T05:54:49.842 INFO:teuthology.orchestra.run.smithi084.stdout:Total 38 MB/s | 326 MB 00:08 2023-10-11T05:54:49.842 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2023-10-11T05:54:49.930 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2023-10-11T05:54:49.930 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2023-10-11T05:54:49.951 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-10-11T05:54:49.951 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-10-11T05:54:52.082 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2023-10-11T05:54:52.083 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2023-10-11T05:54:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-10-11T05:54:56.126 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-11T05:54:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-11T05:54:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-11T05:54:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-11T05:54:57.602 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-11T05:54:57.603 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T05:54:57.603 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-10-11T05:54:57.603 INFO:teuthology.orchestra.run.smithi099.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-11T05:54:57.603 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T05:54:57.603 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-10-11T05:54:57.693 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2023-10-11T05:54:57.898 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-10-11T05:54:57.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:54:58.118 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ 2023-10-11T05:54:58.119 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-11T05:54:58.121 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi099.front.sepia.ceph.com to complete... 2023-10-11T05:54:58.281 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-11T05:54:58.371 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-11T05:54:59.283 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-11T05:54:59.283 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-11T05:54:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-11T05:54:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T05:54:59.691 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2023-10-11T05:54:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-11T05:54:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T05:54:59.692 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2023-10-11T05:54:59.996 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-10-11T05:54:59.997 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-11T05:55:00.156 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ 2023-10-11T05:55:00.157 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-11T05:55:00.159 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi084.front.sepia.ceph.com to complete... 2023-10-11T05:55:23.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11 05:55:23 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [318991812/318991812] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-11T05:55:23.763 INFO:teuthology.orchestra.run.smithi084.stderr:FINISHED --2023-10-11 05:55:23-- 2023-10-11T05:55:23.764 INFO:teuthology.orchestra.run.smithi084.stderr:Total wall clock time: 24s 2023-10-11T05:55:23.764 INFO:teuthology.orchestra.run.smithi084.stderr:Downloaded: 1 files, 304M in 23s (13.2 MB/s) 2023-10-11T05:55:23.765 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-10-11T05:55:23.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-11T05:55:25.604 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying... ######################################## 2023-10-11T05:55:25.660 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing... ######################################## 2023-10-11T05:55:25.739 INFO:teuthology.orchestra.run.smithi084.stdout:Updating / installing... 2023-10-11T05:55:32.774 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-11T05:55:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11 05:55:43 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [318991812/318991812] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-11T05:55:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:FINISHED --2023-10-11 05:55:43-- 2023-10-11T05:55:43.775 INFO:teuthology.orchestra.run.smithi099.stderr:Total wall clock time: 46s 2023-10-11T05:55:43.775 INFO:teuthology.orchestra.run.smithi099.stderr:Downloaded: 1 files, 304M in 45s (6.75 MB/s) 2023-10-11T05:55:43.775 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-10-11T05:55:43.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-11T05:55:45.602 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying... ######################################## 2023-10-11T05:55:45.690 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing... ######################################## 2023-10-11T05:55:45.769 INFO:teuthology.orchestra.run.smithi099.stdout:Updating / installing... 2023-10-11T05:55:52.758 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-11T05:56:08.648 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-11T05:56:08.649 DEBUG:teuthology.orchestra.run.smithi084:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-11T05:56:08.717 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:08.718 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:08.718 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:08.718 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:08.718 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-11T05:56:08.718 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-11T05:56:08.719 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-11T05:56:08.719 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-11T05:56:08.719 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-11T05:56:08.719 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-11T05:56:08.719 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-11T05:56:08.720 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-11T05:56:08.720 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-11T05:56:08.720 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-11T05:56:08.720 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-11T05:56:08.720 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-11T05:56:08.721 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-11T05:56:08.721 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-11T05:56:08.721 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-11T05:56:08.721 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-11T05:56:08.721 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-11T05:56:08.721 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-11T05:56:08.722 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-11T05:56:08.722 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-11T05:56:08.722 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-11T05:56:08.722 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-11T05:56:08.722 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-11T05:56:08.723 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-11T05:56:08.723 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-11T05:56:08.723 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-11T05:56:08.723 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-11T05:56:08.723 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-11T05:56:08.724 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-11T05:56:08.724 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-11T05:56:08.724 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-11T05:56:08.724 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-11T05:56:08.724 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-11T05:56:08.724 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-11T05:56:08.725 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-11T05:56:08.725 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-11T05:56:08.725 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-11T05:56:08.725 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-11T05:56:08.725 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-11T05:56:08.726 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-11T05:56:08.726 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-11T05:56:08.726 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-11T05:56:08.726 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-11T05:56:08.726 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-11T05:56:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-11T05:56:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-11T05:56:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-11T05:56:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-11T05:56:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-11T05:56:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-11T05:56:08.728 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-11T05:56:08.728 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-11T05:56:08.728 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-11T05:56:08.728 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-11T05:56:08.728 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-11T05:56:08.729 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-11T05:56:08.729 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-11T05:56:08.729 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-11T05:56:08.729 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-11T05:56:08.729 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-11T05:56:08.729 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-11T05:56:08.730 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-11T05:56:08.730 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-11T05:56:08.730 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-11T05:56:08.730 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-11T05:56:08.730 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-11T05:56:08.731 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-11T05:56:08.731 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-11T05:56:08.731 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-11T05:56:08.731 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-11T05:56:08.731 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-11T05:56:08.731 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-11T05:56:08.732 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-11T05:56:08.732 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-11T05:56:08.732 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-11T05:56:08.732 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-11T05:56:08.732 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-11T05:56:08.733 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-11T05:56:08.733 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-11T05:56:08.733 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-11T05:56:08.733 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-11T05:56:08.733 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-11T05:56:08.733 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-11T05:56:08.734 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-11T05:56:08.734 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-11T05:56:08.734 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-11T05:56:08.734 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-11T05:56:08.734 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-11T05:56:08.735 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-11T05:56:08.735 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-11T05:56:08.735 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-11T05:56:08.735 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-11T05:56:08.735 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-11T05:56:08.735 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-11T05:56:08.736 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-11T05:56:08.736 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-11T05:56:08.736 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-11T05:56:08.736 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-11T05:56:08.736 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-11T05:56:08.737 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-11T05:56:08.737 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-11T05:56:08.737 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-11T05:56:08.737 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-11T05:56:08.737 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-11T05:56:08.738 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-11T05:56:08.738 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-11T05:56:08.738 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-11T05:56:08.738 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-11T05:56:08.738 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-11T05:56:08.738 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-11T05:56:08.739 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-11T05:56:08.739 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-11T05:56:08.739 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-11T05:56:08.739 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-11T05:56:08.739 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-11T05:56:08.739 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-11T05:56:08.740 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-11T05:56:08.740 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-11T05:56:08.740 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-11T05:56:08.740 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-11T05:56:08.741 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-11T05:56:08.741 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-11T05:56:08.741 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-11T05:56:08.741 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-11T05:56:08.741 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-11T05:56:08.741 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-11T05:56:08.742 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-11T05:56:08.742 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-11T05:56:08.742 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-11T05:56:08.742 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-11T05:56:08.742 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-11T05:56:08.743 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-11T05:56:08.743 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-11T05:56:08.743 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-11T05:56:08.743 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-11T05:56:08.743 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-11T05:56:08.744 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-11T05:56:08.744 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-11T05:56:08.744 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-11T05:56:08.744 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-11T05:56:08.744 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-11T05:56:08.745 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-11T05:56:08.745 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-11T05:56:08.745 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-11T05:56:08.745 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-11T05:56:08.745 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-11T05:56:08.746 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-11T05:56:08.746 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-11T05:56:08.746 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-11T05:56:08.746 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-11T05:56:08.746 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-11T05:56:08.746 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-11T05:56:08.747 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-11T05:56:08.747 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-11T05:56:08.747 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-11T05:56:08.747 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-11T05:56:08.747 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-11T05:56:08.748 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-11T05:56:08.748 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-11T05:56:08.748 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-11T05:56:08.748 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-11T05:56:08.748 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-11T05:56:08.749 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-11T05:56:08.749 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-11T05:56:08.749 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-11T05:56:08.749 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-11T05:56:08.749 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-11T05:56:08.750 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-11T05:56:08.750 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-11T05:56:08.750 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-11T05:56:08.750 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-11T05:56:08.750 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-11T05:56:08.751 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-11T05:56:08.751 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-11T05:56:08.751 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-11T05:56:08.751 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-11T05:56:08.751 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-11T05:56:08.752 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-11T05:56:08.752 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-11T05:56:08.752 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-11T05:56:08.752 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-11T05:56:08.752 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-11T05:56:08.753 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-11T05:56:08.753 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-11T05:56:08.753 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-11T05:56:08.753 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-11T05:56:08.753 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-11T05:56:08.754 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-11T05:56:08.754 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-11T05:56:08.754 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-11T05:56:08.754 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-11T05:56:08.754 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-11T05:56:08.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-11T05:56:08.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-11T05:56:08.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-11T05:56:08.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-11T05:56:08.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-11T05:56:08.755 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-11T05:56:08.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-11T05:56:08.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-11T05:56:08.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-11T05:56:08.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-11T05:56:08.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-11T05:56:08.756 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-11T05:56:08.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-11T05:56:08.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-11T05:56:08.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-11T05:56:08.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-11T05:56:08.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-11T05:56:08.757 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-11T05:56:08.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-11T05:56:08.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-11T05:56:08.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-11T05:56:08.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-11T05:56:08.758 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-11T05:56:08.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-11T05:56:08.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-11T05:56:08.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-11T05:56:08.759 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-11T05:56:08.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-11T05:56:08.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-11T05:56:08.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-11T05:56:08.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-11T05:56:08.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-11T05:56:08.760 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-11T05:56:08.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-11T05:56:08.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-11T05:56:08.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-11T05:56:08.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-11T05:56:08.761 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-11T05:56:08.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-11T05:56:08.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-11T05:56:08.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-11T05:56:08.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-11T05:56:08.762 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-11T05:56:08.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-11T05:56:08.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-11T05:56:08.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-11T05:56:08.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-11T05:56:08.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-11T05:56:08.763 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-11T05:56:08.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-11T05:56:08.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-11T05:56:08.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-11T05:56:08.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-11T05:56:08.764 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-11T05:56:08.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-11T05:56:08.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-11T05:56:08.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-11T05:56:08.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-11T05:56:08.765 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-11T05:56:08.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-11T05:56:08.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-11T05:56:08.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-11T05:56:08.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-11T05:56:08.766 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-11T05:56:08.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-11T05:56:08.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-11T05:56:08.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-11T05:56:08.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-11T05:56:08.767 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-11T05:56:08.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-11T05:56:08.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-11T05:56:08.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-11T05:56:08.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-11T05:56:08.768 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-11T05:56:08.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-11T05:56:08.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-11T05:56:08.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-11T05:56:08.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-11T05:56:08.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-11T05:56:08.769 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-11T05:56:08.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-11T05:56:08.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-11T05:56:08.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-11T05:56:08.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-11T05:56:08.770 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-11T05:56:08.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-11T05:56:08.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-11T05:56:08.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-11T05:56:08.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-11T05:56:08.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-11T05:56:08.771 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-11T05:56:08.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-11T05:56:08.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-11T05:56:08.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-11T05:56:08.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-11T05:56:08.772 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-11T05:56:08.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-11T05:56:08.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-11T05:56:08.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-11T05:56:08.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-11T05:56:08.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-11T05:56:08.773 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-11T05:56:08.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-11T05:56:08.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-11T05:56:08.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-11T05:56:08.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-11T05:56:08.774 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-11T05:56:08.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-11T05:56:08.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-11T05:56:08.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-11T05:56:08.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-11T05:56:08.775 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-11T05:56:08.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-11T05:56:08.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-11T05:56:08.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-11T05:56:08.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-11T05:56:08.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-11T05:56:08.776 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-11T05:56:08.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-11T05:56:08.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-11T05:56:08.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-11T05:56:08.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-11T05:56:08.777 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-11T05:56:08.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-11T05:56:08.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-11T05:56:08.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-11T05:56:08.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-11T05:56:08.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-11T05:56:08.778 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-11T05:56:08.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-11T05:56:08.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-11T05:56:08.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-11T05:56:08.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-11T05:56:08.779 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-11T05:56:08.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-11T05:56:08.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-11T05:56:08.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-11T05:56:08.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-11T05:56:08.780 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-11T05:56:08.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-11T05:56:08.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-11T05:56:08.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-11T05:56:08.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-11T05:56:08.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-11T05:56:08.781 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-11T05:56:08.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-11T05:56:08.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-11T05:56:08.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-11T05:56:08.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-11T05:56:08.782 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-11T05:56:08.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-11T05:56:08.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-11T05:56:08.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-11T05:56:08.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-11T05:56:08.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-11T05:56:08.783 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-11T05:56:08.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-11T05:56:08.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-11T05:56:08.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-11T05:56:08.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-11T05:56:08.784 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-11T05:56:08.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-11T05:56:08.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-11T05:56:08.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-11T05:56:08.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-11T05:56:08.785 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-11T05:56:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-11T05:56:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-11T05:56:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-11T05:56:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-11T05:56:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-11T05:56:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-11T05:56:08.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-11T05:56:08.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-11T05:56:08.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-11T05:56:08.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-11T05:56:08.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-11T05:56:08.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-11T05:56:08.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-11T05:56:08.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-11T05:56:08.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-11T05:56:08.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-11T05:56:08.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-11T05:56:08.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-11T05:56:08.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-11T05:56:08.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-11T05:56:08.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-11T05:56:08.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-11T05:56:08.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-11T05:56:08.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-11T05:56:08.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-11T05:56:08.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-11T05:56:08.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-11T05:56:08.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-11T05:56:08.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-11T05:56:08.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-11T05:56:08.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-11T05:56:08.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-11T05:56:08.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-11T05:56:08.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-11T05:56:08.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-11T05:56:08.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-11T05:56:08.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-11T05:56:08.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-11T05:56:08.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-11T05:56:08.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-11T05:56:08.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-11T05:56:08.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-11T05:56:08.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-11T05:56:08.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-11T05:56:08.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-11T05:56:08.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-11T05:56:08.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-11T05:56:08.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-11T05:56:08.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-11T05:56:08.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-11T05:56:08.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-11T05:56:08.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-11T05:56:08.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-11T05:56:08.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-11T05:56:08.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-11T05:56:08.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-11T05:56:08.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-11T05:56:08.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-11T05:56:08.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-11T05:56:08.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-11T05:56:08.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-11T05:56:08.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-11T05:56:08.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-11T05:56:08.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-11T05:56:08.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-11T05:56:08.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-11T05:56:08.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-11T05:56:08.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-11T05:56:08.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-11T05:56:08.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-11T05:56:08.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-11T05:56:08.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-11T05:56:08.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-11T05:56:08.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-11T05:56:08.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-11T05:56:08.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-11T05:56:08.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-11T05:56:08.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-11T05:56:08.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-11T05:56:08.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-11T05:56:08.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-11T05:56:08.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-11T05:56:08.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-11T05:56:08.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-11T05:56:08.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-11T05:56:08.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-11T05:56:08.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-11T05:56:08.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-11T05:56:08.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-11T05:56:08.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-11T05:56:08.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-11T05:56:08.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-11T05:56:08.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-11T05:56:08.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-11T05:56:08.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-11T05:56:08.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-11T05:56:08.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-11T05:56:08.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-11T05:56:08.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-11T05:56:08.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-11T05:56:08.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-11T05:56:08.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-11T05:56:08.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-11T05:56:08.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-11T05:56:08.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-11T05:56:08.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-11T05:56:08.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-11T05:56:08.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-11T05:56:08.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-11T05:56:08.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-11T05:56:08.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-11T05:56:08.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-11T05:56:08.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-11T05:56:08.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-11T05:56:08.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-11T05:56:08.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-11T05:56:08.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-11T05:56:08.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-11T05:56:08.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-11T05:56:08.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-11T05:56:08.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-11T05:56:08.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-11T05:56:08.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-11T05:56:08.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-11T05:56:08.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-11T05:56:08.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-11T05:56:08.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-11T05:56:08.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-11T05:56:08.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-11T05:56:08.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-11T05:56:08.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-11T05:56:08.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-11T05:56:08.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-11T05:56:08.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-11T05:56:08.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-11T05:56:08.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-11T05:56:08.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-11T05:56:08.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-11T05:56:08.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-11T05:56:08.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-11T05:56:08.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-11T05:56:08.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-11T05:56:08.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-11T05:56:08.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-11T05:56:08.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-11T05:56:08.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-11T05:56:08.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-11T05:56:08.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-11T05:56:08.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-11T05:56:08.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-11T05:56:08.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-11T05:56:08.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-11T05:56:08.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-11T05:56:08.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-11T05:56:08.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-11T05:56:08.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-11T05:56:08.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-11T05:56:08.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-11T05:56:08.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-11T05:56:08.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-11T05:56:08.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-11T05:56:08.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-11T05:56:08.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-11T05:56:08.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-11T05:56:08.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-11T05:56:08.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-11T05:56:08.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-11T05:56:08.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-11T05:56:08.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-11T05:56:08.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-11T05:56:08.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-11T05:56:08.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-11T05:56:08.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-11T05:56:08.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-11T05:56:08.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-11T05:56:08.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-11T05:56:08.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-11T05:56:08.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-11T05:56:08.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-11T05:56:08.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-11T05:56:08.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-11T05:56:08.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-11T05:56:08.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-11T05:56:08.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-11T05:56:08.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-11T05:56:08.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-11T05:56:08.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-11T05:56:08.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-11T05:56:08.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-11T05:56:08.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-11T05:56:08.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-11T05:56:08.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-11T05:56:08.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-11T05:56:08.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-11T05:56:08.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-11T05:56:08.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-11T05:56:08.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-11T05:56:08.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-11T05:56:08.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-11T05:56:08.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-11T05:56:08.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-11T05:56:08.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-11T05:56:08.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-11T05:56:08.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-11T05:56:08.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-11T05:56:08.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-11T05:56:08.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-11T05:56:08.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-11T05:56:08.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-11T05:56:08.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-11T05:56:08.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-11T05:56:08.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-11T05:56:08.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-11T05:56:08.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-11T05:56:08.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-11T05:56:08.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-11T05:56:08.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-11T05:56:08.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-11T05:56:08.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-11T05:56:08.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-11T05:56:08.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-11T05:56:08.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-11T05:56:08.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-11T05:56:08.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-11T05:56:08.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-11T05:56:08.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-11T05:56:08.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-11T05:56:08.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-11T05:56:08.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-11T05:56:08.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-11T05:56:08.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-11T05:56:08.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-11T05:56:08.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-11T05:56:08.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-11T05:56:08.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-11T05:56:08.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-11T05:56:08.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-11T05:56:08.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-11T05:56:08.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-11T05:56:08.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-11T05:56:08.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-11T05:56:08.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-11T05:56:08.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-11T05:56:08.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-11T05:56:08.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-11T05:56:08.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-11T05:56:08.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-11T05:56:08.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-11T05:56:08.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-11T05:56:08.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-11T05:56:08.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-11T05:56:08.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-11T05:56:08.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-11T05:56:08.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-11T05:56:08.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-11T05:56:08.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-11T05:56:08.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-11T05:56:08.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-11T05:56:08.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-11T05:56:08.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-11T05:56:08.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-11T05:56:08.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-11T05:56:08.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-11T05:56:08.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-11T05:56:08.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-11T05:56:08.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-11T05:56:08.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-11T05:56:08.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-11T05:56:08.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-11T05:56:08.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-11T05:56:08.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-11T05:56:08.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-11T05:56:08.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-11T05:56:08.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-11T05:56:08.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-11T05:56:08.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-11T05:56:08.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-11T05:56:08.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-11T05:56:08.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-11T05:56:08.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-11T05:56:08.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-11T05:56:08.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-11T05:56:08.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-11T05:56:08.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-11T05:56:08.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-11T05:56:08.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-11T05:56:08.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-11T05:56:08.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-11T05:56:08.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-11T05:56:08.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-11T05:56:08.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-11T05:56:08.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-11T05:56:08.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-11T05:56:08.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-11T05:56:08.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-11T05:56:08.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-11T05:56:08.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-11T05:56:08.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-11T05:56:08.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-11T05:56:08.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-11T05:56:08.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-11T05:56:08.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-11T05:56:08.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-11T05:56:08.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-11T05:56:08.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-11T05:56:08.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-11T05:56:08.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-11T05:56:08.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-11T05:56:08.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-11T05:56:08.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-11T05:56:08.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-11T05:56:08.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-11T05:56:08.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-11T05:56:08.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-11T05:56:08.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-11T05:56:08.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-11T05:56:08.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-11T05:56:08.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-11T05:56:08.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-11T05:56:08.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-11T05:56:08.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-11T05:56:08.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-11T05:56:08.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-11T05:56:08.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-11T05:56:08.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-11T05:56:08.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-11T05:56:08.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-11T05:56:08.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-11T05:56:08.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-11T05:56:08.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-11T05:56:08.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-11T05:56:08.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-11T05:56:08.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-11T05:56:08.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-11T05:56:08.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-11T05:56:08.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-11T05:56:08.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-11T05:56:08.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-11T05:56:08.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-11T05:56:08.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-11T05:56:08.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-11T05:56:08.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-11T05:56:08.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-11T05:56:08.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-11T05:56:08.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-11T05:56:08.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-11T05:56:08.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-11T05:56:08.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-11T05:56:08.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-11T05:56:08.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-11T05:56:08.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-11T05:56:08.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-11T05:56:08.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-11T05:56:08.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-11T05:56:08.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-11T05:56:08.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-11T05:56:08.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-11T05:56:08.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-11T05:56:08.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-11T05:56:08.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-11T05:56:08.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-11T05:56:08.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-11T05:56:08.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-11T05:56:08.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-11T05:56:08.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-11T05:56:08.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-11T05:56:08.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-11T05:56:08.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-11T05:56:08.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-11T05:56:08.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-11T05:56:08.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-11T05:56:08.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-11T05:56:08.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-11T05:56:08.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-11T05:56:08.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-11T05:56:08.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-11T05:56:08.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-11T05:56:08.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-11T05:56:08.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-11T05:56:08.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-11T05:56:08.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-11T05:56:08.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-11T05:56:08.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-11T05:56:08.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-11T05:56:08.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-11T05:56:08.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-11T05:56:08.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-11T05:56:08.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-11T05:56:08.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-11T05:56:08.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-11T05:56:08.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-11T05:56:08.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-11T05:56:08.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-11T05:56:08.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-11T05:56:08.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-11T05:56:08.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-11T05:56:08.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-11T05:56:08.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-11T05:56:08.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-11T05:56:08.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-11T05:56:08.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-11T05:56:08.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-11T05:56:08.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-11T05:56:08.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-11T05:56:08.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-11T05:56:08.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-11T05:56:08.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-11T05:56:08.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-11T05:56:08.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-11T05:56:08.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-11T05:56:08.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-11T05:56:08.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-11T05:56:08.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-11T05:56:08.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-11T05:56:08.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-11T05:56:08.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-11T05:56:08.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-11T05:56:08.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-11T05:56:08.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-11T05:56:08.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-11T05:56:08.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-11T05:56:08.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-11T05:56:08.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-11T05:56:08.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-11T05:56:08.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-11T05:56:08.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-11T05:56:08.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-11T05:56:08.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-11T05:56:08.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-11T05:56:08.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-11T05:56:08.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-11T05:56:08.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-11T05:56:08.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-11T05:56:08.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-11T05:56:08.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-11T05:56:08.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-11T05:56:08.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-11T05:56:08.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-11T05:56:08.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-11T05:56:08.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-11T05:56:08.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-11T05:56:08.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-11T05:56:08.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-11T05:56:08.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-11T05:56:08.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-11T05:56:08.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-11T05:56:08.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-11T05:56:08.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-11T05:56:08.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-11T05:56:08.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-11T05:56:08.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-11T05:56:08.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-11T05:56:08.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-11T05:56:08.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-11T05:56:08.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-11T05:56:08.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-11T05:56:08.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-11T05:56:08.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-11T05:56:08.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-11T05:56:08.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-11T05:56:08.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-11T05:56:08.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-11T05:56:08.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-11T05:56:08.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-11T05:56:08.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-11T05:56:08.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-11T05:56:08.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-11T05:56:08.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-11T05:56:08.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-11T05:56:08.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-11T05:56:08.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-11T05:56:08.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-11T05:56:08.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-11T05:56:08.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-11T05:56:08.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-11T05:56:08.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-11T05:56:08.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-11T05:56:08.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-11T05:56:08.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-11T05:56:08.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-11T05:56:08.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-11T05:56:08.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-11T05:56:08.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-11T05:56:08.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-11T05:56:08.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-11T05:56:08.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-11T05:56:08.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-11T05:56:08.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-11T05:56:08.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-11T05:56:08.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-11T05:56:08.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-11T05:56:08.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-11T05:56:08.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-11T05:56:08.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-11T05:56:08.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-11T05:56:08.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-11T05:56:08.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-11T05:56:08.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-11T05:56:08.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-11T05:56:08.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-11T05:56:08.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-11T05:56:08.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-11T05:56:08.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-11T05:56:08.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-11T05:56:08.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-11T05:56:08.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-11T05:56:08.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-11T05:56:08.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-11T05:56:08.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-11T05:56:08.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-11T05:56:08.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-11T05:56:08.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-11T05:56:08.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-11T05:56:08.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-11T05:56:08.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-11T05:56:08.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-11T05:56:08.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-11T05:56:08.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-11T05:56:08.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-11T05:56:08.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-11T05:56:08.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-11T05:56:08.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-11T05:56:08.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-11T05:56:08.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-11T05:56:08.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-11T05:56:08.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-11T05:56:08.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-11T05:56:08.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-11T05:56:08.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-11T05:56:08.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-11T05:56:08.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-11T05:56:08.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-11T05:56:08.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-11T05:56:08.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-11T05:56:08.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-11T05:56:08.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-11T05:56:08.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-11T05:56:08.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-11T05:56:08.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-11T05:56:08.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-11T05:56:08.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-11T05:56:08.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-11T05:56:08.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-11T05:56:08.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-11T05:56:08.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-11T05:56:08.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-11T05:56:08.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-11T05:56:08.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-11T05:56:08.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-11T05:56:08.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-11T05:56:08.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-11T05:56:08.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-11T05:56:08.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-11T05:56:08.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-11T05:56:08.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-11T05:56:08.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-11T05:56:08.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-11T05:56:08.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-11T05:56:08.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-11T05:56:08.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-11T05:56:08.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-11T05:56:08.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-11T05:56:08.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-11T05:56:08.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-11T05:56:08.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-11T05:56:08.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-11T05:56:08.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-11T05:56:08.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-11T05:56:08.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-11T05:56:08.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-11T05:56:08.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-11T05:56:08.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-11T05:56:08.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-11T05:56:08.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-11T05:56:08.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-11T05:56:08.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-11T05:56:08.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-11T05:56:08.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-11T05:56:08.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-11T05:56:08.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-11T05:56:08.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-11T05:56:08.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-11T05:56:08.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-11T05:56:08.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-11T05:56:08.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-11T05:56:08.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-11T05:56:08.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-11T05:56:08.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-11T05:56:08.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-11T05:56:08.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-11T05:56:08.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-11T05:56:08.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-11T05:56:08.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-11T05:56:08.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-11T05:56:08.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-11T05:56:08.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-11T05:56:08.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-11T05:56:08.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-11T05:56:08.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-11T05:56:08.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-11T05:56:08.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-11T05:56:08.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-11T05:56:08.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-11T05:56:08.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-11T05:56:08.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-11T05:56:08.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-11T05:56:08.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-11T05:56:08.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-11T05:56:08.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-11T05:56:08.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-11T05:56:08.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-11T05:56:08.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-11T05:56:08.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-11T05:56:08.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-11T05:56:08.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-11T05:56:08.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-11T05:56:08.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-11T05:56:08.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-11T05:56:08.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-11T05:56:08.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-11T05:56:08.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-11T05:56:08.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-11T05:56:08.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-11T05:56:08.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-11T05:56:08.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-11T05:56:08.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-11T05:56:08.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-11T05:56:08.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-11T05:56:08.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-11T05:56:08.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-11T05:56:08.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-11T05:56:08.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-11T05:56:08.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-11T05:56:08.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-11T05:56:08.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-11T05:56:08.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-11T05:56:08.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-11T05:56:08.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-11T05:56:08.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-11T05:56:08.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-11T05:56:08.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-11T05:56:08.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-11T05:56:08.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-11T05:56:08.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-11T05:56:08.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-11T05:56:08.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-11T05:56:08.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-11T05:56:08.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-11T05:56:08.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-11T05:56:08.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-11T05:56:08.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-11T05:56:08.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-11T05:56:08.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-11T05:56:08.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-11T05:56:08.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-11T05:56:08.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-11T05:56:08.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-11T05:56:08.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-11T05:56:08.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-11T05:56:08.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-11T05:56:08.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-11T05:56:08.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-11T05:56:08.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-11T05:56:08.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-11T05:56:08.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-11T05:56:08.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-11T05:56:08.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-11T05:56:08.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-11T05:56:08.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-11T05:56:08.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-11T05:56:08.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-11T05:56:08.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-11T05:56:08.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-11T05:56:08.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-11T05:56:08.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-11T05:56:08.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-11T05:56:08.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-11T05:56:08.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-11T05:56:08.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-11T05:56:08.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-11T05:56:08.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-11T05:56:08.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-11T05:56:08.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-11T05:56:08.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-11T05:56:08.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-11T05:56:08.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-11T05:56:08.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-11T05:56:08.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-11T05:56:08.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-11T05:56:08.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-11T05:56:08.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-11T05:56:08.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-11T05:56:08.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-11T05:56:08.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-11T05:56:08.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-11T05:56:08.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-11T05:56:08.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-11T05:56:08.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-11T05:56:08.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-11T05:56:08.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-11T05:56:08.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-11T05:56:08.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-11T05:56:08.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-11T05:56:08.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-11T05:56:08.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-11T05:56:08.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-11T05:56:08.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-11T05:56:08.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-11T05:56:08.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-11T05:56:08.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-11T05:56:08.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-11T05:56:08.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-11T05:56:08.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-11T05:56:08.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-11T05:56:08.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-11T05:56:08.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-11T05:56:08.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-11T05:56:08.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-11T05:56:08.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-11T05:56:08.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-11T05:56:08.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-11T05:56:08.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-11T05:56:08.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-11T05:56:08.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-11T05:56:08.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-11T05:56:08.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-11T05:56:08.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-11T05:56:08.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-11T05:56:08.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-11T05:56:08.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-11T05:56:08.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-11T05:56:08.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-11T05:56:08.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-11T05:56:08.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-11T05:56:08.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-11T05:56:08.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-11T05:56:08.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-11T05:56:08.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-11T05:56:08.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-11T05:56:08.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-11T05:56:08.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-11T05:56:08.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-11T05:56:08.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-11T05:56:08.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-11T05:56:08.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-11T05:56:08.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-11T05:56:08.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-11T05:56:08.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-11T05:56:08.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-11T05:56:08.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-11T05:56:08.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-11T05:56:08.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-11T05:56:08.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-11T05:56:08.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-11T05:56:08.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-11T05:56:08.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-11T05:56:08.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-11T05:56:08.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-11T05:56:08.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-11T05:56:08.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-11T05:56:08.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-11T05:56:08.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-11T05:56:08.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-11T05:56:08.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-11T05:56:08.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-11T05:56:08.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-11T05:56:08.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-11T05:56:08.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-11T05:56:08.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-11T05:56:08.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-11T05:56:08.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-11T05:56:08.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-11T05:56:08.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-11T05:56:08.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-11T05:56:08.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-11T05:56:08.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-11T05:56:08.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-11T05:56:08.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-11T05:56:08.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-11T05:56:08.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-11T05:56:08.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-11T05:56:08.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-11T05:56:08.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-11T05:56:08.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-11T05:56:08.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-11T05:56:08.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-11T05:56:08.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-11T05:56:08.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-11T05:56:08.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-11T05:56:08.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-11T05:56:08.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-11T05:56:08.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-11T05:56:08.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-11T05:56:08.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-11T05:56:08.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-11T05:56:08.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-11T05:56:08.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-11T05:56:08.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-11T05:56:08.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-11T05:56:08.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-11T05:56:08.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-11T05:56:08.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-11T05:56:08.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-11T05:56:08.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-11T05:56:08.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-11T05:56:08.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-11T05:56:08.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-11T05:56:08.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-11T05:56:08.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-11T05:56:08.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-11T05:56:08.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-11T05:56:08.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-11T05:56:08.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-11T05:56:08.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-11T05:56:08.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-11T05:56:08.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-11T05:56:08.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-11T05:56:08.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-11T05:56:08.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-11T05:56:08.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-11T05:56:08.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-11T05:56:08.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-11T05:56:08.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-11T05:56:08.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-11T05:56:08.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-11T05:56:08.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-11T05:56:08.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-11T05:56:08.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-11T05:56:08.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-11T05:56:08.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-11T05:56:08.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-11T05:56:08.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-11T05:56:08.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-11T05:56:08.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-11T05:56:08.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-11T05:56:08.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-11T05:56:08.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-11T05:56:08.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-11T05:56:08.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-11T05:56:08.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-11T05:56:08.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-11T05:56:08.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-11T05:56:08.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-11T05:56:08.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-11T05:56:08.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-11T05:56:08.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-11T05:56:08.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-11T05:56:08.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-11T05:56:08.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-11T05:56:08.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-11T05:56:08.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-11T05:56:08.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-11T05:56:08.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-11T05:56:08.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-11T05:56:08.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-11T05:56:08.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-11T05:56:08.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-11T05:56:08.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-11T05:56:08.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-11T05:56:08.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-11T05:56:08.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-11T05:56:08.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-11T05:56:08.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-11T05:56:08.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-11T05:56:08.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-11T05:56:08.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-11T05:56:08.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-11T05:56:08.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-11T05:56:08.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-11T05:56:08.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-11T05:56:08.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-11T05:56:08.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-11T05:56:08.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-11T05:56:08.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-11T05:56:08.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-11T05:56:08.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-11T05:56:08.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-11T05:56:08.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-11T05:56:08.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-11T05:56:08.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-11T05:56:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-11T05:56:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-11T05:56:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-11T05:56:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-11T05:56:08.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-11T05:56:08.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-11T05:56:08.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-11T05:56:08.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-11T05:56:08.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-11T05:56:08.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-11T05:56:08.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-11T05:56:08.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-11T05:56:08.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-11T05:56:08.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-11T05:56:08.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-11T05:56:08.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-11T05:56:08.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-11T05:56:08.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-11T05:56:08.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-11T05:56:08.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-11T05:56:08.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-11T05:56:08.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-11T05:56:08.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-11T05:56:08.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-11T05:56:08.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-11T05:56:08.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-11T05:56:08.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-11T05:56:08.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-11T05:56:08.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-11T05:56:08.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-11T05:56:08.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-11T05:56:08.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-11T05:56:08.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-11T05:56:08.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-11T05:56:08.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-11T05:56:08.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-11T05:56:08.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-11T05:56:08.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-11T05:56:08.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-11T05:56:08.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-11T05:56:08.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-11T05:56:08.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-11T05:56:08.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-11T05:56:08.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-11T05:56:08.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-11T05:56:08.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-11T05:56:08.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-11T05:56:08.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-11T05:56:08.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-11T05:56:08.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-11T05:56:08.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-11T05:56:08.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-11T05:56:08.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-11T05:56:08.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-11T05:56:08.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-11T05:56:08.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-11T05:56:08.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-11T05:56:08.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-11T05:56:08.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-11T05:56:08.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-11T05:56:08.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-11T05:56:08.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-11T05:56:08.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-11T05:56:08.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-11T05:56:08.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-11T05:56:08.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-11T05:56:08.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-11T05:56:08.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-11T05:56:08.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-11T05:56:08.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-11T05:56:08.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-11T05:56:08.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-11T05:56:08.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-11T05:56:08.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-11T05:56:08.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-11T05:56:08.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-11T05:56:08.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-11T05:56:08.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-11T05:56:08.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-11T05:56:08.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-11T05:56:08.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-11T05:56:08.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-11T05:56:08.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-11T05:56:08.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-11T05:56:08.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-11T05:56:08.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-11T05:56:08.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-11T05:56:08.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-11T05:56:08.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-11T05:56:08.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-11T05:56:08.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-11T05:56:08.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-11T05:56:08.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-11T05:56:08.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-11T05:56:08.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-11T05:56:08.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-11T05:56:08.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-11T05:56:08.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-11T05:56:08.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-11T05:56:08.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-11T05:56:08.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-11T05:56:08.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-11T05:56:08.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-11T05:56:08.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-11T05:56:08.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-11T05:56:08.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-11T05:56:08.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-11T05:56:08.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-11T05:56:08.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-11T05:56:08.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-11T05:56:08.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-11T05:56:08.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-11T05:56:08.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-11T05:56:08.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-11T05:56:08.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-11T05:56:08.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-11T05:56:08.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-11T05:56:08.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-11T05:56:08.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-11T05:56:08.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-11T05:56:08.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-11T05:56:08.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-11T05:56:08.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-11T05:56:08.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-11T05:56:08.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-11T05:56:08.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-11T05:56:08.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-11T05:56:08.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-11T05:56:08.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-11T05:56:08.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-11T05:56:08.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-11T05:56:08.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-11T05:56:08.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-11T05:56:08.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-11T05:56:08.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-11T05:56:08.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-11T05:56:08.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-11T05:56:08.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-11T05:56:08.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-11T05:56:08.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-11T05:56:08.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-11T05:56:08.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-11T05:56:08.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-11T05:56:08.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-11T05:56:08.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-11T05:56:08.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-11T05:56:08.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-11T05:56:08.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-11T05:56:08.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-11T05:56:08.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-11T05:56:08.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-11T05:56:08.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-11T05:56:08.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-11T05:56:08.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-11T05:56:08.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-11T05:56:08.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-11T05:56:08.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-11T05:56:08.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-11T05:56:08.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-11T05:56:08.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-11T05:56:08.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-11T05:56:08.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-11T05:56:08.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-11T05:56:08.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-11T05:56:08.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-11T05:56:08.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-11T05:56:08.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-11T05:56:08.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-11T05:56:08.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-11T05:56:08.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-11T05:56:08.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-11T05:56:08.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-11T05:56:08.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-11T05:56:08.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-11T05:56:08.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-11T05:56:08.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-11T05:56:08.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-11T05:56:08.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-11T05:56:08.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-11T05:56:08.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-11T05:56:08.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-11T05:56:08.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-11T05:56:08.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-11T05:56:08.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-11T05:56:08.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-11T05:56:08.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-11T05:56:08.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-11T05:56:08.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-11T05:56:08.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-11T05:56:08.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-11T05:56:08.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-11T05:56:08.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-11T05:56:08.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-11T05:56:08.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-11T05:56:08.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-11T05:56:08.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-11T05:56:08.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-11T05:56:08.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-11T05:56:08.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-11T05:56:08.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-11T05:56:08.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-11T05:56:08.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-11T05:56:08.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-11T05:56:08.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-11T05:56:08.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-11T05:56:08.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-11T05:56:08.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-11T05:56:08.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-11T05:56:08.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-11T05:56:08.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-11T05:56:08.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-11T05:56:08.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-11T05:56:08.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-11T05:56:08.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-11T05:56:08.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-11T05:56:08.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-11T05:56:08.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-11T05:56:08.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-11T05:56:08.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-11T05:56:08.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-11T05:56:08.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-11T05:56:08.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-11T05:56:08.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-11T05:56:08.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-11T05:56:08.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-11T05:56:08.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-11T05:56:08.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-11T05:56:08.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-11T05:56:08.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-11T05:56:08.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-11T05:56:08.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-11T05:56:08.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-11T05:56:08.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-11T05:56:08.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-11T05:56:08.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-11T05:56:08.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-11T05:56:08.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-11T05:56:08.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-11T05:56:08.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-11T05:56:08.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-11T05:56:08.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-11T05:56:08.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-11T05:56:08.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-11T05:56:08.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-11T05:56:08.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-11T05:56:08.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-11T05:56:08.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-11T05:56:08.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-11T05:56:08.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-11T05:56:08.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-11T05:56:08.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-11T05:56:08.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-11T05:56:08.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-11T05:56:08.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-11T05:56:08.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-11T05:56:08.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-11T05:56:08.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-11T05:56:08.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-11T05:56:08.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-11T05:56:08.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-11T05:56:08.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-11T05:56:08.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-11T05:56:08.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-11T05:56:08.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-11T05:56:08.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-11T05:56:08.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-11T05:56:08.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-11T05:56:08.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-11T05:56:08.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-11T05:56:08.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-11T05:56:08.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-11T05:56:08.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-11T05:56:08.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-11T05:56:08.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-11T05:56:08.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-11T05:56:08.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-11T05:56:08.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-11T05:56:08.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-11T05:56:08.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-11T05:56:09.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-11T05:56:09.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-11T05:56:09.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-11T05:56:09.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-11T05:56:09.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-11T05:56:09.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-11T05:56:09.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-11T05:56:09.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-11T05:56:09.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-11T05:56:09.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-11T05:56:09.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-11T05:56:09.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-11T05:56:09.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-11T05:56:09.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-11T05:56:09.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-11T05:56:09.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-11T05:56:09.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-11T05:56:09.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-11T05:56:09.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-11T05:56:09.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-11T05:56:09.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-11T05:56:09.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-11T05:56:09.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-11T05:56:09.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-11T05:56:09.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-11T05:56:09.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-11T05:56:09.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-11T05:56:09.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-11T05:56:09.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-11T05:56:09.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-11T05:56:09.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-11T05:56:09.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-11T05:56:09.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-11T05:56:09.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-11T05:56:09.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-11T05:56:09.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-11T05:56:09.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-11T05:56:09.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-11T05:56:09.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-11T05:56:09.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-11T05:56:09.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-11T05:56:09.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-11T05:56:09.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-11T05:56:09.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-11T05:56:09.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-11T05:56:09.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-11T05:56:09.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-11T05:56:09.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-11T05:56:09.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-11T05:56:09.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-11T05:56:09.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-11T05:56:09.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-11T05:56:09.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-11T05:56:09.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-11T05:56:09.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-11T05:56:09.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-11T05:56:09.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-11T05:56:09.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-11T05:56:09.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-11T05:56:09.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-11T05:56:09.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-11T05:56:09.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-11T05:56:09.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-11T05:56:09.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-11T05:56:09.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-11T05:56:09.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-11T05:56:09.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-11T05:56:09.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-11T05:56:09.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-11T05:56:09.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-11T05:56:09.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-11T05:56:09.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-11T05:56:09.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-11T05:56:09.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-11T05:56:09.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-11T05:56:09.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-11T05:56:09.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-11T05:56:09.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-11T05:56:09.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-11T05:56:09.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-11T05:56:09.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-11T05:56:09.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-11T05:56:09.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-11T05:56:09.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-11T05:56:09.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-11T05:56:09.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-11T05:56:09.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-11T05:56:09.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-11T05:56:09.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-11T05:56:09.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-11T05:56:09.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-11T05:56:09.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-11T05:56:09.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-11T05:56:09.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-11T05:56:09.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-11T05:56:09.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-11T05:56:09.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-11T05:56:09.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-11T05:56:09.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-11T05:56:09.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-11T05:56:09.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-11T05:56:09.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-11T05:56:09.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-11T05:56:09.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-11T05:56:09.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-11T05:56:09.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-11T05:56:09.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-11T05:56:09.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-11T05:56:09.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-11T05:56:09.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-11T05:56:09.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-11T05:56:09.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-11T05:56:09.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-11T05:56:09.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-11T05:56:09.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-11T05:56:09.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-11T05:56:09.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-11T05:56:09.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-11T05:56:09.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-11T05:56:09.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-11T05:56:09.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-11T05:56:09.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-11T05:56:09.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-11T05:56:09.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-11T05:56:09.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-11T05:56:09.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-11T05:56:09.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-11T05:56:09.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-11T05:56:09.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-11T05:56:09.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-11T05:56:09.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-11T05:56:09.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-11T05:56:09.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-11T05:56:09.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-11T05:56:09.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-11T05:56:09.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-11T05:56:09.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-11T05:56:09.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-11T05:56:09.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-11T05:56:09.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-11T05:56:09.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-11T05:56:09.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-11T05:56:09.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-11T05:56:09.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-11T05:56:09.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-11T05:56:09.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-11T05:56:09.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-11T05:56:09.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-11T05:56:09.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-11T05:56:09.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-11T05:56:09.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-11T05:56:09.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-11T05:56:09.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-11T05:56:09.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-11T05:56:09.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-11T05:56:09.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-11T05:56:09.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-11T05:56:09.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-11T05:56:09.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-11T05:56:09.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-11T05:56:09.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-11T05:56:09.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-11T05:56:09.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-11T05:56:09.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-11T05:56:09.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-11T05:56:09.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-11T05:56:09.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-11T05:56:09.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-11T05:56:09.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-11T05:56:09.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-11T05:56:09.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-11T05:56:09.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-11T05:56:09.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-11T05:56:09.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-11T05:56:09.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-11T05:56:09.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-11T05:56:09.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-11T05:56:09.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-11T05:56:09.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-11T05:56:09.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-11T05:56:09.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-11T05:56:09.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-11T05:56:09.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-11T05:56:09.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-11T05:56:09.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-11T05:56:09.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-11T05:56:09.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-11T05:56:09.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-11T05:56:09.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-11T05:56:09.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-11T05:56:09.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-11T05:56:09.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-11T05:56:09.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-11T05:56:09.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-11T05:56:09.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-11T05:56:09.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-11T05:56:09.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-11T05:56:09.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-11T05:56:09.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-11T05:56:09.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-11T05:56:09.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-11T05:56:09.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-11T05:56:09.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-11T05:56:09.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-11T05:56:09.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-11T05:56:09.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-11T05:56:09.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-11T05:56:09.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-11T05:56:09.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-11T05:56:09.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-11T05:56:09.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-11T05:56:09.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-11T05:56:09.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-11T05:56:09.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-11T05:56:09.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-11T05:56:09.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-11T05:56:09.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-11T05:56:09.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-11T05:56:09.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-11T05:56:09.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-11T05:56:09.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-11T05:56:09.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-11T05:56:09.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-11T05:56:09.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-11T05:56:09.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-11T05:56:09.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-11T05:56:09.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-11T05:56:09.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-11T05:56:09.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-11T05:56:09.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-11T05:56:09.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-11T05:56:09.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-11T05:56:09.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-11T05:56:09.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-11T05:56:09.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-11T05:56:09.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-11T05:56:09.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-11T05:56:09.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-11T05:56:09.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-11T05:56:09.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-11T05:56:09.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-11T05:56:09.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-11T05:56:09.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-11T05:56:09.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-11T05:56:09.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-11T05:56:09.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-11T05:56:09.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-11T05:56:09.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-11T05:56:09.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-11T05:56:09.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-11T05:56:09.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-11T05:56:09.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-11T05:56:09.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-11T05:56:09.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-11T05:56:09.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-11T05:56:09.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-11T05:56:09.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-11T05:56:09.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-11T05:56:09.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-11T05:56:09.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-11T05:56:09.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-11T05:56:09.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-11T05:56:09.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-11T05:56:09.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-11T05:56:09.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-11T05:56:09.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-11T05:56:09.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-11T05:56:09.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-11T05:56:09.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-11T05:56:09.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-11T05:56:09.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-11T05:56:09.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-11T05:56:09.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-11T05:56:09.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-11T05:56:09.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-11T05:56:09.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-11T05:56:09.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-11T05:56:09.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-11T05:56:09.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-11T05:56:09.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-11T05:56:09.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-11T05:56:09.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-11T05:56:09.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-11T05:56:09.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-11T05:56:09.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-11T05:56:09.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-11T05:56:09.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-11T05:56:09.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-11T05:56:09.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-11T05:56:09.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-11T05:56:09.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-11T05:56:09.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-11T05:56:09.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-11T05:56:09.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-11T05:56:09.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-11T05:56:09.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-11T05:56:09.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-11T05:56:09.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-11T05:56:09.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-11T05:56:09.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-11T05:56:09.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-11T05:56:09.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-11T05:56:09.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-11T05:56:09.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-11T05:56:09.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-11T05:56:09.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-11T05:56:09.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-11T05:56:09.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-11T05:56:09.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-11T05:56:09.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-11T05:56:09.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-11T05:56:09.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-11T05:56:09.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-11T05:56:09.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-11T05:56:09.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-11T05:56:09.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-11T05:56:09.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-11T05:56:09.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-11T05:56:09.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-11T05:56:09.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-11T05:56:09.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-11T05:56:09.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-11T05:56:09.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-11T05:56:09.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-11T05:56:09.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-11T05:56:09.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-11T05:56:09.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-11T05:56:09.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-11T05:56:09.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-11T05:56:09.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-11T05:56:09.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-11T05:56:09.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-11T05:56:09.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-11T05:56:09.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-11T05:56:09.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-11T05:56:09.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-11T05:56:09.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-11T05:56:09.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-11T05:56:09.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-11T05:56:09.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-11T05:56:09.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-11T05:56:09.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-11T05:56:09.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-11T05:56:09.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-11T05:56:09.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-11T05:56:09.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-11T05:56:09.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-11T05:56:09.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-11T05:56:09.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-11T05:56:09.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-11T05:56:09.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-11T05:56:09.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-11T05:56:09.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-11T05:56:09.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-11T05:56:09.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-11T05:56:09.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-11T05:56:09.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-11T05:56:09.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-11T05:56:09.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-11T05:56:09.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-11T05:56:09.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-11T05:56:09.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-11T05:56:09.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-11T05:56:09.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-11T05:56:09.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-11T05:56:09.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-11T05:56:09.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-11T05:56:09.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-11T05:56:09.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-11T05:56:09.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-11T05:56:09.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-11T05:56:09.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-11T05:56:09.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-11T05:56:09.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-11T05:56:09.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-11T05:56:09.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-11T05:56:09.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-11T05:56:09.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-11T05:56:09.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-11T05:56:09.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-11T05:56:09.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-11T05:56:09.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-11T05:56:09.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-11T05:56:09.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-11T05:56:09.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-11T05:56:09.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-11T05:56:09.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-11T05:56:09.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-11T05:56:09.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-11T05:56:09.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-11T05:56:09.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-11T05:56:09.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-11T05:56:09.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-11T05:56:09.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-11T05:56:09.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-11T05:56:09.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-11T05:56:09.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-11T05:56:09.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-11T05:56:09.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-11T05:56:09.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-11T05:56:09.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-11T05:56:09.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-11T05:56:09.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-11T05:56:09.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-11T05:56:09.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-11T05:56:09.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-11T05:56:09.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-11T05:56:09.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-11T05:56:09.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-11T05:56:09.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-11T05:56:09.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-11T05:56:09.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-11T05:56:09.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-11T05:56:09.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-11T05:56:09.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-11T05:56:09.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-11T05:56:09.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-11T05:56:09.076 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-11T05:56:09.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-11T05:56:09.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-11T05:56:09.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-11T05:56:09.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-11T05:56:09.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-11T05:56:09.077 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-11T05:56:09.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-11T05:56:09.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-11T05:56:09.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-11T05:56:09.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-11T05:56:09.078 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-11T05:56:09.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-11T05:56:09.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-11T05:56:09.079 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-11T05:56:09.080 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:09.080 DEBUG:teuthology.orchestra.run.smithi084:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-11T05:56:09.104 INFO:teuthology.orchestra.run.smithi084.stdout:postinstall scriptlet (using /bin/sh): 2023-10-11T05:56:09.104 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-11T05:56:09.104 INFO:teuthology.orchestra.run.smithi084.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:09.104 INFO:teuthology.orchestra.run.smithi084.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:09.104 INFO:teuthology.orchestra.run.smithi084.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:09.105 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:09.105 INFO:teuthology.orchestra.run.smithi084.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:09.105 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-10-11T05:56:09.105 INFO:teuthology.orchestra.run.smithi084.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-11T05:56:09.105 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-11T05:56:09.106 INFO:teuthology.orchestra.run.smithi084.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-11T05:56:09.106 INFO:teuthology.orchestra.run.smithi084.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-11T05:56:09.106 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:09.106 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-10-11T05:56:09.106 INFO:teuthology.orchestra.run.smithi084.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-11T05:56:09.107 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-11T05:56:09.107 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:09.107 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-10-11T05:56:09.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2023-10-11T05:56:09.185 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2023-10-11T05:56:09.185 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2023-10-11T05:56:09.186 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.06 2023-10-11T05:56:09.186 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 68.el9 2023-10-11T05:56:09.186 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2023-10-11T05:56:09.187 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-11T05:56:09.187 INFO:teuthology.orchestra.run.smithi084.stdout:Group : Unspecified 2023-10-11T05:56:09.187 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 8264254 2023-10-11T05:56:09.187 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2023-10-11T05:56:09.187 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-11T05:56:09.187 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-11T05:56:09.188 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-11T05:56:09.188 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-11T05:56:09.188 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : builder@centos.org 2023-10-11T05:56:09.188 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : CentOS 2023-10-11T05:56:09.189 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-11T05:56:09.189 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2023-10-11T05:56:09.189 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2023-10-11T05:56:09.189 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T05:56:09.190 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-11T05:56:09.190 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-11T05:56:09.190 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-11T05:56:09.190 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2023-10-11T05:56:09.191 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T05:56:09.191 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2023-10-11T05:56:09.192 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-11T05:56:09.192 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:09.192 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-11T05:56:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2023-10-11T05:56:11.314 INFO:teuthology.orchestra.run.smithi084.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-11T05:56:11.351 INFO:teuthology.orchestra.run.smithi084.stderr:done 2023-10-11T05:56:11.354 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2023-10-11T05:56:11.377 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.735uQ6mrZh 2023-10-11T05:56:11.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.735uQ6mrZh 2023-10-11T05:56:11.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.735uQ6mrZh 2023-10-11T05:56:11.585 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.735uQ6mrZh is 6KB 2023-10-11T05:56:11.633 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.735uQ6mrZh 2023-10-11T05:56:11.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2023-10-11T05:56:11.720 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-11T05:56:11.720 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-11T05:56:11.721 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-11T05:56:11.721 INFO:teuthology.orchestra.run.smithi084.stdout:15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b.conf 2023-10-11T05:56:11.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:11.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2023-10-11T05:56:28.348 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-11T05:56:28.348 DEBUG:teuthology.orchestra.run.smithi099:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-11T05:56:28.413 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.414 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.414 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.414 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.414 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-11T05:56:28.414 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-11T05:56:28.415 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-11T05:56:28.415 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-11T05:56:28.415 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-11T05:56:28.415 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-11T05:56:28.415 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-11T05:56:28.416 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-11T05:56:28.416 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-11T05:56:28.416 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-11T05:56:28.416 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-11T05:56:28.416 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-11T05:56:28.417 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-11T05:56:28.417 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-11T05:56:28.417 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-11T05:56:28.417 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-11T05:56:28.417 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-11T05:56:28.418 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-11T05:56:28.418 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-11T05:56:28.418 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-11T05:56:28.418 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-11T05:56:28.418 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-11T05:56:28.419 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-11T05:56:28.419 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-11T05:56:28.419 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-11T05:56:28.419 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-11T05:56:28.419 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-11T05:56:28.420 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-11T05:56:28.420 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-11T05:56:28.420 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-11T05:56:28.420 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-11T05:56:28.420 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-11T05:56:28.421 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-11T05:56:28.421 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-11T05:56:28.421 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-11T05:56:28.421 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-11T05:56:28.421 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-11T05:56:28.422 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-11T05:56:28.422 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-11T05:56:28.422 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-11T05:56:28.422 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-11T05:56:28.422 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-11T05:56:28.423 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-11T05:56:28.423 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-11T05:56:28.423 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-11T05:56:28.423 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-11T05:56:28.423 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-11T05:56:28.424 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-11T05:56:28.424 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-11T05:56:28.424 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-11T05:56:28.424 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-11T05:56:28.424 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-11T05:56:28.425 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-11T05:56:28.425 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-11T05:56:28.425 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-11T05:56:28.425 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-11T05:56:28.425 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-11T05:56:28.426 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-11T05:56:28.426 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-11T05:56:28.426 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-11T05:56:28.426 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-11T05:56:28.426 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-11T05:56:28.427 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-11T05:56:28.427 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-11T05:56:28.427 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-11T05:56:28.427 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-11T05:56:28.427 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-11T05:56:28.427 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-11T05:56:28.428 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-11T05:56:28.428 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-11T05:56:28.428 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-11T05:56:28.428 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-11T05:56:28.428 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-11T05:56:28.429 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-11T05:56:28.429 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-11T05:56:28.429 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-11T05:56:28.429 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-11T05:56:28.429 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-11T05:56:28.430 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-11T05:56:28.430 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-11T05:56:28.430 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-11T05:56:28.430 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-11T05:56:28.431 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-11T05:56:28.431 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-11T05:56:28.431 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-11T05:56:28.431 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-11T05:56:28.431 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-11T05:56:28.432 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-11T05:56:28.432 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-11T05:56:28.432 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-11T05:56:28.432 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-11T05:56:28.432 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-11T05:56:28.432 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-11T05:56:28.433 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-11T05:56:28.433 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-11T05:56:28.433 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-11T05:56:28.433 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-11T05:56:28.433 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-11T05:56:28.434 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-11T05:56:28.434 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-11T05:56:28.434 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-11T05:56:28.434 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-11T05:56:28.434 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-11T05:56:28.435 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-11T05:56:28.435 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-11T05:56:28.435 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-11T05:56:28.435 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-11T05:56:28.435 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-11T05:56:28.436 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-11T05:56:28.436 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-11T05:56:28.436 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-11T05:56:28.436 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-11T05:56:28.436 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-11T05:56:28.437 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-11T05:56:28.437 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-11T05:56:28.437 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-11T05:56:28.437 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-11T05:56:28.437 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-11T05:56:28.438 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-11T05:56:28.438 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-11T05:56:28.438 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-11T05:56:28.438 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-11T05:56:28.438 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-11T05:56:28.439 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-11T05:56:28.439 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-11T05:56:28.439 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-11T05:56:28.439 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-11T05:56:28.439 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-11T05:56:28.439 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-11T05:56:28.440 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-11T05:56:28.440 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-11T05:56:28.440 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-11T05:56:28.440 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-11T05:56:28.440 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-11T05:56:28.441 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-11T05:56:28.441 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-11T05:56:28.441 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-11T05:56:28.441 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-11T05:56:28.441 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-11T05:56:28.442 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-11T05:56:28.442 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-11T05:56:28.442 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-11T05:56:28.442 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-11T05:56:28.442 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-11T05:56:28.443 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-11T05:56:28.443 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-11T05:56:28.443 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-11T05:56:28.443 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-11T05:56:28.443 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-11T05:56:28.444 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-11T05:56:28.444 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-11T05:56:28.444 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-11T05:56:28.444 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-11T05:56:28.444 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-11T05:56:28.445 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-11T05:56:28.445 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-11T05:56:28.445 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-11T05:56:28.445 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-11T05:56:28.445 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-11T05:56:28.446 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-11T05:56:28.446 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-11T05:56:28.446 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-11T05:56:28.446 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-11T05:56:28.446 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-11T05:56:28.446 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-11T05:56:28.447 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-11T05:56:28.447 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-11T05:56:28.447 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-11T05:56:28.447 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-11T05:56:28.447 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-11T05:56:28.448 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-11T05:56:28.448 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-11T05:56:28.448 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-11T05:56:28.448 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-11T05:56:28.448 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-11T05:56:28.449 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-11T05:56:28.449 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-11T05:56:28.449 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-11T05:56:28.449 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-11T05:56:28.449 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-11T05:56:28.450 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-11T05:56:28.450 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-11T05:56:28.450 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-11T05:56:28.450 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-11T05:56:28.450 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-11T05:56:28.451 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-11T05:56:28.451 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-11T05:56:28.451 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-11T05:56:28.451 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-11T05:56:28.451 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-11T05:56:28.452 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-11T05:56:28.452 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-11T05:56:28.452 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-11T05:56:28.452 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-11T05:56:28.452 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-11T05:56:28.452 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-11T05:56:28.453 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-11T05:56:28.453 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-11T05:56:28.453 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-11T05:56:28.453 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-11T05:56:28.454 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-11T05:56:28.454 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-11T05:56:28.454 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-11T05:56:28.454 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-11T05:56:28.454 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-11T05:56:28.455 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-11T05:56:28.455 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-11T05:56:28.455 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-11T05:56:28.455 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-11T05:56:28.455 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-11T05:56:28.456 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-11T05:56:28.456 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-11T05:56:28.456 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-11T05:56:28.456 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-11T05:56:28.456 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-11T05:56:28.457 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-11T05:56:28.457 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-11T05:56:28.457 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-11T05:56:28.457 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-11T05:56:28.457 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-11T05:56:28.457 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-11T05:56:28.458 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-11T05:56:28.458 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-11T05:56:28.458 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-11T05:56:28.458 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-11T05:56:28.458 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-11T05:56:28.459 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-11T05:56:28.459 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-11T05:56:28.459 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-11T05:56:28.459 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-11T05:56:28.459 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-11T05:56:28.460 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-11T05:56:28.460 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-11T05:56:28.460 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-11T05:56:28.460 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-11T05:56:28.460 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-11T05:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-11T05:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-11T05:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-11T05:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-11T05:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-11T05:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-11T05:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-11T05:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-11T05:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-11T05:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-11T05:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-11T05:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-11T05:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-11T05:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-11T05:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-11T05:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-11T05:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-11T05:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-11T05:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-11T05:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-11T05:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-11T05:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-11T05:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-11T05:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-11T05:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-11T05:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-11T05:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-11T05:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-11T05:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-11T05:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-11T05:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-11T05:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-11T05:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-11T05:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-11T05:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-11T05:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-11T05:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-11T05:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-11T05:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-11T05:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-11T05:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-11T05:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-11T05:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-11T05:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-11T05:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-11T05:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-11T05:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-11T05:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-11T05:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-11T05:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-11T05:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-11T05:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-11T05:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-11T05:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-11T05:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-11T05:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-11T05:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-11T05:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-11T05:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-11T05:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-11T05:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-11T05:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-11T05:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-11T05:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-11T05:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-11T05:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-11T05:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-11T05:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-11T05:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-11T05:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-11T05:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-11T05:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-11T05:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-11T05:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-11T05:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-11T05:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-11T05:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-11T05:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-11T05:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-11T05:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-11T05:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-11T05:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-11T05:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-11T05:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-11T05:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-11T05:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-11T05:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-11T05:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-11T05:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-11T05:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-11T05:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-11T05:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-11T05:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-11T05:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-11T05:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-11T05:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-11T05:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-11T05:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-11T05:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-11T05:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-11T05:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-11T05:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-11T05:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-11T05:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-11T05:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-11T05:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-11T05:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-11T05:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-11T05:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-11T05:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-11T05:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-11T05:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-11T05:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-11T05:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-11T05:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-11T05:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-11T05:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-11T05:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-11T05:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-11T05:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-11T05:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-11T05:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-11T05:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-11T05:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-11T05:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-11T05:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-11T05:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-11T05:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-11T05:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-11T05:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-11T05:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-11T05:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-11T05:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-11T05:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-11T05:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-11T05:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-11T05:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-11T05:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-11T05:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-11T05:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-11T05:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-11T05:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-11T05:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-11T05:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-11T05:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-11T05:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-11T05:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-11T05:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-11T05:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-11T05:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-11T05:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-11T05:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-11T05:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-11T05:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-11T05:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-11T05:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-11T05:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-11T05:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-11T05:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-11T05:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-11T05:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-11T05:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-11T05:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-11T05:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-11T05:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-11T05:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-11T05:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-11T05:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-11T05:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-11T05:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-11T05:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-11T05:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-11T05:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-11T05:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-11T05:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-11T05:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-11T05:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-11T05:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-11T05:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-11T05:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-11T05:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-11T05:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-11T05:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-11T05:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-11T05:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-11T05:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-11T05:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-11T05:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-11T05:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-11T05:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-11T05:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-11T05:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-11T05:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-11T05:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-11T05:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-11T05:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-11T05:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-11T05:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-11T05:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-11T05:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-11T05:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-11T05:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-11T05:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-11T05:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-11T05:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-11T05:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-11T05:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-11T05:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-11T05:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-11T05:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-11T05:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-11T05:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-11T05:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-11T05:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-11T05:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-11T05:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-11T05:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-11T05:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-11T05:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-11T05:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-11T05:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-11T05:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-11T05:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-11T05:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-11T05:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-11T05:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-11T05:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-11T05:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-11T05:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-11T05:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-11T05:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-11T05:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-11T05:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-11T05:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-11T05:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-11T05:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-11T05:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-11T05:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-11T05:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-11T05:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-11T05:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-11T05:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-11T05:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-11T05:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-11T05:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-11T05:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-11T05:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-11T05:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-11T05:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-11T05:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-11T05:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-11T05:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-11T05:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-11T05:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-11T05:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-11T05:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-11T05:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-11T05:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-11T05:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-11T05:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-11T05:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-11T05:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-11T05:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-11T05:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-11T05:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-11T05:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-11T05:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-11T05:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-11T05:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-11T05:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-11T05:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-11T05:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-11T05:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-11T05:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-11T05:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-11T05:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-11T05:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-11T05:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-11T05:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-11T05:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-11T05:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-11T05:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-11T05:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-11T05:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-11T05:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-11T05:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-11T05:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-11T05:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-11T05:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-11T05:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-11T05:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-11T05:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-11T05:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-11T05:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-11T05:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-11T05:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-11T05:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-11T05:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-11T05:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-11T05:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-11T05:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-11T05:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-11T05:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-11T05:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-11T05:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-11T05:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-11T05:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-11T05:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-11T05:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-11T05:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-11T05:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-11T05:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-11T05:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-11T05:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-11T05:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-11T05:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-11T05:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-11T05:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-11T05:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-11T05:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-11T05:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-11T05:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-11T05:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-11T05:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-11T05:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-11T05:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-11T05:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-11T05:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-11T05:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-11T05:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-11T05:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-11T05:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-11T05:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-11T05:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-11T05:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-11T05:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-11T05:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-11T05:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-11T05:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-11T05:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-11T05:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-11T05:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-11T05:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-11T05:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-11T05:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-11T05:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-11T05:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-11T05:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-11T05:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-11T05:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-11T05:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-11T05:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-11T05:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-11T05:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-11T05:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-11T05:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-11T05:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-11T05:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-11T05:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-11T05:56:28.530 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-11T05:56:28.530 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-11T05:56:28.530 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-11T05:56:28.530 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-11T05:56:28.530 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-11T05:56:28.531 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-11T05:56:28.531 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-11T05:56:28.531 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-11T05:56:28.531 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-11T05:56:28.531 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-11T05:56:28.531 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-11T05:56:28.532 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-11T05:56:28.532 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-11T05:56:28.532 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-11T05:56:28.532 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-11T05:56:28.532 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-11T05:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-11T05:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-11T05:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-11T05:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-11T05:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-11T05:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-11T05:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-11T05:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-11T05:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-11T05:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-11T05:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-11T05:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-11T05:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-11T05:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-11T05:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-11T05:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-11T05:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-11T05:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-11T05:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-11T05:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-11T05:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-11T05:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-11T05:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-11T05:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-11T05:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-11T05:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-11T05:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-11T05:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-11T05:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-11T05:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-11T05:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-11T05:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-11T05:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-11T05:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-11T05:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-11T05:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-11T05:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-11T05:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-11T05:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-11T05:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-11T05:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-11T05:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-11T05:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-11T05:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-11T05:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-11T05:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-11T05:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-11T05:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-11T05:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-11T05:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-11T05:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-11T05:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-11T05:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-11T05:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-11T05:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-11T05:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-11T05:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-11T05:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-11T05:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-11T05:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-11T05:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-11T05:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-11T05:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-11T05:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-11T05:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-11T05:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-11T05:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-11T05:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-11T05:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-11T05:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-11T05:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-11T05:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-11T05:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-11T05:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-11T05:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-11T05:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-11T05:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-11T05:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-11T05:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-11T05:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-11T05:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-11T05:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-11T05:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-11T05:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-11T05:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-11T05:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-11T05:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-11T05:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-11T05:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-11T05:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-11T05:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-11T05:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-11T05:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-11T05:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-11T05:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-11T05:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-11T05:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-11T05:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-11T05:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-11T05:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-11T05:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-11T05:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-11T05:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-11T05:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-11T05:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-11T05:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-11T05:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-11T05:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-11T05:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-11T05:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-11T05:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-11T05:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-11T05:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-11T05:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-11T05:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-11T05:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-11T05:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-11T05:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-11T05:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-11T05:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-11T05:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-11T05:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-11T05:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-11T05:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-11T05:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-11T05:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-11T05:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-11T05:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-11T05:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-11T05:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-11T05:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-11T05:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-11T05:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-11T05:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-11T05:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-11T05:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-11T05:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-11T05:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-11T05:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-11T05:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-11T05:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-11T05:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-11T05:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-11T05:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-11T05:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-11T05:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-11T05:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-11T05:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-11T05:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-11T05:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-11T05:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-11T05:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-11T05:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-11T05:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-11T05:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-11T05:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-11T05:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-11T05:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-11T05:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-11T05:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-11T05:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-11T05:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-11T05:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-11T05:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-11T05:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-11T05:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-11T05:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-11T05:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-11T05:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-11T05:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-11T05:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-11T05:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-11T05:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-11T05:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-11T05:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-11T05:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-11T05:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-11T05:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-11T05:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-11T05:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-11T05:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-11T05:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-11T05:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-11T05:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-11T05:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-11T05:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-11T05:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-11T05:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-11T05:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-11T05:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-11T05:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-11T05:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-11T05:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-11T05:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-11T05:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-11T05:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-11T05:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-11T05:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-11T05:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-11T05:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-11T05:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-11T05:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-11T05:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-11T05:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-11T05:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-11T05:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-11T05:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-11T05:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-11T05:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-11T05:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-11T05:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-11T05:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-11T05:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-11T05:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-11T05:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-11T05:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-11T05:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-11T05:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-11T05:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-11T05:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-11T05:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-11T05:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-11T05:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-11T05:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-11T05:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-11T05:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-11T05:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-11T05:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-11T05:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-11T05:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-11T05:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-11T05:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-11T05:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-11T05:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-11T05:56:28.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-11T05:56:28.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-11T05:56:28.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-11T05:56:28.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-11T05:56:28.584 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-11T05:56:28.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-11T05:56:28.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-11T05:56:28.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-11T05:56:28.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-11T05:56:28.585 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-11T05:56:28.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-11T05:56:28.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-11T05:56:28.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-11T05:56:28.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-11T05:56:28.586 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-11T05:56:28.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-11T05:56:28.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-11T05:56:28.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-11T05:56:28.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-11T05:56:28.587 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-11T05:56:28.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-11T05:56:28.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-11T05:56:28.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-11T05:56:28.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-11T05:56:28.588 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-11T05:56:28.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-11T05:56:28.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-11T05:56:28.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-11T05:56:28.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-11T05:56:28.589 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-11T05:56:28.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-11T05:56:28.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-11T05:56:28.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-11T05:56:28.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-11T05:56:28.590 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-11T05:56:28.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-11T05:56:28.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-11T05:56:28.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-11T05:56:28.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-11T05:56:28.591 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-11T05:56:28.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-11T05:56:28.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-11T05:56:28.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-11T05:56:28.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-11T05:56:28.592 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-11T05:56:28.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-11T05:56:28.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-11T05:56:28.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-11T05:56:28.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-11T05:56:28.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-11T05:56:28.593 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-11T05:56:28.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-11T05:56:28.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-11T05:56:28.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-11T05:56:28.594 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-11T05:56:28.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-11T05:56:28.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-11T05:56:28.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-11T05:56:28.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-11T05:56:28.595 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-11T05:56:28.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-11T05:56:28.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-11T05:56:28.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-11T05:56:28.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-11T05:56:28.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-11T05:56:28.596 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-11T05:56:28.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-11T05:56:28.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-11T05:56:28.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-11T05:56:28.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-11T05:56:28.597 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-11T05:56:28.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-11T05:56:28.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-11T05:56:28.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-11T05:56:28.598 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-11T05:56:28.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-11T05:56:28.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-11T05:56:28.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-11T05:56:28.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-11T05:56:28.599 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-11T05:56:28.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-11T05:56:28.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-11T05:56:28.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-11T05:56:28.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-11T05:56:28.600 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-11T05:56:28.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-11T05:56:28.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-11T05:56:28.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-11T05:56:28.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-11T05:56:28.601 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-11T05:56:28.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-11T05:56:28.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-11T05:56:28.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-11T05:56:28.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-11T05:56:28.602 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-11T05:56:28.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-11T05:56:28.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-11T05:56:28.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-11T05:56:28.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-11T05:56:28.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-11T05:56:28.603 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-11T05:56:28.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-11T05:56:28.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-11T05:56:28.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-11T05:56:28.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-11T05:56:28.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-11T05:56:28.604 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-11T05:56:28.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-11T05:56:28.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-11T05:56:28.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-11T05:56:28.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-11T05:56:28.605 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-11T05:56:28.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-11T05:56:28.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-11T05:56:28.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-11T05:56:28.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-11T05:56:28.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-11T05:56:28.606 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-11T05:56:28.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-11T05:56:28.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-11T05:56:28.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-11T05:56:28.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-11T05:56:28.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-11T05:56:28.607 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-11T05:56:28.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-11T05:56:28.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-11T05:56:28.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-11T05:56:28.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-11T05:56:28.608 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-11T05:56:28.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-11T05:56:28.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-11T05:56:28.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-11T05:56:28.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-11T05:56:28.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-11T05:56:28.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-11T05:56:28.609 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-11T05:56:28.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-11T05:56:28.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-11T05:56:28.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-11T05:56:28.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-11T05:56:28.610 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-11T05:56:28.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-11T05:56:28.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-11T05:56:28.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-11T05:56:28.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-11T05:56:28.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-11T05:56:28.611 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-11T05:56:28.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-11T05:56:28.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-11T05:56:28.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-11T05:56:28.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-11T05:56:28.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-11T05:56:28.612 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-11T05:56:28.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-11T05:56:28.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-11T05:56:28.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-11T05:56:28.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-11T05:56:28.613 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-11T05:56:28.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-11T05:56:28.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-11T05:56:28.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-11T05:56:28.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-11T05:56:28.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-11T05:56:28.614 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-11T05:56:28.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-11T05:56:28.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-11T05:56:28.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-11T05:56:28.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-11T05:56:28.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-11T05:56:28.615 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-11T05:56:28.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-11T05:56:28.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-11T05:56:28.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-11T05:56:28.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-11T05:56:28.616 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-11T05:56:28.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-11T05:56:28.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-11T05:56:28.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-11T05:56:28.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-11T05:56:28.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-11T05:56:28.617 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-11T05:56:28.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-11T05:56:28.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-11T05:56:28.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-11T05:56:28.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-11T05:56:28.618 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-11T05:56:28.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-11T05:56:28.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-11T05:56:28.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-11T05:56:28.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-11T05:56:28.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-11T05:56:28.619 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-11T05:56:28.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-11T05:56:28.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-11T05:56:28.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-11T05:56:28.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-11T05:56:28.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-11T05:56:28.620 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-11T05:56:28.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-11T05:56:28.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-11T05:56:28.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-11T05:56:28.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-11T05:56:28.621 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-11T05:56:28.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-11T05:56:28.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-11T05:56:28.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-11T05:56:28.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-11T05:56:28.622 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-11T05:56:28.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-11T05:56:28.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-11T05:56:28.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-11T05:56:28.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-11T05:56:28.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-11T05:56:28.623 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-11T05:56:28.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-11T05:56:28.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-11T05:56:28.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-11T05:56:28.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-11T05:56:28.624 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-11T05:56:28.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-11T05:56:28.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-11T05:56:28.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-11T05:56:28.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-11T05:56:28.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-11T05:56:28.625 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-11T05:56:28.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-11T05:56:28.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-11T05:56:28.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-11T05:56:28.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-11T05:56:28.626 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-11T05:56:28.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-11T05:56:28.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-11T05:56:28.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-11T05:56:28.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-11T05:56:28.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-11T05:56:28.627 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-11T05:56:28.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-11T05:56:28.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-11T05:56:28.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-11T05:56:28.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-11T05:56:28.628 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-11T05:56:28.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-11T05:56:28.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-11T05:56:28.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-11T05:56:28.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-11T05:56:28.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-11T05:56:28.629 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-11T05:56:28.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-11T05:56:28.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-11T05:56:28.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-11T05:56:28.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-11T05:56:28.630 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-11T05:56:28.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-11T05:56:28.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-11T05:56:28.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-11T05:56:28.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-11T05:56:28.631 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-11T05:56:28.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-11T05:56:28.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-11T05:56:28.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-11T05:56:28.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-11T05:56:28.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-11T05:56:28.632 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-11T05:56:28.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-11T05:56:28.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-11T05:56:28.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-11T05:56:28.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-11T05:56:28.633 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-11T05:56:28.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-11T05:56:28.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-11T05:56:28.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-11T05:56:28.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-11T05:56:28.634 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-11T05:56:28.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-11T05:56:28.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-11T05:56:28.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-11T05:56:28.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-11T05:56:28.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-11T05:56:28.635 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-11T05:56:28.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-11T05:56:28.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-11T05:56:28.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-11T05:56:28.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-11T05:56:28.636 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-11T05:56:28.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-11T05:56:28.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-11T05:56:28.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-11T05:56:28.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-11T05:56:28.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-11T05:56:28.637 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-11T05:56:28.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-11T05:56:28.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-11T05:56:28.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-11T05:56:28.638 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-11T05:56:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-11T05:56:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-11T05:56:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-11T05:56:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-11T05:56:28.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-11T05:56:28.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-11T05:56:28.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-11T05:56:28.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-11T05:56:28.640 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-11T05:56:28.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-11T05:56:28.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-11T05:56:28.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-11T05:56:28.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-11T05:56:28.641 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-11T05:56:28.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-11T05:56:28.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-11T05:56:28.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-11T05:56:28.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-11T05:56:28.642 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-11T05:56:28.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-11T05:56:28.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-11T05:56:28.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-11T05:56:28.643 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-11T05:56:28.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-11T05:56:28.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-11T05:56:28.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-11T05:56:28.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-11T05:56:28.644 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-11T05:56:28.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-11T05:56:28.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-11T05:56:28.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-11T05:56:28.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-11T05:56:28.645 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-11T05:56:28.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-11T05:56:28.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-11T05:56:28.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-11T05:56:28.646 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-11T05:56:28.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-11T05:56:28.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-11T05:56:28.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-11T05:56:28.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-11T05:56:28.647 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-11T05:56:28.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-11T05:56:28.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-11T05:56:28.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-11T05:56:28.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-11T05:56:28.648 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-11T05:56:28.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-11T05:56:28.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-11T05:56:28.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-11T05:56:28.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-11T05:56:28.649 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-11T05:56:28.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-11T05:56:28.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-11T05:56:28.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-11T05:56:28.650 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-11T05:56:28.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-11T05:56:28.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-11T05:56:28.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-11T05:56:28.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-11T05:56:28.651 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-11T05:56:28.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-11T05:56:28.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-11T05:56:28.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-11T05:56:28.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-11T05:56:28.652 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-11T05:56:28.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-11T05:56:28.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-11T05:56:28.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-11T05:56:28.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-11T05:56:28.653 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-11T05:56:28.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-11T05:56:28.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-11T05:56:28.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-11T05:56:28.654 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-11T05:56:28.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-11T05:56:28.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-11T05:56:28.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-11T05:56:28.655 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-11T05:56:28.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-11T05:56:28.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-11T05:56:28.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-11T05:56:28.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-11T05:56:28.656 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-11T05:56:28.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-11T05:56:28.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-11T05:56:28.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-11T05:56:28.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-11T05:56:28.657 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-11T05:56:28.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-11T05:56:28.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-11T05:56:28.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-11T05:56:28.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-11T05:56:28.658 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-11T05:56:28.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-11T05:56:28.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-11T05:56:28.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-11T05:56:28.659 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-11T05:56:28.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-11T05:56:28.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-11T05:56:28.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-11T05:56:28.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-11T05:56:28.660 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-11T05:56:28.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-11T05:56:28.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-11T05:56:28.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-11T05:56:28.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-11T05:56:28.661 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-11T05:56:28.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-11T05:56:28.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-11T05:56:28.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-11T05:56:28.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-11T05:56:28.662 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-11T05:56:28.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-11T05:56:28.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-11T05:56:28.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-11T05:56:28.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-11T05:56:28.663 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-11T05:56:28.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-11T05:56:28.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-11T05:56:28.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-11T05:56:28.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-11T05:56:28.664 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-11T05:56:28.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-11T05:56:28.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-11T05:56:28.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-11T05:56:28.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-11T05:56:28.665 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-11T05:56:28.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-11T05:56:28.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-11T05:56:28.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-11T05:56:28.666 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-11T05:56:28.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-11T05:56:28.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-11T05:56:28.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-11T05:56:28.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-11T05:56:28.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-11T05:56:28.667 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-11T05:56:28.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-11T05:56:28.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-11T05:56:28.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-11T05:56:28.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-11T05:56:28.668 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-11T05:56:28.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-11T05:56:28.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-11T05:56:28.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-11T05:56:28.669 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-11T05:56:28.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-11T05:56:28.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-11T05:56:28.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-11T05:56:28.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-11T05:56:28.670 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-11T05:56:28.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-11T05:56:28.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-11T05:56:28.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-11T05:56:28.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-11T05:56:28.671 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-11T05:56:28.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-11T05:56:28.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-11T05:56:28.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-11T05:56:28.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-11T05:56:28.672 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-11T05:56:28.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-11T05:56:28.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-11T05:56:28.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-11T05:56:28.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-11T05:56:28.673 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-11T05:56:28.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-11T05:56:28.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-11T05:56:28.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-11T05:56:28.674 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-11T05:56:28.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-11T05:56:28.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-11T05:56:28.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-11T05:56:28.675 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-11T05:56:28.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-11T05:56:28.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-11T05:56:28.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-11T05:56:28.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-11T05:56:28.676 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-11T05:56:28.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-11T05:56:28.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-11T05:56:28.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-11T05:56:28.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-11T05:56:28.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-11T05:56:28.677 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-11T05:56:28.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-11T05:56:28.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-11T05:56:28.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-11T05:56:28.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-11T05:56:28.678 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-11T05:56:28.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-11T05:56:28.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-11T05:56:28.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-11T05:56:28.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-11T05:56:28.679 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-11T05:56:28.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-11T05:56:28.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-11T05:56:28.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-11T05:56:28.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-11T05:56:28.680 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-11T05:56:28.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-11T05:56:28.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-11T05:56:28.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-11T05:56:28.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-11T05:56:28.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-11T05:56:28.681 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-11T05:56:28.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-11T05:56:28.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-11T05:56:28.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-11T05:56:28.682 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-11T05:56:28.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-11T05:56:28.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-11T05:56:28.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-11T05:56:28.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-11T05:56:28.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-11T05:56:28.683 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-11T05:56:28.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-11T05:56:28.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-11T05:56:28.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-11T05:56:28.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-11T05:56:28.684 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-11T05:56:28.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-11T05:56:28.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-11T05:56:28.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-11T05:56:28.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-11T05:56:28.685 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-11T05:56:28.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-11T05:56:28.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-11T05:56:28.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-11T05:56:28.686 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-11T05:56:28.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-11T05:56:28.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-11T05:56:28.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-11T05:56:28.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-11T05:56:28.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-11T05:56:28.687 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-11T05:56:28.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-11T05:56:28.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-11T05:56:28.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-11T05:56:28.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-11T05:56:28.688 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-11T05:56:28.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-11T05:56:28.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-11T05:56:28.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-11T05:56:28.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-11T05:56:28.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-11T05:56:28.689 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-11T05:56:28.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-11T05:56:28.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-11T05:56:28.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-11T05:56:28.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-11T05:56:28.690 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-11T05:56:28.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-11T05:56:28.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-11T05:56:28.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-11T05:56:28.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-11T05:56:28.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-11T05:56:28.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-11T05:56:28.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-11T05:56:28.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-11T05:56:28.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-11T05:56:28.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-11T05:56:28.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-11T05:56:28.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-11T05:56:28.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-11T05:56:28.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-11T05:56:28.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-11T05:56:28.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-11T05:56:28.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-11T05:56:28.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-11T05:56:28.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-11T05:56:28.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-11T05:56:28.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-11T05:56:28.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-11T05:56:28.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-11T05:56:28.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-11T05:56:28.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-11T05:56:28.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-11T05:56:28.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-11T05:56:28.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-11T05:56:28.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-11T05:56:28.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-11T05:56:28.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-11T05:56:28.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-11T05:56:28.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-11T05:56:28.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-11T05:56:28.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-11T05:56:28.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-11T05:56:28.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-11T05:56:28.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-11T05:56:28.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-11T05:56:28.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-11T05:56:28.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-11T05:56:28.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-11T05:56:28.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-11T05:56:28.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-11T05:56:28.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-11T05:56:28.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-11T05:56:28.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-11T05:56:28.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-11T05:56:28.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-11T05:56:28.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-11T05:56:28.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-11T05:56:28.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-11T05:56:28.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-11T05:56:28.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-11T05:56:28.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-11T05:56:28.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-11T05:56:28.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-11T05:56:28.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-11T05:56:28.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-11T05:56:28.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-11T05:56:28.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-11T05:56:28.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-11T05:56:28.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-11T05:56:28.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-11T05:56:28.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-11T05:56:28.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-11T05:56:28.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-11T05:56:28.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-11T05:56:28.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-11T05:56:28.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-11T05:56:28.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-11T05:56:28.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-11T05:56:28.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-11T05:56:28.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-11T05:56:28.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-11T05:56:28.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-11T05:56:28.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-11T05:56:28.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-11T05:56:28.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-11T05:56:28.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-11T05:56:28.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-11T05:56:28.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-11T05:56:28.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-11T05:56:28.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-11T05:56:28.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-11T05:56:28.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-11T05:56:28.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-11T05:56:28.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-11T05:56:28.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-11T05:56:28.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-11T05:56:28.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-11T05:56:28.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-11T05:56:28.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-11T05:56:28.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-11T05:56:28.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-11T05:56:28.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-11T05:56:28.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-11T05:56:28.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-11T05:56:28.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-11T05:56:28.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-11T05:56:28.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-11T05:56:28.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-11T05:56:28.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-11T05:56:28.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-11T05:56:28.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-11T05:56:28.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-11T05:56:28.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-11T05:56:28.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-11T05:56:28.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-11T05:56:28.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-11T05:56:28.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-11T05:56:28.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-11T05:56:28.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-11T05:56:28.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-11T05:56:28.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-11T05:56:28.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-11T05:56:28.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-11T05:56:28.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-11T05:56:28.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-11T05:56:28.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-11T05:56:28.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-11T05:56:28.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-11T05:56:28.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-11T05:56:28.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-11T05:56:28.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-11T05:56:28.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-11T05:56:28.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-11T05:56:28.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-11T05:56:28.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-11T05:56:28.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-11T05:56:28.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-11T05:56:28.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-11T05:56:28.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-11T05:56:28.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-11T05:56:28.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-11T05:56:28.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-11T05:56:28.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-11T05:56:28.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-11T05:56:28.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-11T05:56:28.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-11T05:56:28.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-11T05:56:28.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-11T05:56:28.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-11T05:56:28.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-11T05:56:28.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-11T05:56:28.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-11T05:56:28.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-11T05:56:28.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-11T05:56:28.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-11T05:56:28.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-11T05:56:28.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-11T05:56:28.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-11T05:56:28.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-11T05:56:28.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-11T05:56:28.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-11T05:56:28.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-11T05:56:28.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-11T05:56:28.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-11T05:56:28.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-11T05:56:28.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-11T05:56:28.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-11T05:56:28.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-11T05:56:28.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-11T05:56:28.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-11T05:56:28.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-11T05:56:28.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-11T05:56:28.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-11T05:56:28.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-11T05:56:28.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-11T05:56:28.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-11T05:56:28.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-11T05:56:28.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-11T05:56:28.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-11T05:56:28.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-11T05:56:28.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-11T05:56:28.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-11T05:56:28.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-11T05:56:28.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-11T05:56:28.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-11T05:56:28.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-11T05:56:28.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-11T05:56:28.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-11T05:56:28.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-11T05:56:28.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-11T05:56:28.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-11T05:56:28.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-11T05:56:28.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-11T05:56:28.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-11T05:56:28.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-11T05:56:28.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-11T05:56:28.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-11T05:56:28.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-11T05:56:28.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-11T05:56:28.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-11T05:56:28.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-11T05:56:28.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-11T05:56:28.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-11T05:56:28.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-11T05:56:28.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-11T05:56:28.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-11T05:56:28.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-11T05:56:28.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-11T05:56:28.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-11T05:56:28.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-11T05:56:28.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-11T05:56:28.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-11T05:56:28.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-11T05:56:28.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-11T05:56:28.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-11T05:56:28.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-11T05:56:28.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-11T05:56:28.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-11T05:56:28.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-11T05:56:28.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-11T05:56:28.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-11T05:56:28.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-11T05:56:28.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-11T05:56:28.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-11T05:56:28.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-11T05:56:28.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-11T05:56:28.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-11T05:56:28.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-11T05:56:28.736 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-11T05:56:28.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-11T05:56:28.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-11T05:56:28.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-11T05:56:28.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-11T05:56:28.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-11T05:56:28.737 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-11T05:56:28.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-11T05:56:28.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-11T05:56:28.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-11T05:56:28.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-11T05:56:28.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-11T05:56:28.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-11T05:56:28.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-11T05:56:28.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-11T05:56:28.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-11T05:56:28.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-11T05:56:28.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-11T05:56:28.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-11T05:56:28.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-11T05:56:28.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-11T05:56:28.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-11T05:56:28.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-11T05:56:28.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-11T05:56:28.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-11T05:56:28.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-11T05:56:28.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-11T05:56:28.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-11T05:56:28.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-11T05:56:28.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-11T05:56:28.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-11T05:56:28.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-11T05:56:28.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-11T05:56:28.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-11T05:56:28.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-11T05:56:28.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-11T05:56:28.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-11T05:56:28.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-11T05:56:28.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-11T05:56:28.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-11T05:56:28.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-11T05:56:28.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-11T05:56:28.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-11T05:56:28.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-11T05:56:28.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-11T05:56:28.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-11T05:56:28.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-11T05:56:28.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-11T05:56:28.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-11T05:56:28.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-11T05:56:28.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-11T05:56:28.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-11T05:56:28.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-11T05:56:28.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-11T05:56:28.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-11T05:56:28.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-11T05:56:28.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-11T05:56:28.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-11T05:56:28.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-11T05:56:28.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-11T05:56:28.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-11T05:56:28.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-11T05:56:28.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-11T05:56:28.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-11T05:56:28.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-11T05:56:28.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-11T05:56:28.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-11T05:56:28.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-11T05:56:28.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-11T05:56:28.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-11T05:56:28.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-11T05:56:28.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-11T05:56:28.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-11T05:56:28.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-11T05:56:28.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-11T05:56:28.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-11T05:56:28.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-11T05:56:28.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-11T05:56:28.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-11T05:56:28.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-11T05:56:28.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-11T05:56:28.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-11T05:56:28.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-11T05:56:28.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-11T05:56:28.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-11T05:56:28.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-11T05:56:28.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-11T05:56:28.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-11T05:56:28.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-11T05:56:28.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-11T05:56:28.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-11T05:56:28.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-11T05:56:28.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-11T05:56:28.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-11T05:56:28.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-11T05:56:28.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-11T05:56:28.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-11T05:56:28.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-11T05:56:28.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-11T05:56:28.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-11T05:56:28.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-11T05:56:28.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-11T05:56:28.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-11T05:56:28.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-11T05:56:28.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-11T05:56:28.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-11T05:56:28.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-11T05:56:28.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-11T05:56:28.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-11T05:56:28.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-11T05:56:28.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-11T05:56:28.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-11T05:56:28.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-11T05:56:28.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-11T05:56:28.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-11T05:56:28.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-11T05:56:28.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-11T05:56:28.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-11T05:56:28.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-11T05:56:28.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-11T05:56:28.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-11T05:56:28.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-11T05:56:28.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-11T05:56:28.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-11T05:56:28.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-11T05:56:28.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-11T05:56:28.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-11T05:56:28.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-11T05:56:28.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-11T05:56:28.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-11T05:56:28.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-11T05:56:28.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-11T05:56:28.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-11T05:56:28.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-11T05:56:28.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-11T05:56:28.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-11T05:56:28.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-11T05:56:28.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-11T05:56:28.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-11T05:56:28.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-11T05:56:28.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-11T05:56:28.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-11T05:56:28.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-11T05:56:28.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-11T05:56:28.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-11T05:56:28.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-11T05:56:28.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-11T05:56:28.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-11T05:56:28.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-11T05:56:28.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-11T05:56:28.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-11T05:56:28.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-11T05:56:28.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-11T05:56:28.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-11T05:56:28.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-11T05:56:28.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-11T05:56:28.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-11T05:56:28.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-11T05:56:28.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-11T05:56:28.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-11T05:56:28.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-11T05:56:28.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-11T05:56:28.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-11T05:56:28.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-11T05:56:28.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-11T05:56:28.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-11T05:56:28.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-11T05:56:28.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-11T05:56:28.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-11T05:56:28.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-11T05:56:28.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-11T05:56:28.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-11T05:56:28.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-11T05:56:28.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-11T05:56:28.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-11T05:56:28.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-11T05:56:28.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-11T05:56:28.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-11T05:56:28.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-11T05:56:28.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-11T05:56:28.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-11T05:56:28.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-11T05:56:28.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-11T05:56:28.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-11T05:56:28.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-11T05:56:28.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-11T05:56:28.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-11T05:56:28.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-11T05:56:28.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-11T05:56:28.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-11T05:56:28.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-11T05:56:28.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-11T05:56:28.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-11T05:56:28.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-11T05:56:28.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-11T05:56:28.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-11T05:56:28.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-11T05:56:28.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-11T05:56:28.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-11T05:56:28.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-11T05:56:28.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-11T05:56:28.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-11T05:56:28.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-11T05:56:28.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-11T05:56:28.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-11T05:56:28.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-11T05:56:28.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-11T05:56:28.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-11T05:56:28.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-11T05:56:28.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-11T05:56:28.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-11T05:56:28.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-11T05:56:28.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-11T05:56:28.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-11T05:56:28.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-11T05:56:28.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-11T05:56:28.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-11T05:56:28.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-11T05:56:28.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-11T05:56:28.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-11T05:56:28.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-11T05:56:28.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-11T05:56:28.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-11T05:56:28.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-11T05:56:28.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-11T05:56:28.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-11T05:56:28.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-11T05:56:28.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-11T05:56:28.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-11T05:56:28.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-11T05:56:28.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-11T05:56:28.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-11T05:56:28.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-11T05:56:28.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-11T05:56:28.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-11T05:56:28.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-11T05:56:28.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-11T05:56:28.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-11T05:56:28.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-11T05:56:28.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-11T05:56:28.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-11T05:56:28.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-11T05:56:28.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-11T05:56:28.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-11T05:56:28.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-11T05:56:28.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-11T05:56:28.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-11T05:56:28.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-11T05:56:28.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-11T05:56:28.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-11T05:56:28.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-11T05:56:28.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-11T05:56:28.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-11T05:56:28.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-11T05:56:28.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-11T05:56:28.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-11T05:56:28.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-11T05:56:28.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-11T05:56:28.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-11T05:56:28.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-11T05:56:28.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-11T05:56:28.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-11T05:56:28.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-11T05:56:28.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-11T05:56:28.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-11T05:56:28.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-11T05:56:28.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-11T05:56:28.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-11T05:56:28.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-11T05:56:28.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-11T05:56:28.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-11T05:56:28.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-11T05:56:28.794 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-11T05:56:28.794 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-11T05:56:28.794 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-11T05:56:28.794 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-11T05:56:28.794 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-11T05:56:28.795 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-11T05:56:28.795 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-11T05:56:28.795 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-11T05:56:28.795 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-11T05:56:28.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-11T05:56:28.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-11T05:56:28.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-11T05:56:28.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-11T05:56:28.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-11T05:56:28.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-11T05:56:28.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-11T05:56:28.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-11T05:56:28.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-11T05:56:28.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-11T05:56:28.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-11T05:56:28.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-11T05:56:28.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-11T05:56:28.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-11T05:56:28.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-11T05:56:28.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-11T05:56:28.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-11T05:56:28.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-11T05:56:28.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-11T05:56:28.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-11T05:56:28.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-11T05:56:28.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-11T05:56:28.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-11T05:56:28.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-11T05:56:28.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-11T05:56:28.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-11T05:56:28.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-11T05:56:28.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-11T05:56:28.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-11T05:56:28.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-11T05:56:28.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-11T05:56:28.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-11T05:56:28.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-11T05:56:28.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-11T05:56:28.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-11T05:56:28.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-11T05:56:28.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-11T05:56:28.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-11T05:56:28.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-11T05:56:28.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-11T05:56:28.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-11T05:56:28.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-11T05:56:28.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-11T05:56:28.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-11T05:56:28.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-11T05:56:28.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-11T05:56:28.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-11T05:56:28.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-11T05:56:28.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-11T05:56:28.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-11T05:56:28.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-11T05:56:28.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-11T05:56:28.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-11T05:56:28.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-11T05:56:28.807 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.808 DEBUG:teuthology.orchestra.run.smithi099:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-11T05:56:28.831 INFO:teuthology.orchestra.run.smithi099.stdout:postinstall scriptlet (using /bin/sh): 2023-10-11T05:56:28.831 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-11T05:56:28.832 INFO:teuthology.orchestra.run.smithi099.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:28.832 INFO:teuthology.orchestra.run.smithi099.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:28.832 INFO:teuthology.orchestra.run.smithi099.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.832 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:28.832 INFO:teuthology.orchestra.run.smithi099.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-11T05:56:28.833 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2023-10-11T05:56:28.833 INFO:teuthology.orchestra.run.smithi099.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-11T05:56:28.833 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-11T05:56:28.833 INFO:teuthology.orchestra.run.smithi099.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-11T05:56:28.833 INFO:teuthology.orchestra.run.smithi099.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-11T05:56:28.834 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.834 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2023-10-11T05:56:28.834 INFO:teuthology.orchestra.run.smithi099.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-11T05:56:28.834 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-11T05:56:28.834 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.835 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2023-10-11T05:56:28.836 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2023-10-11T05:56:28.912 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2023-10-11T05:56:28.913 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2023-10-11T05:56:28.913 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.06 2023-10-11T05:56:28.913 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 68.el9 2023-10-11T05:56:28.913 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2023-10-11T05:56:28.914 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-11T05:56:28.914 INFO:teuthology.orchestra.run.smithi099.stdout:Group : Unspecified 2023-10-11T05:56:28.914 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 8264254 2023-10-11T05:56:28.914 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2023-10-11T05:56:28.914 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-11T05:56:28.915 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-11T05:56:28.915 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-11T05:56:28.915 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-11T05:56:28.915 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : builder@centos.org 2023-10-11T05:56:28.915 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2023-10-11T05:56:28.916 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-11T05:56:28.916 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2023-10-11T05:56:28.916 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2023-10-11T05:56:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T05:56:28.916 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-11T05:56:28.917 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-11T05:56:28.917 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-11T05:56:28.917 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2023-10-11T05:56:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T05:56:28.917 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2023-10-11T05:56:28.918 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-11T05:56:28.918 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:28.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-11T05:56:29.654 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2023-10-11T05:56:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-11T05:56:31.123 INFO:teuthology.orchestra.run.smithi099.stderr:done 2023-10-11T05:56:31.125 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-10-11T05:56:31.140 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.zBU5eL6guJ 2023-10-11T05:56:31.141 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zBU5eL6guJ 2023-10-11T05:56:31.209 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.zBU5eL6guJ 2023-10-11T05:56:31.349 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.zBU5eL6guJ is 6KB 2023-10-11T05:56:31.398 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.zBU5eL6guJ 2023-10-11T05:56:31.412 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2023-10-11T05:56:31.477 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-11T05:56:31.478 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-11T05:56:31.478 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-11T05:56:31.478 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b.conf 2023-10-11T05:56:31.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b 2023-10-11T05:56:31.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2023-10-11T05:56:41.890 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-11T05:56:41.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-11T05:56:41.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:57:00.410 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2023-10-11T05:57:01.647 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-11T05:57:01.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-11T05:57:01.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:57:09.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-11T05:57:09.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:57:12.478 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2023-10-11T05:57:20.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-10-11T05:57:24.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-11T05:57:24.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:57:29.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-11T05:57:29.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:57:32.186 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-10-11T05:57:42.906 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2023-10-11T05:57:44.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-11T05:57:44.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:57:57.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-11T05:57:57.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T05:57:58.406 DEBUG:teuthology.orchestra.run.smithi084:> true 2023-10-11T05:57:59.066 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-11T05:57:59.066 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-11T05:57:59.067 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2023-10-11T05:57:59.109 INFO:teuthology.orchestra.run.smithi084.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-11T05:57:59.110 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:57:59.110 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-11T05:57:59.110 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-11T05:58:00.112 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-11T05:58:00.112 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-11T05:58:00.232 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2023-10-11T05:58:00.285 DEBUG:teuthology.parallel:result is None 2023-10-11T05:58:16.113 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-10-11T05:58:16.807 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-11T05:58:16.808 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-11T05:58:16.808 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2023-10-11T05:58:16.849 INFO:teuthology.orchestra.run.smithi099.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-11T05:58:16.849 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-11T05:58:16.850 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-11T05:58:16.850 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-11T05:58:17.851 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-11T05:58:17.852 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-11T05:58:18.026 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2023-10-11T05:58:18.077 DEBUG:teuthology.parallel:result is None 2023-10-11T05:58:18.078 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-11T05:58:18.085 INFO:teuthology.task.internal:Creating test directory... 2023-10-11T05:58:18.086 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-11T05:58:18.096 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-11T05:58:18.148 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-11T05:58:18.155 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-11T05:58:18.162 INFO:teuthology.task.internal:Creating archive directory... 2023-10-11T05:58:18.163 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-11T05:58:18.169 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-11T05:58:18.256 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-11T05:58:18.263 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-11T05:58:18.264 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-11T05:58:18.270 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-11T05:58:18.360 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T05:58:18.380 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T05:58:18.403 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T05:58:18.427 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-11T05:58:18.433 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-11T05:58:18.440 INFO:teuthology.task.internal:Configuring sudo... 2023-10-11T05:58:18.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-11T05:58:18.446 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-11T05:58:18.541 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-11T05:58:18.551 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-11T05:58:18.552 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-11T05:58:18.557 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-11T05:58:18.607 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-11T05:58:18.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-11T05:58:18.793 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-11T05:58:18.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-11T05:58:18.971 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T05:58:18.971 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-11T05:58:19.064 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-11T05:58:19.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-11T05:58:19.228 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-11T05:58:19.295 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-11T05:58:19.389 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T05:58:19.390 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-11T05:58:19.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2023-10-11T05:58:19.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2023-10-11T05:58:19.596 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-11T05:58:19.619 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-11T05:58:20.117 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-11T05:58:20.124 INFO:teuthology.task.internal:Starting timer... 2023-10-11T05:58:20.125 INFO:teuthology.run_tasks:Running task pcp... 2023-10-11T05:58:20.184 INFO:teuthology.run_tasks:Running task selinux... 2023-10-11T05:58:20.192 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2023-10-11T05:58:20.344 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2023-10-11T05:58:20.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2023-10-11T05:58:20.493 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2023-10-11T05:58:20.497 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-11T05:58:20.498 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2023-10-11T05:58:20.566 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2023-10-11T05:58:20.567 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2023-10-11T05:58:20.635 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2023-10-11T05:58:20.635 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive'} 2023-10-11T05:58:20.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-11T05:58:20.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T05:58:20.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-11T05:58:20.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T05:58:20.819 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-11T05:58:20.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2023-10-11T05:58:20.921 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2023-10-11T05:58:21.014 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-11T05:58:21.025 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-11T05:58:21.030 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-11T05:58:21.149 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-11T05:58:21.168 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-11T05:58:21.169 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi084.front.sepia.ceph.com,smithi099.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-11T06:08:05.527 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com')] 2023-10-11T06:08:05.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-11T06:08:05.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T06:08:05.661 DEBUG:teuthology.orchestra.run.smithi084:> true 2023-10-11T06:08:05.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-10-11T06:08:05.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-11T06:08:05.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-11T06:08:05.955 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-10-11T06:08:06.109 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-10-11T06:08:06.110 INFO:teuthology.run_tasks:Running task clock... 2023-10-11T06:08:06.121 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-11T06:08:06.121 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-11T06:08:06.122 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T06:08:06.132 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-11T06:08:06.132 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T06:08:06.232 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-11T06:08:06.237 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-11T06:08:06.282 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-11T06:08:06.291 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-11T06:08:06.376 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2023-10-11T06:08:06.383 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2023-10-11T06:08:06.415 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2023-10-11T06:08:06.423 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2023-10-11T06:08:06.464 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-11T06:08:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-11T06:08:06.513 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-11T06:08:06.525 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-11T06:08:06.633 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2023-10-11T06:08:06.643 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-11T06:08:06.644 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2023-10-11T06:08:06.644 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.644 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.644 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.644 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.654 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2023-10-11T06:08:06.663 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-11T06:08:06.663 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-10-11T06:08:06.663 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.664 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.664 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.664 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:08:06.665 INFO:teuthology.run_tasks:Running task install... 2023-10-11T06:08:06.674 DEBUG:teuthology.task.install:project ceph 2023-10-11T06:08:06.674 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}} 2023-10-11T06:08:06.675 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'} 2023-10-11T06:08:06.675 INFO:teuthology.task.install:Using flavor: default 2023-10-11T06:08:06.680 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-11T06:08:06.681 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-11T06:08:06.681 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af', 'tag': None, 'wait_for_package': False} 2023-10-11T06:08:06.682 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T06:08:06.684 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af', 'tag': None, 'wait_for_package': False} 2023-10-11T06:08:06.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T06:08:06.827 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/ 2023-10-11T06:08:06.828 INFO:teuthology.task.install.rpm:Package version is 18.2.0-662.g9a774796 2023-10-11T06:08:06.904 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/ 2023-10-11T06:08:06.904 INFO:teuthology.task.install.rpm:Package version is 18.2.0-662.g9a774796 2023-10-11T06:08:06.937 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-11T06:08:06.937 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:08:06.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-11T06:08:07.040 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-11T06:08:07.040 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a774796f0ecdd11a4b771de396e627aa1a187af/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-11T06:08:07.069 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-reef-20231009.131610/9a774796f0ecdd11a4b771de396e627aa1a187af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-11T06:08:07.070 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:08:07.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-11T06:08:07.161 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-11T06:08:07.161 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a774796f0ecdd11a4b771de396e627aa1a187af/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-11T06:08:07.183 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-11T06:08:07.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-11T06:08:07.310 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-11T06:08:07.403 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2023-10-11T06:08:07.407 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2023-10-11T06:08:07.437 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-11T06:08:07.533 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2023-10-11T06:08:07.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2023-10-11T06:08:07.756 INFO:teuthology.orchestra.run.smithi099.stdout:70 files removed 2023-10-11T06:08:07.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-11T06:08:07.892 INFO:teuthology.orchestra.run.smithi084.stdout:70 files removed 2023-10-11T06:08:07.931 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-11T06:08:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 136 kB/s | 77 kB 00:00 2023-10-11T06:08:08.882 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 173 kB/s | 77 kB 00:00 2023-10-11T06:08:09.424 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 21 kB/s | 9.5 kB 00:00 2023-10-11T06:08:09.427 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 20 kB/s | 9.5 kB 00:00 2023-10-11T06:08:09.945 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2023-10-11T06:08:10.000 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2023-10-11T06:08:10.599 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-11T06:08:10.619 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-10-11T06:08:12.169 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2023-10-11T06:08:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2023-10-11T06:08:13.287 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 7.9 MB 00:00 2023-10-11T06:08:15.616 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 7.9 MB 00:03 2023-10-11T06:08:16.346 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 18 MB 00:01 2023-10-11T06:08:17.525 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 24 MB/s | 18 MB 00:00 2023-10-11T06:08:20.688 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 5.7 MB 00:00 2023-10-11T06:08:22.322 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 14 kB 00:00 2023-10-11T06:08:22.834 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 5.7 MB 00:01 2023-10-11T06:08:24.301 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 14 kB 00:00 2023-10-11T06:08:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 19 MB 00:02 2023-10-11T06:08:25.986 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 13 MB/s | 19 MB 00:01 2023-10-11T06:08:30.048 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 8.5 kB/s | 1.7 kB 00:00 2023-10-11T06:08:31.136 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2023-10-11T06:08:32.651 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-11T06:08:32.653 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-11T06:08:32.672 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-11T06:08:32.674 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-11T06:08:32.675 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-11T06:08:32.677 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-11T06:08:32.679 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-11T06:08:32.683 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-11T06:08:32.684 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-11T06:08:32.761 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2023-10-11T06:08:32.774 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================== 2023-10-11T06:08:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2023-10-11T06:08:32.775 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================== 2023-10-11T06:08:32.775 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2023-10-11T06:08:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-11T06:08:32.775 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.2.0-662.g9a774796.el9 ceph 6.5 k 2023-10-11T06:08:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.2.0-662.g9a774796.el9 ceph 5.2 M 2023-10-11T06:08:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.2.0-662.g9a774796.el9 ceph 839 k 2023-10-11T06:08:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-662.g9a774796.el9 ceph 142 k 2023-10-11T06:08:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.2.0-662.g9a774796.el9 ceph 1.4 M 2023-10-11T06:08:32.776 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 132 k 2023-10-11T06:08:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 1.8 M 2023-10-11T06:08:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 7.4 M 2023-10-11T06:08:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 47 k 2023-10-11T06:08:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.2.0-662.g9a774796.el9 ceph 7.7 M 2023-10-11T06:08:32.777 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.2.0-662.g9a774796.el9 ceph 40 M 2023-10-11T06:08:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 262 k 2023-10-11T06:08:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 218 k 2023-10-11T06:08:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:18.2.0-662.g9a774796.el9 ceph 219 k 2023-10-11T06:08:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 18 k 2023-10-11T06:08:32.778 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-11T06:08:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-11T06:08:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-11T06:08:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.2.0-662.g9a774796.el9 ceph 31 k 2023-10-11T06:08:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.2.0-662.g9a774796.el9 ceph 658 k 2023-10-11T06:08:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.2.0-662.g9a774796.el9 ceph 127 k 2023-10-11T06:08:32.779 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-11T06:08:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-11T06:08:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-11T06:08:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-11T06:08:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-11T06:08:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-11T06:08:32.780 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.2.0-662.g9a774796.el9 ceph 162 k 2023-10-11T06:08:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.2.0-662.g9a774796.el9 ceph 321 k 2023-10-11T06:08:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.2.0-662.g9a774796.el9 ceph 298 k 2023-10-11T06:08:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.2.0-662.g9a774796.el9 ceph 99 k 2023-10-11T06:08:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.2.0-662.g9a774796.el9 ceph 86 k 2023-10-11T06:08:32.781 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.2.0-662.g9a774796.el9 ceph 3.0 M 2023-10-11T06:08:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.2.0-662.g9a774796.el9 ceph 170 k 2023-10-11T06:08:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-11T06:08:32.782 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2023-10-11T06:08:32.782 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-11T06:08:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-11T06:08:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-11T06:08:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-11T06:08:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-11T06:08:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-11T06:08:32.783 INFO:teuthology.orchestra.run.smithi099.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-11T06:08:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.2.0-662.g9a774796.el9 ceph 3.3 M 2023-10-11T06:08:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.2.0-662.g9a774796.el9 ceph 3.0 M 2023-10-11T06:08:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-11T06:08:32.784 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2023-10-11T06:08:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-11T06:08:32.784 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-11T06:08:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.2.0-662.g9a774796.el9 ceph 18 M 2023-10-11T06:08:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 23 k 2023-10-11T06:08:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.2.0-662.g9a774796.el9 ceph 2.1 M 2023-10-11T06:08:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 242 k 2023-10-11T06:08:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.2.0-662.g9a774796.el9 ceph 4.4 M 2023-10-11T06:08:32.785 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.2.0-662.g9a774796.el9 ceph 18 M 2023-10-11T06:08:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 15 k 2023-10-11T06:08:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.2.0-662.g9a774796.el9 ceph 24 k 2023-10-11T06:08:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-11T06:08:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-11T06:08:32.786 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-11T06:08:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-11T06:08:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-11T06:08:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-11T06:08:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-11T06:08:32.787 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-11T06:08:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-11T06:08:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.2.0-662.g9a774796.el9 ceph 161 k 2023-10-11T06:08:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-11T06:08:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-11T06:08:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-11T06:08:32.788 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-11T06:08:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-11T06:08:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.2.0-662.g9a774796.el9 ceph 474 k 2023-10-11T06:08:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-11T06:08:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.2.0-662.g9a774796.el9 ceph 4.4 M 2023-10-11T06:08:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-11T06:08:32.789 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-11T06:08:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-11T06:08:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-11T06:08:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-11T06:08:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-11T06:08:32.790 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-11T06:08:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-11T06:08:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-11T06:08:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-11T06:08:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-11T06:08:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-11T06:08:32.791 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.2.0-662.g9a774796.el9 ceph 45 k 2023-10-11T06:08:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.2.0-662.g9a774796.el9 ceph 124 k 2023-10-11T06:08:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-11T06:08:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-11T06:08:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-11T06:08:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-11T06:08:32.792 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-11T06:08:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-11T06:08:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-11T06:08:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-11T06:08:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-11T06:08:32.793 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-11T06:08:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-11T06:08:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-11T06:08:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-11T06:08:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-11T06:08:32.794 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-11T06:08:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-11T06:08:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-11T06:08:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-11T06:08:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-11T06:08:32.795 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-11T06:08:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-11T06:08:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-11T06:08:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-11T06:08:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-11T06:08:32.796 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-11T06:08:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-11T06:08:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-11T06:08:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-11T06:08:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-11T06:08:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-11T06:08:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-11T06:08:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-11T06:08:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-11T06:08:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-11T06:08:32.798 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-11T06:08:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-11T06:08:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-11T06:08:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-11T06:08:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-11T06:08:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-11T06:08:32.799 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-11T06:08:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-11T06:08:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-11T06:08:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-11T06:08:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-11T06:08:32.800 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-11T06:08:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-11T06:08:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-11T06:08:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-11T06:08:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-11T06:08:32.801 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-11T06:08:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-11T06:08:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-11T06:08:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-11T06:08:32.802 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2023-10-11T06:08:32.802 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-11T06:08:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:08:32.803 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2023-10-11T06:08:32.803 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================== 2023-10-11T06:08:32.803 INFO:teuthology.orchestra.run.smithi099.stdout:Install 133 Packages 2023-10-11T06:08:32.803 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 10 Packages 2023-10-11T06:08:32.804 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:08:32.804 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 196 M 2023-10-11T06:08:32.804 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2023-10-11T06:08:33.609 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-11T06:08:33.610 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-11T06:08:33.631 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-11T06:08:33.632 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-11T06:08:33.634 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-11T06:08:33.635 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-11T06:08:33.638 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-11T06:08:33.642 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-11T06:08:33.643 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-11T06:08:33.720 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2023-10-11T06:08:33.733 INFO:teuthology.orchestra.run.smithi099.stdout:(1/143): ceph-18.2.0-662.g9a774796.el9.x86_64.r 20 kB/s | 6.5 kB 00:00 2023-10-11T06:08:33.739 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================== 2023-10-11T06:08:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2023-10-11T06:08:33.740 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================== 2023-10-11T06:08:33.740 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2023-10-11T06:08:33.740 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-11T06:08:33.740 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:18.2.0-662.g9a774796.el9 ceph 6.5 k 2023-10-11T06:08:33.740 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:18.2.0-662.g9a774796.el9 ceph 5.2 M 2023-10-11T06:08:33.740 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:18.2.0-662.g9a774796.el9 ceph 839 k 2023-10-11T06:08:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-662.g9a774796.el9 ceph 142 k 2023-10-11T06:08:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:18.2.0-662.g9a774796.el9 ceph 1.4 M 2023-10-11T06:08:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 132 k 2023-10-11T06:08:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 1.8 M 2023-10-11T06:08:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 7.4 M 2023-10-11T06:08:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 47 k 2023-10-11T06:08:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:18.2.0-662.g9a774796.el9 ceph 7.7 M 2023-10-11T06:08:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:18.2.0-662.g9a774796.el9 ceph 40 M 2023-10-11T06:08:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 262 k 2023-10-11T06:08:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 218 k 2023-10-11T06:08:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:18.2.0-662.g9a774796.el9 ceph 219 k 2023-10-11T06:08:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 18 k 2023-10-11T06:08:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-11T06:08:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-11T06:08:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-11T06:08:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:18.2.0-662.g9a774796.el9 ceph 31 k 2023-10-11T06:08:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:18.2.0-662.g9a774796.el9 ceph 658 k 2023-10-11T06:08:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:18.2.0-662.g9a774796.el9 ceph 127 k 2023-10-11T06:08:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-11T06:08:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-11T06:08:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-11T06:08:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-11T06:08:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-11T06:08:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-11T06:08:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:18.2.0-662.g9a774796.el9 ceph 162 k 2023-10-11T06:08:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:18.2.0-662.g9a774796.el9 ceph 321 k 2023-10-11T06:08:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:18.2.0-662.g9a774796.el9 ceph 298 k 2023-10-11T06:08:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:18.2.0-662.g9a774796.el9 ceph 99 k 2023-10-11T06:08:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:18.2.0-662.g9a774796.el9 ceph 86 k 2023-10-11T06:08:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:18.2.0-662.g9a774796.el9 ceph 3.0 M 2023-10-11T06:08:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:18.2.0-662.g9a774796.el9 ceph 170 k 2023-10-11T06:08:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-11T06:08:33.747 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2023-10-11T06:08:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-11T06:08:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-11T06:08:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-11T06:08:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-11T06:08:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-11T06:08:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-11T06:08:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-11T06:08:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:18.2.0-662.g9a774796.el9 ceph 3.3 M 2023-10-11T06:08:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:18.2.0-662.g9a774796.el9 ceph 3.0 M 2023-10-11T06:08:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-11T06:08:33.749 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2023-10-11T06:08:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-11T06:08:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-11T06:08:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:18.2.0-662.g9a774796.el9 ceph 18 M 2023-10-11T06:08:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 23 k 2023-10-11T06:08:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:18.2.0-662.g9a774796.el9 ceph 2.1 M 2023-10-11T06:08:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 242 k 2023-10-11T06:08:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:18.2.0-662.g9a774796.el9 ceph 4.4 M 2023-10-11T06:08:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:18.2.0-662.g9a774796.el9 ceph 18 M 2023-10-11T06:08:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:18.2.0-662.g9a774796.el9 ceph-noarch 15 k 2023-10-11T06:08:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:18.2.0-662.g9a774796.el9 ceph 24 k 2023-10-11T06:08:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-11T06:08:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-11T06:08:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-11T06:08:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-11T06:08:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-11T06:08:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-11T06:08:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-11T06:08:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-11T06:08:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-11T06:08:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:18.2.0-662.g9a774796.el9 ceph 161 k 2023-10-11T06:08:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-11T06:08:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-11T06:08:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-11T06:08:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-11T06:08:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-11T06:08:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:18.2.0-662.g9a774796.el9 ceph 474 k 2023-10-11T06:08:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-11T06:08:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:18.2.0-662.g9a774796.el9 ceph 4.4 M 2023-10-11T06:08:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-11T06:08:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-11T06:08:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-11T06:08:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-11T06:08:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-11T06:08:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-11T06:08:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-11T06:08:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-11T06:08:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-11T06:08:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-11T06:08:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-11T06:08:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-11T06:08:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:18.2.0-662.g9a774796.el9 ceph 45 k 2023-10-11T06:08:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:18.2.0-662.g9a774796.el9 ceph 124 k 2023-10-11T06:08:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-11T06:08:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-11T06:08:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-11T06:08:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-11T06:08:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-11T06:08:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-11T06:08:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-11T06:08:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-11T06:08:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-11T06:08:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-11T06:08:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-11T06:08:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-11T06:08:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-11T06:08:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-11T06:08:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-11T06:08:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-11T06:08:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-11T06:08:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-11T06:08:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-11T06:08:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-11T06:08:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-11T06:08:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-11T06:08:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-11T06:08:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-11T06:08:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-11T06:08:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-11T06:08:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-11T06:08:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-11T06:08:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-11T06:08:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-11T06:08:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-11T06:08:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-11T06:08:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-11T06:08:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-11T06:08:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-11T06:08:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-11T06:08:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-11T06:08:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-11T06:08:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-11T06:08:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-11T06:08:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-11T06:08:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-11T06:08:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-11T06:08:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-11T06:08:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-11T06:08:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-11T06:08:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-11T06:08:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-11T06:08:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-11T06:08:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-11T06:08:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-11T06:08:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-11T06:08:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-11T06:08:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-11T06:08:33.768 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2023-10-11T06:08:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-11T06:08:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:08:33.768 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2023-10-11T06:08:33.768 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================== 2023-10-11T06:08:33.769 INFO:teuthology.orchestra.run.smithi084.stdout:Install 133 Packages 2023-10-11T06:08:33.769 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 10 Packages 2023-10-11T06:08:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:08:33.769 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 196 M 2023-10-11T06:08:33.770 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2023-10-11T06:08:34.066 INFO:teuthology.orchestra.run.smithi099.stdout:(2/143): ceph-fuse-18.2.0-662.g9a774796.el9.x86 2.5 MB/s | 839 kB 00:00 2023-10-11T06:08:34.142 INFO:teuthology.orchestra.run.smithi099.stdout:(3/143): ceph-immutable-object-cache-18.2.0-662 1.9 MB/s | 142 kB 00:00 2023-10-11T06:08:34.283 INFO:teuthology.orchestra.run.smithi099.stdout:(4/143): ceph-base-18.2.0-662.g9a774796.el9.x86 5.9 MB/s | 5.2 MB 00:00 2023-10-11T06:08:34.329 INFO:teuthology.orchestra.run.smithi084.stdout:(1/143): ceph-18.2.0-662.g9a774796.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2023-10-11T06:08:34.434 INFO:teuthology.orchestra.run.smithi099.stdout:(5/143): ceph-mgr-18.2.0-662.g9a774796.el9.x86_ 9.7 MB/s | 1.4 MB 00:00 2023-10-11T06:08:34.596 INFO:teuthology.orchestra.run.smithi084.stdout:(2/143): ceph-fuse-18.2.0-662.g9a774796.el9.x86 3.1 MB/s | 839 kB 00:00 2023-10-11T06:08:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:(6/143): ceph-mds-18.2.0-662.g9a774796.el9.x86_ 4.1 MB/s | 2.1 MB 00:00 2023-10-11T06:08:34.662 INFO:teuthology.orchestra.run.smithi084.stdout:(3/143): ceph-immutable-object-cache-18.2.0-662 2.1 MB/s | 142 kB 00:00 2023-10-11T06:08:34.876 INFO:teuthology.orchestra.run.smithi099.stdout:(7/143): ceph-mon-18.2.0-662.g9a774796.el9.x86_ 10 MB/s | 4.4 MB 00:00 2023-10-11T06:08:35.204 INFO:teuthology.orchestra.run.smithi084.stdout:(4/143): ceph-mds-18.2.0-662.g9a774796.el9.x86_ 3.9 MB/s | 2.1 MB 00:00 2023-10-11T06:08:35.663 INFO:teuthology.orchestra.run.smithi084.stdout:(5/143): ceph-mgr-18.2.0-662.g9a774796.el9.x86_ 3.2 MB/s | 1.4 MB 00:00 2023-10-11T06:08:35.726 INFO:teuthology.orchestra.run.smithi099.stdout:(8/143): ceph-radosgw-18.2.0-662.g9a774796.el9. 9.0 MB/s | 7.7 MB 00:00 2023-10-11T06:08:35.776 INFO:teuthology.orchestra.run.smithi099.stdout:(9/143): ceph-selinux-18.2.0-662.g9a774796.el9. 492 kB/s | 24 kB 00:00 2023-10-11T06:08:36.221 INFO:teuthology.orchestra.run.smithi084.stdout:(6/143): ceph-base-18.2.0-662.g9a774796.el9.x86 2.4 MB/s | 5.2 MB 00:02 2023-10-11T06:08:36.805 INFO:teuthology.orchestra.run.smithi084.stdout:(7/143): ceph-mon-18.2.0-662.g9a774796.el9.x86_ 3.9 MB/s | 4.4 MB 00:01 2023-10-11T06:08:37.011 INFO:teuthology.orchestra.run.smithi099.stdout:(10/143): ceph-common-18.2.0-662.g9a774796.el9. 5.1 MB/s | 18 MB 00:03 2023-10-11T06:08:37.118 INFO:teuthology.orchestra.run.smithi099.stdout:(11/143): cephfs-mirror-18.2.0-662.g9a774796.el 2.0 MB/s | 219 kB 00:00 2023-10-11T06:08:37.218 INFO:teuthology.orchestra.run.smithi099.stdout:(12/143): libcephfs-devel-18.2.0-662.g9a774796. 314 kB/s | 31 kB 00:00 2023-10-11T06:08:37.427 INFO:teuthology.orchestra.run.smithi099.stdout:(13/143): libcephfs2-18.2.0-662.g9a774796.el9.x 3.1 MB/s | 658 kB 00:00 2023-10-11T06:08:37.527 INFO:teuthology.orchestra.run.smithi099.stdout:(14/143): libcephsqlite-18.2.0-662.g9a774796.el 1.6 MB/s | 161 kB 00:00 2023-10-11T06:08:37.636 INFO:teuthology.orchestra.run.smithi099.stdout:(15/143): librados-devel-18.2.0-662.g9a774796.e 1.1 MB/s | 127 kB 00:00 2023-10-11T06:08:37.769 INFO:teuthology.orchestra.run.smithi099.stdout:(16/143): libradosstriper1-18.2.0-662.g9a774796 3.5 MB/s | 474 kB 00:00 2023-10-11T06:08:38.555 INFO:teuthology.orchestra.run.smithi084.stdout:(8/143): ceph-radosgw-18.2.0-662.g9a774796.el9. 4.4 MB/s | 7.7 MB 00:01 2023-10-11T06:08:38.586 INFO:teuthology.orchestra.run.smithi099.stdout:(17/143): librgw2-18.2.0-662.g9a774796.el9.x86_ 5.4 MB/s | 4.4 MB 00:00 2023-10-11T06:08:38.597 INFO:teuthology.orchestra.run.smithi084.stdout:(9/143): ceph-selinux-18.2.0-662.g9a774796.el9. 595 kB/s | 24 kB 00:00 2023-10-11T06:08:38.812 INFO:teuthology.orchestra.run.smithi099.stdout:(18/143): ceph-osd-18.2.0-662.g9a774796.el9.x86 4.2 MB/s | 18 MB 00:04 2023-10-11T06:08:38.837 INFO:teuthology.orchestra.run.smithi099.stdout:(19/143): python3-ceph-argparse-18.2.0-662.g9a7 180 kB/s | 45 kB 00:00 2023-10-11T06:08:38.895 INFO:teuthology.orchestra.run.smithi099.stdout:(20/143): python3-ceph-common-18.2.0-662.g9a774 1.5 MB/s | 124 kB 00:00 2023-10-11T06:08:38.937 INFO:teuthology.orchestra.run.smithi099.stdout:(21/143): python3-cephfs-18.2.0-662.g9a774796.e 1.6 MB/s | 162 kB 00:00 2023-10-11T06:08:39.041 INFO:teuthology.orchestra.run.smithi099.stdout:(22/143): python3-rbd-18.2.0-662.g9a774796.el9. 2.8 MB/s | 298 kB 00:00 2023-10-11T06:08:39.074 INFO:teuthology.orchestra.run.smithi099.stdout:(23/143): python3-rados-18.2.0-662.g9a774796.el 1.8 MB/s | 321 kB 00:00 2023-10-11T06:08:39.125 INFO:teuthology.orchestra.run.smithi099.stdout:(24/143): python3-rgw-18.2.0-662.g9a774796.el9. 1.2 MB/s | 99 kB 00:00 2023-10-11T06:08:39.158 INFO:teuthology.orchestra.run.smithi099.stdout:(25/143): rbd-fuse-18.2.0-662.g9a774796.el9.x86 1.0 MB/s | 86 kB 00:00 2023-10-11T06:08:39.268 INFO:teuthology.orchestra.run.smithi099.stdout:(26/143): rbd-nbd-18.2.0-662.g9a774796.el9.x86_ 1.5 MB/s | 170 kB 00:00 2023-10-11T06:08:39.360 INFO:teuthology.orchestra.run.smithi099.stdout:(27/143): ceph-grafana-dashboards-18.2.0-662.g9 255 kB/s | 23 kB 00:00 2023-10-11T06:08:39.468 INFO:teuthology.orchestra.run.smithi099.stdout:(28/143): ceph-mgr-cephadm-18.2.0-662.g9a774796 1.2 MB/s | 132 kB 00:00 2023-10-11T06:08:39.902 INFO:teuthology.orchestra.run.smithi099.stdout:(29/143): ceph-test-18.2.0-662.g9a774796.el9.x8 9.6 MB/s | 40 MB 00:04 2023-10-11T06:08:39.994 INFO:teuthology.orchestra.run.smithi099.stdout:(30/143): rbd-mirror-18.2.0-662.g9a774796.el9.x 3.4 MB/s | 3.0 MB 00:00 2023-10-11T06:08:40.078 INFO:teuthology.orchestra.run.smithi099.stdout:(31/143): ceph-mgr-dashboard-18.2.0-662.g9a7747 2.9 MB/s | 1.8 MB 00:00 2023-10-11T06:08:40.111 INFO:teuthology.orchestra.run.smithi099.stdout:(32/143): ceph-mgr-modules-core-18.2.0-662.g9a7 2.0 MB/s | 242 kB 00:00 2023-10-11T06:08:40.161 INFO:teuthology.orchestra.run.smithi099.stdout:(33/143): ceph-mgr-rook-18.2.0-662.g9a774796.el 576 kB/s | 47 kB 00:00 2023-10-11T06:08:40.195 INFO:teuthology.orchestra.run.smithi099.stdout:(34/143): ceph-prometheus-alerts-18.2.0-662.g9a 176 kB/s | 15 kB 00:00 2023-10-11T06:08:40.295 INFO:teuthology.orchestra.run.smithi099.stdout:(35/143): cephadm-18.2.0-662.g9a774796.el9.noar 2.2 MB/s | 218 kB 00:00 2023-10-11T06:08:40.329 INFO:teuthology.orchestra.run.smithi099.stdout:(36/143): ceph-volume-18.2.0-662.g9a774796.el9. 1.5 MB/s | 262 kB 00:00 2023-10-11T06:08:40.429 INFO:teuthology.orchestra.run.smithi099.stdout:(37/143): cephfs-top-18.2.0-662.g9a774796.el9.n 136 kB/s | 18 kB 00:00 2023-10-11T06:08:40.521 INFO:teuthology.orchestra.run.smithi099.stdout:(38/143): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2023-10-11T06:08:40.546 INFO:teuthology.orchestra.run.smithi099.stdout:(39/143): boost-program-options-1.75.0-8.el9.x8 930 kB/s | 106 kB 00:00 2023-10-11T06:08:40.580 INFO:teuthology.orchestra.run.smithi099.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2023-10-11T06:08:40.605 INFO:teuthology.orchestra.run.smithi099.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 575 kB/s | 32 kB 00:00 2023-10-11T06:08:40.705 INFO:teuthology.orchestra.run.smithi084.stdout:(10/143): ceph-osd-18.2.0-662.g9a774796.el9.x86 3.9 MB/s | 18 MB 00:04 2023-10-11T06:08:40.706 INFO:teuthology.orchestra.run.smithi099.stdout:(42/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2023-10-11T06:08:40.731 INFO:teuthology.orchestra.run.smithi099.stdout:(43/143): flexiblas-openblas-openmp-3.0.4-8.el9 161 kB/s | 17 kB 00:00 2023-10-11T06:08:40.813 INFO:teuthology.orchestra.run.smithi084.stdout:(11/143): cephfs-mirror-18.2.0-662.g9a774796.el 2.0 MB/s | 219 kB 00:00 2023-10-11T06:08:40.864 INFO:teuthology.orchestra.run.smithi099.stdout:(44/143): ceph-mgr-diskprediction-local-18.2.0- 7.7 MB/s | 7.4 MB 00:00 2023-10-11T06:08:40.872 INFO:teuthology.orchestra.run.smithi084.stdout:(12/143): libcephfs-devel-18.2.0-662.g9a774796. 542 kB/s | 31 kB 00:00 2023-10-11T06:08:40.890 INFO:teuthology.orchestra.run.smithi099.stdout:(45/143): libaio-devel-0.3.111-13.el9.x86_64.rp 64 kB/s | 10 kB 00:00 2023-10-11T06:08:40.915 INFO:teuthology.orchestra.run.smithi099.stdout:(46/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 872 kB/s | 159 kB 00:00 2023-10-11T06:08:40.957 INFO:teuthology.orchestra.run.smithi099.stdout:(47/143): librdkafka-1.6.1-102.el9.x86_64.rpm 7.2 MB/s | 661 kB 00:00 2023-10-11T06:08:41.007 INFO:teuthology.orchestra.run.smithi099.stdout:(48/143): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 578 kB 00:00 2023-10-11T06:08:41.030 INFO:teuthology.orchestra.run.smithi084.stdout:(13/143): libcephfs2-18.2.0-662.g9a774796.el9.x 4.1 MB/s | 658 kB 00:00 2023-10-11T06:08:41.032 INFO:teuthology.orchestra.run.smithi099.stdout:(49/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 425 kB/s | 46 kB 00:00 2023-10-11T06:08:41.066 INFO:teuthology.orchestra.run.smithi099.stdout:(50/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.7 MB/s | 282 kB 00:00 2023-10-11T06:08:41.091 INFO:teuthology.orchestra.run.smithi099.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 627 kB/s | 34 kB 00:00 2023-10-11T06:08:41.106 INFO:teuthology.orchestra.run.smithi084.stdout:(14/143): libcephsqlite-18.2.0-662.g9a774796.el 2.1 MB/s | 161 kB 00:00 2023-10-11T06:08:41.141 INFO:teuthology.orchestra.run.smithi099.stdout:(52/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.0 MB/s | 217 kB 00:00 2023-10-11T06:08:41.186 INFO:teuthology.orchestra.run.smithi084.stdout:(15/143): librados-devel-18.2.0-662.g9a774796.e 1.6 MB/s | 127 kB 00:00 2023-10-11T06:08:41.267 INFO:teuthology.orchestra.run.smithi099.stdout:(53/143): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2023-10-11T06:08:41.295 INFO:teuthology.orchestra.run.smithi084.stdout:(16/143): libradosstriper1-18.2.0-662.g9a774796 4.3 MB/s | 474 kB 00:00 2023-10-11T06:08:41.317 INFO:teuthology.orchestra.run.smithi099.stdout:(54/143): postgresql-13.11-1.el9.x86_64.rpm 7.4 MB/s | 1.5 MB 00:00 2023-10-11T06:08:41.358 INFO:teuthology.orchestra.run.smithi099.stdout:(55/143): postgresql-contrib-13.11-1.el9.x86_64 4.0 MB/s | 811 kB 00:00 2023-10-11T06:08:41.393 INFO:teuthology.orchestra.run.smithi099.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2023-10-11T06:08:41.618 INFO:teuthology.orchestra.run.smithi099.stdout:(57/143): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2023-10-11T06:08:41.645 INFO:teuthology.orchestra.run.smithi084.stdout:(17/143): ceph-common-18.2.0-662.g9a774796.el9. 2.4 MB/s | 18 MB 00:07 2023-10-11T06:08:41.704 INFO:teuthology.orchestra.run.smithi084.stdout:(18/143): python3-ceph-argparse-18.2.0-662.g9a7 780 kB/s | 45 kB 00:00 2023-10-11T06:08:41.743 INFO:teuthology.orchestra.run.smithi099.stdout:(58/143): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2023-10-11T06:08:41.768 INFO:teuthology.orchestra.run.smithi099.stdout:(59/143): python3-devel-3.9.17-2.el9.x86_64.rpm 568 kB/s | 205 kB 00:00 2023-10-11T06:08:41.802 INFO:teuthology.orchestra.run.smithi099.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 1.3 MB/s | 228 kB 00:00 2023-10-11T06:08:41.813 INFO:teuthology.orchestra.run.smithi084.stdout:(19/143): python3-ceph-common-18.2.0-662.g9a774 1.1 MB/s | 124 kB 00:00 2023-10-11T06:08:41.819 INFO:teuthology.orchestra.run.smithi099.stdout:(61/143): python3-jsonpatch-1.21-16.el9.noarch. 522 kB/s | 24 kB 00:00 2023-10-11T06:08:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:(62/143): python3-jsonpointer-2.0-4.el9.noarch. 285 kB/s | 18 kB 00:00 2023-10-11T06:08:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2023-10-11T06:08:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 434 kB/s | 32 kB 00:00 2023-10-11T06:08:41.938 INFO:teuthology.orchestra.run.smithi084.stdout:(20/143): python3-cephfs-18.2.0-662.g9a774796.e 1.3 MB/s | 162 kB 00:00 2023-10-11T06:08:42.037 INFO:teuthology.orchestra.run.smithi099.stdout:(65/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2023-10-11T06:08:42.055 INFO:teuthology.orchestra.run.smithi084.stdout:(21/143): python3-rados-18.2.0-662.g9a774796.el 2.7 MB/s | 321 kB 00:00 2023-10-11T06:08:42.062 INFO:teuthology.orchestra.run.smithi099.stdout:(66/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-10-11T06:08:42.095 INFO:teuthology.orchestra.run.smithi099.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2023-10-11T06:08:42.137 INFO:teuthology.orchestra.run.smithi099.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2023-10-11T06:08:42.171 INFO:teuthology.orchestra.run.smithi099.stdout:(69/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2023-10-11T06:08:42.189 INFO:teuthology.orchestra.run.smithi084.stdout:(22/143): librgw2-18.2.0-662.g9a774796.el9.x86_ 5.0 MB/s | 4.4 MB 00:00 2023-10-11T06:08:42.196 INFO:teuthology.orchestra.run.smithi099.stdout:(70/143): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 211 kB 00:00 2023-10-11T06:08:42.221 INFO:teuthology.orchestra.run.smithi099.stdout:(71/143): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 43 kB 00:00 2023-10-11T06:08:42.222 INFO:teuthology.orchestra.run.smithi084.stdout:(23/143): python3-rbd-18.2.0-662.g9a774796.el9. 1.8 MB/s | 298 kB 00:00 2023-10-11T06:08:42.246 INFO:teuthology.orchestra.run.smithi099.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 800 kB/s | 39 kB 00:00 2023-10-11T06:08:42.256 INFO:teuthology.orchestra.run.smithi084.stdout:(24/143): python3-rgw-18.2.0-662.g9a774796.el9. 1.5 MB/s | 99 kB 00:00 2023-10-11T06:08:42.357 INFO:teuthology.orchestra.run.smithi084.stdout:(25/143): rbd-fuse-18.2.0-662.g9a774796.el9.x86 648 kB/s | 86 kB 00:00 2023-10-11T06:08:42.405 INFO:teuthology.orchestra.run.smithi099.stdout:(73/143): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 300 kB 00:00 2023-10-11T06:08:42.423 INFO:teuthology.orchestra.run.smithi084.stdout:(26/143): rbd-nbd-18.2.0-662.g9a774796.el9.x86_ 2.5 MB/s | 170 kB 00:00 2023-10-11T06:08:42.499 INFO:teuthology.orchestra.run.smithi084.stdout:(27/143): ceph-grafana-dashboards-18.2.0-662.g9 313 kB/s | 23 kB 00:00 2023-10-11T06:08:42.574 INFO:teuthology.orchestra.run.smithi084.stdout:(28/143): ceph-mgr-cephadm-18.2.0-662.g9a774796 1.7 MB/s | 132 kB 00:00 2023-10-11T06:08:42.597 INFO:teuthology.orchestra.run.smithi099.stdout:(74/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2023-10-11T06:08:42.622 INFO:teuthology.orchestra.run.smithi099.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 158 kB/s | 56 kB 00:00 2023-10-11T06:08:42.647 INFO:teuthology.orchestra.run.smithi099.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 268 kB/s | 63 kB 00:00 2023-10-11T06:08:42.681 INFO:teuthology.orchestra.run.smithi099.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 1.4 MB/s | 79 kB 00:00 2023-10-11T06:08:42.714 INFO:teuthology.orchestra.run.smithi099.stdout:(78/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.7 MB/s | 803 kB 00:00 2023-10-11T06:08:42.754 INFO:teuthology.orchestra.run.smithi099.stdout:(79/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.0 MB/s | 195 kB 00:00 2023-10-11T06:08:42.779 INFO:teuthology.orchestra.run.smithi099.stdout:(80/143): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2023-10-11T06:08:42.829 INFO:teuthology.orchestra.run.smithi099.stdout:(81/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 241 kB 00:00 2023-10-11T06:08:42.855 INFO:teuthology.orchestra.run.smithi099.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-10-11T06:08:43.014 INFO:teuthology.orchestra.run.smithi099.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 490 kB/s | 113 kB 00:00 2023-10-11T06:08:43.039 INFO:teuthology.orchestra.run.smithi099.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 1.0 MB/s | 188 kB 00:00 2023-10-11T06:08:43.099 INFO:teuthology.orchestra.run.smithi084.stdout:(29/143): ceph-test-18.2.0-662.g9a774796.el9.x8 8.8 MB/s | 40 MB 00:04 2023-10-11T06:08:43.164 INFO:teuthology.orchestra.run.smithi099.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 179 kB/s | 22 kB 00:00 2023-10-11T06:08:43.191 INFO:teuthology.orchestra.run.smithi084.stdout:(30/143): rbd-mirror-18.2.0-662.g9a774796.el9.x 3.2 MB/s | 3.0 MB 00:00 2023-10-11T06:08:43.248 INFO:teuthology.orchestra.run.smithi099.stdout:(86/143): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2023-10-11T06:08:43.267 INFO:teuthology.orchestra.run.smithi084.stdout:(31/143): ceph-mgr-dashboard-18.2.0-662.g9a7747 2.5 MB/s | 1.8 MB 00:00 2023-10-11T06:08:43.273 INFO:teuthology.orchestra.run.smithi099.stdout:(87/143): jq-1.6-15.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-10-11T06:08:43.300 INFO:teuthology.orchestra.run.smithi084.stdout:(32/143): ceph-mgr-modules-core-18.2.0-662.g9a7 2.2 MB/s | 242 kB 00:00 2023-10-11T06:08:43.323 INFO:teuthology.orchestra.run.smithi099.stdout:(88/143): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2023-10-11T06:08:43.334 INFO:teuthology.orchestra.run.smithi084.stdout:(33/143): ceph-mgr-rook-18.2.0-662.g9a774796.el 722 kB/s | 47 kB 00:00 2023-10-11T06:08:43.348 INFO:teuthology.orchestra.run.smithi099.stdout:(89/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 455 kB/s | 45 kB 00:00 2023-10-11T06:08:43.359 INFO:teuthology.orchestra.run.smithi084.stdout:(34/143): ceph-prometheus-alerts-18.2.0-662.g9a 253 kB/s | 15 kB 00:00 2023-10-11T06:08:43.374 INFO:teuthology.orchestra.run.smithi099.stdout:(90/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 257 kB/s | 26 kB 00:00 2023-10-11T06:08:43.432 INFO:teuthology.orchestra.run.smithi099.stdout:(91/143): python3-pytz-2021.1-5.el9.noarch.rpm 615 kB/s | 51 kB 00:00 2023-10-11T06:08:43.434 INFO:teuthology.orchestra.run.smithi084.stdout:(35/143): cephadm-18.2.0-662.g9a774796.el9.noar 2.9 MB/s | 218 kB 00:00 2023-10-11T06:08:43.468 INFO:teuthology.orchestra.run.smithi084.stdout:(36/143): ceph-volume-18.2.0-662.g9a774796.el9. 1.9 MB/s | 262 kB 00:00 2023-10-11T06:08:43.483 INFO:teuthology.orchestra.run.smithi099.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 802 kB/s | 86 kB 00:00 2023-10-11T06:08:43.501 INFO:teuthology.orchestra.run.smithi084.stdout:(37/143): cephfs-top-18.2.0-662.g9a774796.el9.n 274 kB/s | 18 kB 00:00 2023-10-11T06:08:43.541 INFO:teuthology.orchestra.run.smithi099.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 420 kB/s | 45 kB 00:00 2023-10-11T06:08:43.591 INFO:teuthology.orchestra.run.smithi099.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 1.0 MB/s | 111 kB 00:00 2023-10-11T06:08:43.642 INFO:teuthology.orchestra.run.smithi099.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 3.0 MB/s | 309 kB 00:00 2023-10-11T06:08:43.685 INFO:teuthology.orchestra.run.smithi084.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 922 kB/s | 106 kB 00:00 2023-10-11T06:08:43.692 INFO:teuthology.orchestra.run.smithi099.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 496 kB/s | 24 kB 00:00 2023-10-11T06:08:43.727 INFO:teuthology.orchestra.run.smithi084.stdout:(39/143): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 920 kB 00:00 2023-10-11T06:08:43.734 INFO:teuthology.orchestra.run.smithi099.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2023-10-11T06:08:43.760 INFO:teuthology.orchestra.run.smithi084.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 4.3 MB/s | 307 kB 00:00 2023-10-11T06:08:43.786 INFO:teuthology.orchestra.run.smithi084.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 569 kB/s | 32 kB 00:00 2023-10-11T06:08:43.826 INFO:teuthology.orchestra.run.smithi099.stdout:(98/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 8.2 MB/s | 4.1 MB 00:00 2023-10-11T06:08:43.851 INFO:teuthology.orchestra.run.smithi099.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 581 kB/s | 67 kB 00:00 2023-10-11T06:08:43.903 INFO:teuthology.orchestra.run.smithi084.stdout:(42/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2023-10-11T06:08:43.926 INFO:teuthology.orchestra.run.smithi099.stdout:(100/143): python3-asyncssh-2.13.2-3.el9.noarch 7.2 MB/s | 548 kB 00:00 2023-10-11T06:08:43.928 INFO:teuthology.orchestra.run.smithi084.stdout:(43/143): flexiblas-openblas-openmp-3.0.4-8.el9 137 kB/s | 17 kB 00:00 2023-10-11T06:08:43.960 INFO:teuthology.orchestra.run.smithi099.stdout:(101/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2023-10-11T06:08:43.961 INFO:teuthology.orchestra.run.smithi084.stdout:(44/143): libaio-devel-0.3.111-13.el9.x86_64.rp 313 kB/s | 10 kB 00:00 2023-10-11T06:08:43.978 INFO:teuthology.orchestra.run.smithi084.stdout:(45/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2023-10-11T06:08:43.993 INFO:teuthology.orchestra.run.smithi099.stdout:(102/143): python3-cachetools-4.2.4-1.el9.noarc 1.0 MB/s | 32 kB 00:00 2023-10-11T06:08:44.037 INFO:teuthology.orchestra.run.smithi084.stdout:(46/143): librdkafka-1.6.1-102.el9.x86_64.rpm 8.9 MB/s | 661 kB 00:00 2023-10-11T06:08:44.085 INFO:teuthology.orchestra.run.smithi099.stdout:(103/143): libarrow-9.0.0-9.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2023-10-11T06:08:44.087 INFO:teuthology.orchestra.run.smithi084.stdout:(47/143): libtool-2.4.6-45.el9.x86_64.rpm 5.6 MB/s | 578 kB 00:00 2023-10-11T06:08:44.110 INFO:teuthology.orchestra.run.smithi099.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2023-10-11T06:08:44.113 INFO:teuthology.orchestra.run.smithi084.stdout:(48/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 668 kB/s | 46 kB 00:00 2023-10-11T06:08:44.144 INFO:teuthology.orchestra.run.smithi099.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2023-10-11T06:08:44.146 INFO:teuthology.orchestra.run.smithi084.stdout:(49/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-10-11T06:08:44.172 INFO:teuthology.orchestra.run.smithi084.stdout:(50/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.9 MB/s | 217 kB 00:00 2023-10-11T06:08:44.194 INFO:teuthology.orchestra.run.smithi099.stdout:(106/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.2 MB/s | 837 kB 00:00 2023-10-11T06:08:44.197 INFO:teuthology.orchestra.run.smithi084.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 722 kB/s | 34 kB 00:00 2023-10-11T06:08:44.228 INFO:teuthology.orchestra.run.smithi099.stdout:(107/143): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2023-10-11T06:08:44.253 INFO:teuthology.orchestra.run.smithi099.stdout:(108/143): python3-google-auth-2.23.2-1.el9.noa 1.8 MB/s | 196 kB 00:00 2023-10-11T06:08:44.278 INFO:teuthology.orchestra.run.smithi099.stdout:(109/143): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2023-10-11T06:08:44.304 INFO:teuthology.orchestra.run.smithi099.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2023-10-11T06:08:44.329 INFO:teuthology.orchestra.run.smithi099.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 469 kB/s | 23 kB 00:00 2023-10-11T06:08:44.339 INFO:teuthology.orchestra.run.smithi084.stdout:(52/143): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2023-10-11T06:08:44.354 INFO:teuthology.orchestra.run.smithi099.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2023-10-11T06:08:44.379 INFO:teuthology.orchestra.run.smithi099.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2023-10-11T06:08:44.389 INFO:teuthology.orchestra.run.smithi084.stdout:(53/143): postgresql-13.11-1.el9.x86_64.rpm 8.7 MB/s | 1.5 MB 00:00 2023-10-11T06:08:44.396 INFO:teuthology.orchestra.run.smithi099.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 222 kB/s | 9.0 kB 00:00 2023-10-11T06:08:44.413 INFO:teuthology.orchestra.run.smithi099.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 712 kB/s | 41 kB 00:00 2023-10-11T06:08:44.431 INFO:teuthology.orchestra.run.smithi084.stdout:(54/143): postgresql-contrib-13.11-1.el9.x86_64 8.7 MB/s | 811 kB 00:00 2023-10-11T06:08:44.439 INFO:teuthology.orchestra.run.smithi099.stdout:(116/143): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2023-10-11T06:08:44.456 INFO:teuthology.orchestra.run.smithi084.stdout:(55/143): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2023-10-11T06:08:44.464 INFO:teuthology.orchestra.run.smithi099.stdout:(117/143): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2023-10-11T06:08:44.481 INFO:teuthology.orchestra.run.smithi099.stdout:(118/143): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2023-10-11T06:08:44.506 INFO:teuthology.orchestra.run.smithi099.stdout:(119/143): python3-portend-3.1.0-2.el9.noarch.r 676 kB/s | 16 kB 00:00 2023-10-11T06:08:44.556 INFO:teuthology.orchestra.run.smithi099.stdout:(120/143): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2023-10-11T06:08:44.590 INFO:teuthology.orchestra.run.smithi099.stdout:(121/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2023-10-11T06:08:44.615 INFO:teuthology.orchestra.run.smithi084.stdout:(56/143): ceph-mgr-diskprediction-local-18.2.0- 4.9 MB/s | 7.4 MB 00:01 2023-10-11T06:08:44.623 INFO:teuthology.orchestra.run.smithi099.stdout:(122/143): python3-repoze-lru-0.7-16.el9.noarch 466 kB/s | 31 kB 00:00 2023-10-11T06:08:44.657 INFO:teuthology.orchestra.run.smithi099.stdout:(123/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 600 kB/s | 90 kB 00:00 2023-10-11T06:08:44.682 INFO:teuthology.orchestra.run.smithi099.stdout:(124/143): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2023-10-11T06:08:44.708 INFO:teuthology.orchestra.run.smithi099.stdout:(125/143): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2023-10-11T06:08:44.733 INFO:teuthology.orchestra.run.smithi099.stdout:(126/143): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2023-10-11T06:08:44.758 INFO:teuthology.orchestra.run.smithi099.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2023-10-11T06:08:44.774 INFO:teuthology.orchestra.run.smithi084.stdout:(57/143): postgresql-server-13.11-1.el9.x86_64. 17 MB/s | 5.7 MB 00:00 2023-10-11T06:08:44.783 INFO:teuthology.orchestra.run.smithi099.stdout:(128/143): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2023-10-11T06:08:44.808 INFO:teuthology.orchestra.run.smithi099.stdout:(129/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2023-10-11T06:08:44.834 INFO:teuthology.orchestra.run.smithi099.stdout:(130/143): python3-zc-lockfile-2.0-10.el9.noarc 402 kB/s | 20 kB 00:00 2023-10-11T06:08:44.867 INFO:teuthology.orchestra.run.smithi099.stdout:(131/143): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2023-10-11T06:08:44.893 INFO:teuthology.orchestra.run.smithi099.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2023-10-11T06:08:44.916 INFO:teuthology.orchestra.run.smithi084.stdout:(58/143): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2023-10-11T06:08:44.941 INFO:teuthology.orchestra.run.smithi084.stdout:(59/143): python3-devel-3.9.17-2.el9.x86_64.rpm 681 kB/s | 205 kB 00:00 2023-10-11T06:08:44.959 INFO:teuthology.orchestra.run.smithi099.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2023-10-11T06:08:44.966 INFO:teuthology.orchestra.run.smithi084.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 1.3 MB/s | 228 kB 00:00 2023-10-11T06:08:45.008 INFO:teuthology.orchestra.run.smithi084.stdout:(61/143): python3-jsonpointer-2.0-4.el9.noarch. 452 kB/s | 18 kB 00:00 2023-10-11T06:08:45.010 INFO:teuthology.orchestra.run.smithi099.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-10-11T06:08:45.033 INFO:teuthology.orchestra.run.smithi084.stdout:(62/143): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 144 kB 00:00 2023-10-11T06:08:45.058 INFO:teuthology.orchestra.run.smithi084.stdout:(63/143): python3-jsonpatch-1.21-16.el9.noarch. 269 kB/s | 24 kB 00:00 2023-10-11T06:08:45.083 INFO:teuthology.orchestra.run.smithi084.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 424 kB/s | 32 kB 00:00 2023-10-11T06:08:45.102 INFO:teuthology.orchestra.run.smithi099.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 25 MB/s | 2.2 MB 00:00 2023-10-11T06:08:45.202 INFO:teuthology.orchestra.run.smithi099.stdout:(136/143): NetworkManager-libnm-1.45.4-1.el9.x8 18 MB/s | 1.8 MB 00:00 2023-10-11T06:08:45.217 INFO:teuthology.orchestra.run.smithi084.stdout:(65/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2023-10-11T06:08:45.235 INFO:teuthology.orchestra.run.smithi099.stdout:(137/143): NetworkManager-team-1.45.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2023-10-11T06:08:45.242 INFO:teuthology.orchestra.run.smithi084.stdout:(66/143): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2023-10-11T06:08:45.294 INFO:teuthology.orchestra.run.smithi084.stdout:(67/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-10-11T06:08:45.311 INFO:teuthology.orchestra.run.smithi099.stdout:(138/143): librbd1-18.2.0-662.g9a774796.el9.x86 7.2 MB/s | 3.0 MB 00:00 2023-10-11T06:08:45.329 INFO:teuthology.orchestra.run.smithi084.stdout:(68/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2023-10-11T06:08:45.344 INFO:teuthology.orchestra.run.smithi099.stdout:(139/143): NetworkManager-tui-1.45.4-1.el9.x86_ 2.2 MB/s | 244 kB 00:00 2023-10-11T06:08:45.354 INFO:teuthology.orchestra.run.smithi084.stdout:(69/143): python3-prettytable-0.7.2-27.el9.noar 378 kB/s | 41 kB 00:00 2023-10-11T06:08:45.370 INFO:teuthology.orchestra.run.smithi099.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-10-11T06:08:45.379 INFO:teuthology.orchestra.run.smithi084.stdout:(70/143): python3-pyasn1-modules-0.4.8-6.el9.no 2.4 MB/s | 211 kB 00:00 2023-10-11T06:08:45.395 INFO:teuthology.orchestra.run.smithi099.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 451 kB/s | 22 kB 00:00 2023-10-11T06:08:45.406 INFO:teuthology.orchestra.run.smithi084.stdout:(71/143): python3-requests-oauthlib-1.3.0-12.el 591 kB/s | 43 kB 00:00 2023-10-11T06:08:45.438 INFO:teuthology.orchestra.run.smithi084.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 677 kB/s | 39 kB 00:00 2023-10-11T06:08:45.520 INFO:teuthology.orchestra.run.smithi099.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 2.1 MB/s | 321 kB 00:00 2023-10-11T06:08:45.637 INFO:teuthology.orchestra.run.smithi099.stdout:(143/143): librados2-18.2.0-662.g9a774796.el9.x 4.3 MB/s | 3.3 MB 00:00 2023-10-11T06:08:45.646 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2023-10-11T06:08:45.646 INFO:teuthology.orchestra.run.smithi099.stdout:Total 15 MB/s | 196 MB 00:12 2023-10-11T06:08:45.772 INFO:teuthology.orchestra.run.smithi084.stdout:(73/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2023-10-11T06:08:45.806 INFO:teuthology.orchestra.run.smithi084.stdout:(74/143): socat-1.7.4.1-5.el9.x86_64.rpm 751 kB/s | 300 kB 00:00 2023-10-11T06:08:45.831 INFO:teuthology.orchestra.run.smithi084.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 150 kB/s | 56 kB 00:00 2023-10-11T06:08:45.856 INFO:teuthology.orchestra.run.smithi084.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 785 kB/s | 63 kB 00:00 2023-10-11T06:08:45.890 INFO:teuthology.orchestra.run.smithi084.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-10-11T06:08:45.923 INFO:teuthology.orchestra.run.smithi084.stdout:(78/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.7 MB/s | 803 kB 00:00 2023-10-11T06:08:45.957 INFO:teuthology.orchestra.run.smithi084.stdout:(79/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2023-10-11T06:08:45.974 INFO:teuthology.orchestra.run.smithi084.stdout:(80/143): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2023-10-11T06:08:45.999 INFO:teuthology.orchestra.run.smithi084.stdout:(81/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2023-10-11T06:08:46.016 INFO:teuthology.orchestra.run.smithi084.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2023-10-11T06:08:46.042 INFO:teuthology.orchestra.run.smithi084.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 1.7 MB/s | 113 kB 00:00 2023-10-11T06:08:46.075 INFO:teuthology.orchestra.run.smithi084.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 3.2 MB/s | 188 kB 00:00 2023-10-11T06:08:46.081 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2023-10-11T06:08:46.267 INFO:teuthology.orchestra.run.smithi084.stdout:(85/143): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2023-10-11T06:08:46.285 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2023-10-11T06:08:46.285 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2023-10-11T06:08:46.342 INFO:teuthology.orchestra.run.smithi084.stdout:(86/143): elfutils-libelf-devel-0.189-3.el9.x86 84 kB/s | 22 kB 00:00 2023-10-11T06:08:46.542 INFO:teuthology.orchestra.run.smithi084.stdout:(87/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 227 kB/s | 45 kB 00:00 2023-10-11T06:08:46.651 INFO:teuthology.orchestra.run.smithi084.stdout:(88/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 237 kB/s | 26 kB 00:00 2023-10-11T06:08:46.760 INFO:teuthology.orchestra.run.smithi084.stdout:(89/143): automake-1.16.2-8.el9.noarch.rpm 964 kB/s | 689 kB 00:00 2023-10-11T06:08:46.785 INFO:teuthology.orchestra.run.smithi084.stdout:(90/143): jq-1.6-15.el9.x86_64.rpm 362 kB/s | 187 kB 00:00 2023-10-11T06:08:46.877 INFO:teuthology.orchestra.run.smithi084.stdout:(91/143): python3-pytz-2021.1-5.el9.noarch.rpm 439 kB/s | 51 kB 00:00 2023-10-11T06:08:46.902 INFO:teuthology.orchestra.run.smithi084.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 742 kB/s | 86 kB 00:00 2023-10-11T06:08:46.985 INFO:teuthology.orchestra.run.smithi084.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2023-10-11T06:08:47.461 INFO:teuthology.orchestra.run.smithi084.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 199 kB/s | 111 kB 00:00 2023-10-11T06:08:47.553 INFO:teuthology.orchestra.run.smithi084.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 623 kB/s | 309 kB 00:00 2023-10-11T06:08:47.644 INFO:teuthology.orchestra.run.smithi084.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 268 kB/s | 24 kB 00:00 2023-10-11T06:08:47.728 INFO:teuthology.orchestra.run.smithi084.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 588 kB/s | 49 kB 00:00 2023-10-11T06:08:47.820 INFO:teuthology.orchestra.run.smithi084.stdout:(98/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 3.5 MB/s | 4.1 MB 00:01 2023-10-11T06:08:47.871 INFO:teuthology.orchestra.run.smithi084.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 477 kB/s | 67 kB 00:00 2023-10-11T06:08:48.045 INFO:teuthology.orchestra.run.smithi084.stdout:(100/143): libarrow-9.0.0-9.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2023-10-11T06:08:48.121 INFO:teuthology.orchestra.run.smithi084.stdout:(101/143): python3-asyncssh-2.13.2-3.el9.noarch 3.0 MB/s | 548 kB 00:00 2023-10-11T06:08:48.146 INFO:teuthology.orchestra.run.smithi084.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 448 kB/s | 43 kB 00:00 2023-10-11T06:08:48.196 INFO:teuthology.orchestra.run.smithi084.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 435 kB/s | 32 kB 00:00 2023-10-11T06:08:48.221 INFO:teuthology.orchestra.run.smithi084.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2023-10-11T06:08:48.280 INFO:teuthology.orchestra.run.smithi084.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-10-11T06:08:48.322 INFO:teuthology.orchestra.run.smithi084.stdout:(106/143): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2023-10-11T06:08:48.364 INFO:teuthology.orchestra.run.smithi084.stdout:(107/143): python3-google-auth-2.23.2-1.el9.noa 2.3 MB/s | 196 kB 00:00 2023-10-11T06:08:48.389 INFO:teuthology.orchestra.run.smithi084.stdout:(108/143): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2023-10-11T06:08:48.423 INFO:teuthology.orchestra.run.smithi084.stdout:(109/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.4 MB/s | 837 kB 00:00 2023-10-11T06:08:48.448 INFO:teuthology.orchestra.run.smithi084.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2023-10-11T06:08:48.473 INFO:teuthology.orchestra.run.smithi084.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2023-10-11T06:08:48.499 INFO:teuthology.orchestra.run.smithi084.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2023-10-11T06:08:48.524 INFO:teuthology.orchestra.run.smithi084.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2023-10-11T06:08:48.549 INFO:teuthology.orchestra.run.smithi084.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 121 kB/s | 9.0 kB 00:00 2023-10-11T06:08:48.574 INFO:teuthology.orchestra.run.smithi084.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 548 kB/s | 41 kB 00:00 2023-10-11T06:08:48.633 INFO:teuthology.orchestra.run.smithi084.stdout:(116/143): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2023-10-11T06:08:48.638 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2023-10-11T06:08:48.640 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2023-10-11T06:08:48.658 INFO:teuthology.orchestra.run.smithi084.stdout:(117/143): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2023-10-11T06:08:48.684 INFO:teuthology.orchestra.run.smithi084.stdout:(118/143): python3-more-itertools-8.12.0-2.el9. 729 kB/s | 79 kB 00:00 2023-10-11T06:08:48.709 INFO:teuthology.orchestra.run.smithi084.stdout:(119/143): python3-natsort-7.1.1-5.el9.noarch.r 774 kB/s | 58 kB 00:00 2023-10-11T06:08:48.751 INFO:teuthology.orchestra.run.smithi084.stdout:(120/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2023-10-11T06:08:48.776 INFO:teuthology.orchestra.run.smithi084.stdout:(121/143): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2023-10-11T06:08:48.801 INFO:teuthology.orchestra.run.smithi084.stdout:(122/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 984 kB/s | 90 kB 00:00 2023-10-11T06:08:48.826 INFO:teuthology.orchestra.run.smithi084.stdout:(123/143): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2023-10-11T06:08:48.860 INFO:teuthology.orchestra.run.smithi084.stdout:(124/143): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2023-10-11T06:08:48.885 INFO:teuthology.orchestra.run.smithi084.stdout:(125/143): python3-rsa-4.9-2.el9.noarch.rpm 713 kB/s | 59 kB 00:00 2023-10-11T06:08:48.910 INFO:teuthology.orchestra.run.smithi084.stdout:(126/143): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2023-10-11T06:08:48.936 INFO:teuthology.orchestra.run.smithi084.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2023-10-11T06:08:48.970 INFO:teuthology.orchestra.run.smithi084.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-10-11T06:08:48.994 INFO:teuthology.orchestra.run.smithi084.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2023-10-11T06:08:49.036 INFO:teuthology.orchestra.run.smithi084.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2023-10-11T06:08:49.062 INFO:teuthology.orchestra.run.smithi084.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2023-10-11T06:08:49.087 INFO:teuthology.orchestra.run.smithi084.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2023-10-11T06:08:49.195 INFO:teuthology.orchestra.run.smithi084.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-10-11T06:08:49.237 INFO:teuthology.orchestra.run.smithi084.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-10-11T06:08:49.329 INFO:teuthology.orchestra.run.smithi084.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 25 MB/s | 2.2 MB 00:00 2023-10-11T06:08:49.413 INFO:teuthology.orchestra.run.smithi084.stdout:(136/143): NetworkManager-libnm-1.45.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2023-10-11T06:08:49.447 INFO:teuthology.orchestra.run.smithi084.stdout:(137/143): NetworkManager-team-1.45.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2023-10-11T06:08:49.497 INFO:teuthology.orchestra.run.smithi084.stdout:(138/143): NetworkManager-tui-1.45.4-1.el9.x86_ 4.9 MB/s | 244 kB 00:00 2023-10-11T06:08:49.589 INFO:teuthology.orchestra.run.smithi084.stdout:(139/143): librados2-18.2.0-662.g9a774796.el9.x 6.2 MB/s | 3.3 MB 00:00 2023-10-11T06:08:49.614 INFO:teuthology.orchestra.run.smithi084.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 593 kB/s | 69 kB 00:00 2023-10-11T06:08:49.639 INFO:teuthology.orchestra.run.smithi084.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 451 kB/s | 22 kB 00:00 2023-10-11T06:08:49.673 INFO:teuthology.orchestra.run.smithi084.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-10-11T06:08:49.856 INFO:teuthology.orchestra.run.smithi084.stdout:(143/143): librbd1-18.2.0-662.g9a774796.el9.x86 3.9 MB/s | 3.0 MB 00:00 2023-10-11T06:08:49.870 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2023-10-11T06:08:49.871 INFO:teuthology.orchestra.run.smithi084.stdout:Total 12 MB/s | 196 MB 00:16 2023-10-11T06:08:50.306 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2023-10-11T06:08:50.510 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2023-10-11T06:08:50.510 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2023-10-11T06:08:52.758 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2023-10-11T06:08:52.760 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2023-10-11T06:08:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2023-10-11T06:08:54.477 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-11T06:08:54.507 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-11T06:08:54.537 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-11T06:08:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-11T06:08:55.064 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-11T06:08:55.149 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.2.0-662.g9a774796.el9.x86_64 6/153 2023-10-11T06:08:55.382 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.2.0-662.g9a774796.el9.x86_64 6/153 2023-10-11T06:08:55.408 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 7/153 2023-10-11T06:08:55.537 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 7/153 2023-10-11T06:08:55.597 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.2.0-662.g9a774796.el9.x86_64 8/153 2023-10-11T06:08:55.643 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-11T06:08:55.917 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-11T06:08:55.979 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-11T06:08:56.004 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 12/153 2023-10-11T06:08:56.140 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 12/153 2023-10-11T06:08:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_ 13/153 2023-10-11T06:08:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_ 13/153 2023-10-11T06:08:56.609 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-11T06:08:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-11T06:08:56.873 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-11T06:08:57.130 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-11T06:08:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-11T06:08:57.604 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-11T06:08:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-11T06:08:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-11T06:08:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-11T06:08:57.859 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-11T06:08:57.898 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-11T06:08:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-11T06:08:58.056 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-11T06:08:58.108 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-11T06:08:58.144 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-11T06:08:58.246 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-11T06:08:58.288 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.2.0-662.g9a774796.el9.x 27/153 2023-10-11T06:08:58.332 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.2.0-662.g9a774796.el9 28/153 2023-10-11T06:08:58.436 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.2.0-662.g9a774796.el9.x86_64 29/153 2023-10-11T06:08:58.622 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-11T06:08:58.645 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2023-10-11T06:08:58.680 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-11T06:08:58.748 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-11T06:08:58.777 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-11T06:08:58.897 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-11T06:08:59.000 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-11T06:08:59.374 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-11T06:08:59.422 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-11T06:08:59.451 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:18.2.0-662.g9a774796.el9.x86_64 6/153 2023-10-11T06:08:59.471 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-11T06:08:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-11T06:08:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.2.0-662.g9a774796.el9.x86_64 35/153 2023-10-11T06:08:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:18.2.0-662.g9a774796.el9.x86_64 6/153 2023-10-11T06:08:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-11T06:08:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 7/153 2023-10-11T06:08:59.743 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 7/153 2023-10-11T06:08:59.756 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-11T06:08:59.796 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-11T06:08:59.803 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:18.2.0-662.g9a774796.el9.x86_64 8/153 2023-10-11T06:08:59.861 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-11T06:08:59.863 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-11T06:08:59.958 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-11T06:08:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-11T06:09:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-11T06:09:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-11T06:09:00.111 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-11T06:09:00.170 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-11T06:09:00.181 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-11T06:09:00.206 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 12/153 2023-10-11T06:09:00.253 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-11T06:09:00.351 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 12/153 2023-10-11T06:09:00.383 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_ 13/153 2023-10-11T06:09:00.466 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_ 13/153 2023-10-11T06:09:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-11T06:09:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-11T06:09:00.794 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-11T06:09:00.867 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-11T06:09:00.932 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-11T06:09:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-11T06:09:00.987 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-11T06:09:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-11T06:09:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-11T06:09:01.075 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-11T06:09:01.090 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-11T06:09:01.133 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-11T06:09:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-11T06:09:01.303 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-11T06:09:01.338 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-11T06:09:01.348 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-11T06:09:01.396 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-11T06:09:01.473 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-11T06:09:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-11T06:09:01.803 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-11T06:09:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-11T06:09:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-11T06:09:01.888 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-11T06:09:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-11T06:09:02.034 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-11T06:09:02.076 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-11T06:09:02.103 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-11T06:09:02.124 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-11T06:09:02.148 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-11T06:09:02.156 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-11T06:09:02.185 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-11T06:09:02.208 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-11T06:09:02.229 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-11T06:09:02.272 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-11T06:09:02.315 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-11T06:09:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-11T06:09:02.359 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-11T06:09:02.393 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-11T06:09:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-11T06:09:02.465 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-11T06:09:02.502 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-11T06:09:02.507 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-11T06:09:02.540 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:18.2.0-662.g9a774796.el9.x 27/153 2023-10-11T06:09:02.589 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:18.2.0-662.g9a774796.el9 28/153 2023-10-11T06:09:02.631 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-11T06:09:02.671 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-11T06:09:02.682 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:18.2.0-662.g9a774796.el9.x86_64 29/153 2023-10-11T06:09:02.705 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-11T06:09:02.766 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-11T06:09:02.798 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-11T06:09:02.871 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-11T06:09:02.920 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-11T06:09:02.935 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-11T06:09:02.965 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-11T06:09:03.103 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-11T06:09:03.180 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-11T06:09:03.651 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-11T06:09:03.748 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-11T06:09:03.801 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-11T06:09:03.840 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:18.2.0-662.g9a774796.el9.x86_64 35/153 2023-10-11T06:09:03.888 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-11T06:09:04.046 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-11T06:09:04.090 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-11T06:09:04.155 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-11T06:09:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-11T06:09:04.236 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-11T06:09:04.242 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-11T06:09:04.269 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-11T06:09:04.342 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-11T06:09:04.395 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-11T06:09:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-11T06:09:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-11T06:09:04.644 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-11T06:09:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-11T06:09:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-11T06:09:04.713 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-11T06:09:04.781 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-11T06:09:04.805 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-11T06:09:04.977 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-11T06:09:05.030 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-11T06:09:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-11T06:09:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-11T06:09:05.221 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-11T06:09:05.267 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-11T06:09:05.295 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-11T06:09:05.333 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-11T06:09:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-11T06:09:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-11T06:09:05.563 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-11T06:09:05.621 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-11T06:09:05.621 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-11T06:09:05.702 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-11T06:09:05.771 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-11T06:09:05.780 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-11T06:09:06.051 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-11T06:09:06.114 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-11T06:09:06.294 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-11T06:09:06.350 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-11T06:09:06.382 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-11T06:09:06.422 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-11T06:09:06.466 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-11T06:09:06.517 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-11T06:09:06.567 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-11T06:09:06.651 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-11T06:09:06.710 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-11T06:09:06.752 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-11T06:09:06.877 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-11T06:09:06.922 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-11T06:09:06.956 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-11T06:09:07.016 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-11T06:09:07.083 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-11T06:09:07.233 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-11T06:09:07.281 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-11T06:09:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-11T06:09:07.421 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-11T06:09:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-11T06:09:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-11T06:09:07.497 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-11T06:09:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-11T06:09:07.562 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-11T06:09:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-11T06:09:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.2.0-662.g9a774796.el9.x86_64 97/153 2023-10-11T06:09:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.2.0-662.g9a774796.el9.x86_64 97/153 2023-10-11T06:09:08.115 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.2.0-662.g9a774796.el9.x86_64 98/153 2023-10-11T06:09:08.171 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-11T06:09:08.489 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-11T06:09:08.545 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-11T06:09:08.578 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-11T06:09:08.649 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-11T06:09:08.679 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.2.0-662.g9a774796.el9.x86_64 101/153 2023-10-11T06:09:08.701 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-11T06:09:08.745 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-11T06:09:08.771 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.2.0-662.g9a774796.el9.x86_64 101/153 2023-10-11T06:09:08.798 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.2.0-662.g9a774796.el9.x86_64 102/153 2023-10-11T06:09:08.985 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-11T06:09:09.017 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-11T06:09:09.114 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-11T06:09:09.929 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-11T06:09:10.063 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-11T06:09:11.263 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 103/153 2023-10-11T06:09:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 103/153 2023-10-11T06:09:11.662 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-11T06:09:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-11T06:09:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-11T06:09:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-11T06:09:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-11T06:09:11.931 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 103/153 2023-10-11T06:09:11.989 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 104/153 2023-10-11T06:09:12.155 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 104/153 2023-10-11T06:09:12.155 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-11T06:09:12.156 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-11T06:09:12.156 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 105/153 2023-10-11T06:09:12.271 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-11T06:09:12.301 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:18.2.0-662.g9a774796.el9.x86_64 97/153 2023-10-11T06:09:12.413 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:18.2.0-662.g9a774796.el9.x86_64 97/153 2023-10-11T06:09:12.448 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:18.2.0-662.g9a774796.el9.x86_64 98/153 2023-10-11T06:09:12.505 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-11T06:09:12.961 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-11T06:09:12.995 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:18.2.0-662.g9a774796.el9.x86_64 101/153 2023-10-11T06:09:13.070 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:18.2.0-662.g9a774796.el9.x86_64 101/153 2023-10-11T06:09:13.097 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:18.2.0-662.g9a774796.el9.x86_64 102/153 2023-10-11T06:09:15.466 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 103/153 2023-10-11T06:09:15.510 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 103/153 2023-10-11T06:09:16.161 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 103/153 2023-10-11T06:09:16.213 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 104/153 2023-10-11T06:09:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 104/153 2023-10-11T06:09:16.373 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-11T06:09:16.373 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-11T06:09:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:16.430 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 105/153 2023-10-11T06:09:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 105/153 2023-10-11T06:09:24.697 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2023-10-11T06:09:24.697 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2023-10-11T06:09:24.697 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2023-10-11T06:09:24.697 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2023-10-11T06:09:24.697 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2023-10-11T06:09:24.698 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2023-10-11T06:09:24.698 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2023-10-11T06:09:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:24.948 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 106/153 2023-10-11T06:09:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 106/153 2023-10-11T06:09:25.001 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:25.001 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-11T06:09:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:26.760 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 107/153 2023-10-11T06:09:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 107/153 2023-10-11T06:09:26.839 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:26.840 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-11T06:09:26.840 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-11T06:09:26.840 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-11T06:09:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:27.085 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 108/153 2023-10-11T06:09:27.158 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 108/153 2023-10-11T06:09:27.158 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:27.158 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-11T06:09:27.158 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-11T06:09:27.159 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-11T06:09:27.159 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:27.669 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 109/153 2023-10-11T06:09:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 109/153 2023-10-11T06:09:27.750 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:27.750 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-11T06:09:27.750 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-11T06:09:27.750 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-11T06:09:27.750 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.2.0-662.g9a774796.el9.noarch 110/153 2023-10-11T06:09:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.2.0-662.g9a774796.el9.noarch 110/153 2023-10-11T06:09:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-662.g9a774796.el 111/153 2023-10-11T06:09:28.611 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-662.g9a774796.e 112/153 2023-10-11T06:09:28.649 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noar 113/153 2023-10-11T06:09:28.890 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 105/153 2023-10-11T06:09:28.891 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2023-10-11T06:09:28.891 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2023-10-11T06:09:28.891 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2023-10-11T06:09:28.891 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2023-10-11T06:09:28.892 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2023-10-11T06:09:28.892 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2023-10-11T06:09:28.892 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2023-10-11T06:09:28.892 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:28.981 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noar 113/153 2023-10-11T06:09:29.016 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.no 114/153 2023-10-11T06:09:29.098 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 106/153 2023-10-11T06:09:29.145 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 106/153 2023-10-11T06:09:29.145 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:29.145 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-11T06:09:29.146 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.no 114/153 2023-10-11T06:09:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-662.g9a77 115/153 2023-10-11T06:09:30.342 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-662.g9a77 115/153 2023-10-11T06:09:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-662.g9a774796.el9 116/153 2023-10-11T06:09:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 117/153 2023-10-11T06:09:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 117/153 2023-10-11T06:09:30.612 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:30.612 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-11T06:09:30.613 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-11T06:09:30.613 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-11T06:09:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 118/153 2023-10-11T06:09:30.707 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 118/153 2023-10-11T06:09:30.825 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 107/153 2023-10-11T06:09:30.924 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 107/153 2023-10-11T06:09:30.924 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:30.924 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-11T06:09:30.925 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-11T06:09:30.925 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-11T06:09:30.925 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:31.175 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 108/153 2023-10-11T06:09:31.249 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 108/153 2023-10-11T06:09:31.250 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:31.250 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-11T06:09:31.250 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-11T06:09:31.250 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-11T06:09:31.250 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.2.0-662.g9a774796.el9.x86_64 119/153 2023-10-11T06:09:31.529 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 120/153 2023-10-11T06:09:31.604 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 120/153 2023-10-11T06:09:31.604 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:31.604 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-11T06:09:31.605 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-11T06:09:31.605 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-11T06:09:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:31.654 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-662.g9a7747 121/153 2023-10-11T06:09:31.750 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-662.g9a7747 121/153 2023-10-11T06:09:31.751 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:31.751 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-11T06:09:31.751 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:31.776 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 109/153 2023-10-11T06:09:31.830 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 122/153 2023-10-11T06:09:31.881 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 109/153 2023-10-11T06:09:31.882 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:31.882 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-11T06:09:31.882 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-11T06:09:31.882 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-11T06:09:31.883 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:31.913 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 122/153 2023-10-11T06:09:31.913 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:31.913 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-11T06:09:31.913 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-11T06:09:31.914 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-11T06:09:31.914 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 123/153 2023-10-11T06:09:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 123/153 2023-10-11T06:09:32.373 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:32.373 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-11T06:09:32.373 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-11T06:09:32.374 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-11T06:09:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:32.503 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:18.2.0-662.g9a774796.el9.noarch 110/153 2023-10-11T06:09:32.546 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:18.2.0-662.g9a774796.el9.noarch 110/153 2023-10-11T06:09:32.601 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-662.g9a774796.el 111/153 2023-10-11T06:09:32.714 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-662.g9a774796.e 112/153 2023-10-11T06:09:32.748 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noar 113/153 2023-10-11T06:09:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noar 113/153 2023-10-11T06:09:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.no 114/153 2023-10-11T06:09:34.266 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.no 114/153 2023-10-11T06:09:34.290 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-662.g9a77 115/153 2023-10-11T06:09:34.431 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-662.g9a77 115/153 2023-10-11T06:09:34.601 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-662.g9a774796.el9 116/153 2023-10-11T06:09:34.626 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 117/153 2023-10-11T06:09:34.732 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 117/153 2023-10-11T06:09:34.732 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:34.732 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-11T06:09:34.733 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-11T06:09:34.733 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-11T06:09:34.733 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:34.785 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 118/153 2023-10-11T06:09:34.823 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 118/153 2023-10-11T06:09:35.666 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:18.2.0-662.g9a774796.el9.x86_64 119/153 2023-10-11T06:09:35.702 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 120/153 2023-10-11T06:09:35.777 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 120/153 2023-10-11T06:09:35.777 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:35.777 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-11T06:09:35.777 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-11T06:09:35.777 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-11T06:09:35.778 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:35.828 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-662.g9a7747 121/153 2023-10-11T06:09:35.903 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-662.g9a7747 121/153 2023-10-11T06:09:35.903 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:35.903 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-11T06:09:35.904 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:35.979 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 122/153 2023-10-11T06:09:36.060 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 122/153 2023-10-11T06:09:36.060 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:36.061 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-11T06:09:36.061 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-11T06:09:36.061 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-11T06:09:36.061 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:36.421 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 123/153 2023-10-11T06:09:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.2.0-662.g9a774796.el9.x86_64 124/153 2023-10-11T06:09:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.2.0-662.g9a774796.el9.x86_64 125/153 2023-10-11T06:09:36.496 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 123/153 2023-10-11T06:09:36.496 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-11T06:09:36.496 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-11T06:09:36.496 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-11T06:09:36.497 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-11T06:09:36.497 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.2.0-662.g9a774796.el9.x86_64 126/153 2023-10-11T06:09:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-11T06:09:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-11T06:09:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.2.0-662.g9a774796.el9.x86_64 129/153 2023-10-11T06:09:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-11T06:09:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-11T06:09:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.2.0-662.g9a774796.el9.x86_6 132/153 2023-10-11T06:09:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-11T06:09:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-11T06:09:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-11T06:09:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-11T06:09:38.243 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-11T06:09:38.288 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-11T06:09:38.348 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-11T06:09:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:18.2.0-662.g9a774796.el9.noarch 138/153 2023-10-11T06:09:39.793 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-11T06:09:39.901 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-11T06:09:39.949 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-11T06:09:40.078 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-11T06:09:40.113 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-11T06:09:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-11T06:09:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-11T06:09:40.270 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-11T06:09:40.301 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-11T06:09:40.334 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-11T06:09:40.367 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-11T06:09:40.405 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-11T06:09:40.405 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-11T06:09:40.518 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:18.2.0-662.g9a774796.el9.x86_64 124/153 2023-10-11T06:09:40.573 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:18.2.0-662.g9a774796.el9.x86_64 125/153 2023-10-11T06:09:40.632 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:18.2.0-662.g9a774796.el9.x86_64 126/153 2023-10-11T06:09:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-11T06:09:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-11T06:09:40.872 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-11T06:09:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-11T06:09:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-11T06:09:40.904 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-11T06:09:41.023 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-11T06:09:41.024 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-11T06:09:41.094 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:18.2.0-662.g9a774796.el9.x86_64 129/153 2023-10-11T06:09:41.143 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-11T06:09:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-11T06:09:41.180 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-11T06:09:41.209 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:18.2.0-662.g9a774796.el9.x86_6 132/153 2023-10-11T06:09:41.255 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-11T06:09:41.356 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-11T06:09:41.399 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-11T06:09:41.399 INFO:teuthology.orchestra.run.smithi099.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-11T06:09:41.400 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:41.428 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-11T06:09:41.465 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-11T06:09:42.426 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-11T06:09:42.471 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-11T06:09:42.529 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-11T06:09:42.597 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:18.2.0-662.g9a774796.el9.noarch 138/153 2023-10-11T06:09:43.970 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-11T06:09:44.069 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-11T06:09:44.115 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-11T06:09:44.246 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-11T06:09:44.287 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-11T06:09:44.324 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-11T06:09:44.324 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-11T06:09:44.511 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-11T06:09:44.541 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-11T06:09:44.574 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-11T06:09:44.607 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-11T06:09:44.645 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-11T06:09:44.646 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-11T06:09:44.990 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-11T06:09:44.990 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-11T06:09:45.129 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-11T06:09:45.159 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-11T06:09:45.160 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-11T06:09:45.331 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-11T06:09:45.454 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-11T06:09:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-11T06:09:45.719 INFO:teuthology.orchestra.run.smithi084.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-11T06:09:45.719 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:46.013 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.2.0-662.g9a774796.el9.x86_64 1/153 2023-10-11T06:09:46.014 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 2/153 2023-10-11T06:09:46.014 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 3/153 2023-10-11T06:09:46.014 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.2.0-662.g9a774796.el9.x86_64 4/153 2023-10-11T06:09:46.014 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-662.g9a7747 5/153 2023-10-11T06:09:46.015 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 6/153 2023-10-11T06:09:46.015 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 7/153 2023-10-11T06:09:46.015 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 8/153 2023-10-11T06:09:46.015 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 9/153 2023-10-11T06:09:46.015 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 10/153 2023-10-11T06:09:46.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 11/153 2023-10-11T06:09:46.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.2.0-662.g9a774796.el9.x86_64 12/153 2023-10-11T06:09:46.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 13/153 2023-10-11T06:09:46.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.2.0-662.g9a774796.el9.x86_6 14/153 2023-10-11T06:09:46.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 15/153 2023-10-11T06:09:46.017 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 16/153 2023-10-11T06:09:46.017 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.2.0-662.g9a774796.el9.x86_64 17/153 2023-10-11T06:09:46.017 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_ 18/153 2023-10-11T06:09:46.017 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.2.0-662.g9a774796.el9.x86_64 19/153 2023-10-11T06:09:46.017 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.2.0-662.g9a774796.el9 20/153 2023-10-11T06:09:46.017 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.2.0-662.g9a774796.el9.x 21/153 2023-10-11T06:09:46.018 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.2.0-662.g9a774796.el9.x86_64 22/153 2023-10-11T06:09:46.018 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.2.0-662.g9a774796.el9.x86_64 23/153 2023-10-11T06:09:46.018 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.2.0-662.g9a774796.el9.x86_64 24/153 2023-10-11T06:09:46.018 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.2.0-662.g9a774796.el9.x86_64 25/153 2023-10-11T06:09:46.019 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.2.0-662.g9a774796.el9.x86_64 26/153 2023-10-11T06:09:46.019 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 27/153 2023-10-11T06:09:46.019 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.2.0-662.g9a774796.el9.x86_64 28/153 2023-10-11T06:09:46.019 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-662.g9a774796.e 29/153 2023-10-11T06:09:46.019 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noar 30/153 2023-10-11T06:09:46.019 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.no 31/153 2023-10-11T06:09:46.020 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-662.g9a77 32/153 2023-10-11T06:09:46.020 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-662.g9a774796.el9 33/153 2023-10-11T06:09:46.020 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 34/153 2023-10-11T06:09:46.020 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-662.g9a774796.el 35/153 2023-10-11T06:09:46.020 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 36/153 2023-10-11T06:09:46.021 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.2.0-662.g9a774796.el9.noarch 37/153 2023-10-11T06:09:46.021 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:18.2.0-662.g9a774796.el9.noarch 38/153 2023-10-11T06:09:46.021 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-11T06:09:46.021 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-11T06:09:46.021 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-11T06:09:46.022 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-11T06:09:46.022 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-11T06:09:46.022 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-11T06:09:46.022 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-11T06:09:46.022 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-11T06:09:46.023 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-11T06:09:46.023 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-11T06:09:46.023 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-11T06:09:46.023 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-11T06:09:46.023 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-11T06:09:46.024 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-11T06:09:46.024 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-11T06:09:46.024 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-11T06:09:46.024 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-11T06:09:46.024 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-11T06:09:46.024 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-11T06:09:46.025 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-11T06:09:46.025 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-11T06:09:46.025 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-11T06:09:46.025 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-11T06:09:46.025 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-11T06:09:46.026 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-11T06:09:46.026 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-11T06:09:46.026 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-11T06:09:46.026 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-11T06:09:46.026 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-11T06:09:46.027 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-11T06:09:46.027 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-11T06:09:46.027 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-11T06:09:46.027 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-11T06:09:46.027 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-11T06:09:46.027 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-11T06:09:46.028 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-11T06:09:46.028 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-11T06:09:46.028 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-11T06:09:46.028 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-11T06:09:46.028 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-11T06:09:46.028 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-11T06:09:46.029 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-11T06:09:46.029 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-11T06:09:46.029 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-11T06:09:46.029 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-11T06:09:46.029 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-11T06:09:46.030 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-11T06:09:46.030 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-11T06:09:46.030 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-11T06:09:46.030 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-11T06:09:46.030 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-11T06:09:46.031 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-11T06:09:46.031 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-11T06:09:46.031 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-11T06:09:46.031 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-11T06:09:46.031 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-11T06:09:46.032 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-11T06:09:46.032 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-11T06:09:46.032 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-11T06:09:46.032 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-11T06:09:46.032 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-11T06:09:46.032 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-11T06:09:46.033 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-11T06:09:46.033 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-11T06:09:46.033 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-11T06:09:46.033 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-11T06:09:46.033 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-11T06:09:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-11T06:09:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-11T06:09:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-11T06:09:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-11T06:09:46.034 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-11T06:09:46.035 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-11T06:09:46.035 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-11T06:09:46.035 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-11T06:09:46.035 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-11T06:09:46.035 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-11T06:09:46.035 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-11T06:09:46.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-11T06:09:46.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-11T06:09:46.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-11T06:09:46.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-11T06:09:46.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-11T06:09:46.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-11T06:09:46.037 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-11T06:09:46.037 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-11T06:09:46.037 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-11T06:09:46.037 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-11T06:09:46.037 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-11T06:09:46.038 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-11T06:09:46.038 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-11T06:09:46.038 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-11T06:09:46.038 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-11T06:09:46.038 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-11T06:09:46.038 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-11T06:09:46.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.2.0-662.g9a774796.el9.x86_64 134/153 2023-10-11T06:09:46.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-11T06:09:46.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.2.0-662.g9a774796.el9.x86_64 136/153 2023-10-11T06:09:46.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-11T06:09:46.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-11T06:09:46.040 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-11T06:09:46.040 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-11T06:09:46.040 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-11T06:09:46.040 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-11T06:09:46.040 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-11T06:09:46.040 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-11T06:09:46.041 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-11T06:09:46.041 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-11T06:09:46.041 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-11T06:09:46.041 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-11T06:09:46.041 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-11T06:09:46.041 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-11T06:09:46.042 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-11T06:09:46.042 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-11T06:09:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-11T06:09:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:48.108 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2023-10-11T06:09:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: acl-2.3.1-4.el9.x86_64 2023-10-11T06:09:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-11T06:09:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-11T06:09:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-11T06:09:48.110 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2023-10-11T06:09:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2023-10-11T06:09:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2023-10-11T06:09:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-11T06:09:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-11T06:09:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2023-10-11T06:09:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-11T06:09:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-11T06:09:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-11T06:09:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-11T06:09:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-11T06:09:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-11T06:09:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-15.el9.x86_64 2023-10-11T06:09:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-11T06:09:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-11T06:09:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-11T06:09:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-11T06:09:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-11T06:09:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-11T06:09:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-11T06:09:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-11T06:09:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-11T06:09:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-11T06:09:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-11T06:09:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-11T06:09:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-11T06:09:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-11T06:09:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-11T06:09:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-11T06:09:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-11T06:09:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-11T06:09:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-11T06:09:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.11-1.el9.x86_64 2023-10-11T06:09:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-11T06:09:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-11T06:09:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-11T06:09:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-11T06:09:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-11T06:09:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-11T06:09:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-11T06:09:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-11T06:09:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-11T06:09:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-11T06:09:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-11T06:09:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-11T06:09:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-11T06:09:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-11T06:09:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-11T06:09:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-11T06:09:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-11T06:09:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-11T06:09:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-11T06:09:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-11T06:09:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-11T06:09:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-11T06:09:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-11T06:09:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-11T06:09:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-11T06:09:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-11T06:09:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-11T06:09:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-11T06:09:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-11T06:09:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-11T06:09:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-11T06:09:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-11T06:09:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-11T06:09:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-11T06:09:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2023-10-11T06:09:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-11T06:09:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-11T06:09:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-11T06:09:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-11T06:09:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-11T06:09:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-11T06:09:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-11T06:09:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-11T06:09:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-11T06:09:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-11T06:09:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-11T06:09:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-11T06:09:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-11T06:09:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-11T06:09:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-11T06:09:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-11T06:09:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-11T06:09:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-11T06:09:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-11T06:09:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-11T06:09:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-11T06:09:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-3.el9.x86_64 2023-10-11T06:09:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-11T06:09:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-11T06:09:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-11T06:09:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-11T06:09:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-11T06:09:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-11T06:09:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:48.136 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2023-10-11T06:09:48.639 DEBUG:teuthology.parallel:result is None 2023-10-11T06:09:50.157 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:18.2.0-662.g9a774796.el9.x86_64 1/153 2023-10-11T06:09:50.158 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 2/153 2023-10-11T06:09:50.158 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 3/153 2023-10-11T06:09:50.158 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:18.2.0-662.g9a774796.el9.x86_64 4/153 2023-10-11T06:09:50.158 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-662.g9a7747 5/153 2023-10-11T06:09:50.158 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 6/153 2023-10-11T06:09:50.159 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 7/153 2023-10-11T06:09:50.159 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 8/153 2023-10-11T06:09:50.159 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 9/153 2023-10-11T06:09:50.159 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 10/153 2023-10-11T06:09:50.159 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 11/153 2023-10-11T06:09:50.160 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:18.2.0-662.g9a774796.el9.x86_64 12/153 2023-10-11T06:09:50.160 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 13/153 2023-10-11T06:09:50.160 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:18.2.0-662.g9a774796.el9.x86_6 14/153 2023-10-11T06:09:50.160 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 15/153 2023-10-11T06:09:50.160 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 16/153 2023-10-11T06:09:50.161 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:18.2.0-662.g9a774796.el9.x86_64 17/153 2023-10-11T06:09:50.161 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_ 18/153 2023-10-11T06:09:50.161 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:18.2.0-662.g9a774796.el9.x86_64 19/153 2023-10-11T06:09:50.161 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:18.2.0-662.g9a774796.el9 20/153 2023-10-11T06:09:50.161 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:18.2.0-662.g9a774796.el9.x 21/153 2023-10-11T06:09:50.162 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:18.2.0-662.g9a774796.el9.x86_64 22/153 2023-10-11T06:09:50.162 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:18.2.0-662.g9a774796.el9.x86_64 23/153 2023-10-11T06:09:50.162 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:18.2.0-662.g9a774796.el9.x86_64 24/153 2023-10-11T06:09:50.162 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:18.2.0-662.g9a774796.el9.x86_64 25/153 2023-10-11T06:09:50.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:18.2.0-662.g9a774796.el9.x86_64 26/153 2023-10-11T06:09:50.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 27/153 2023-10-11T06:09:50.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:18.2.0-662.g9a774796.el9.x86_64 28/153 2023-10-11T06:09:50.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-662.g9a774796.e 29/153 2023-10-11T06:09:50.173 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noar 30/153 2023-10-11T06:09:50.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.no 31/153 2023-10-11T06:09:50.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-662.g9a77 32/153 2023-10-11T06:09:50.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-662.g9a774796.el9 33/153 2023-10-11T06:09:50.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 34/153 2023-10-11T06:09:50.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-662.g9a774796.el 35/153 2023-10-11T06:09:50.175 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 36/153 2023-10-11T06:09:50.175 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:18.2.0-662.g9a774796.el9.noarch 37/153 2023-10-11T06:09:50.175 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:18.2.0-662.g9a774796.el9.noarch 38/153 2023-10-11T06:09:50.175 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-11T06:09:50.175 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-11T06:09:50.176 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-11T06:09:50.176 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-11T06:09:50.176 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-11T06:09:50.176 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-11T06:09:50.176 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-11T06:09:50.177 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-11T06:09:50.177 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-11T06:09:50.177 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-11T06:09:50.177 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-11T06:09:50.177 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-11T06:09:50.177 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-11T06:09:50.178 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-11T06:09:50.178 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-11T06:09:50.178 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-11T06:09:50.178 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-11T06:09:50.178 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-11T06:09:50.179 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-11T06:09:50.179 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-11T06:09:50.179 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-11T06:09:50.179 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-11T06:09:50.179 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-11T06:09:50.179 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-11T06:09:50.180 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-11T06:09:50.180 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-11T06:09:50.180 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-11T06:09:50.180 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-11T06:09:50.180 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-11T06:09:50.180 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-11T06:09:50.181 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-11T06:09:50.181 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-11T06:09:50.181 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-11T06:09:50.181 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-11T06:09:50.181 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-11T06:09:50.181 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-11T06:09:50.182 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-11T06:09:50.182 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-11T06:09:50.182 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-11T06:09:50.182 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-11T06:09:50.182 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-11T06:09:50.183 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-11T06:09:50.183 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-11T06:09:50.183 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-11T06:09:50.183 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-11T06:09:50.183 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-11T06:09:50.183 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-11T06:09:50.184 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-11T06:09:50.184 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-11T06:09:50.184 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-11T06:09:50.184 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-11T06:09:50.184 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-11T06:09:50.184 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-11T06:09:50.185 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-11T06:09:50.185 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-11T06:09:50.185 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-11T06:09:50.185 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-11T06:09:50.185 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-11T06:09:50.186 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-11T06:09:50.186 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-11T06:09:50.186 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-11T06:09:50.186 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-11T06:09:50.187 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-11T06:09:50.187 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-11T06:09:50.187 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-11T06:09:50.187 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-11T06:09:50.187 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-11T06:09:50.187 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-11T06:09:50.188 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-11T06:09:50.188 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-11T06:09:50.188 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-11T06:09:50.188 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-11T06:09:50.188 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-11T06:09:50.189 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-11T06:09:50.189 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-11T06:09:50.189 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-11T06:09:50.189 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-11T06:09:50.189 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-11T06:09:50.189 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-11T06:09:50.190 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-11T06:09:50.190 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-11T06:09:50.190 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-11T06:09:50.190 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-11T06:09:50.191 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-11T06:09:50.191 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-11T06:09:50.191 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-11T06:09:50.191 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-11T06:09:50.191 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-11T06:09:50.191 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-11T06:09:50.192 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-11T06:09:50.192 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-11T06:09:50.192 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-11T06:09:50.192 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-11T06:09:50.193 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-11T06:09:50.193 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-11T06:09:50.193 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:18.2.0-662.g9a774796.el9.x86_64 134/153 2023-10-11T06:09:50.193 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-11T06:09:50.193 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:18.2.0-662.g9a774796.el9.x86_64 136/153 2023-10-11T06:09:50.194 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-11T06:09:50.194 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-11T06:09:50.194 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-11T06:09:50.194 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-11T06:09:50.194 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-11T06:09:50.195 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-11T06:09:50.195 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-11T06:09:50.195 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-11T06:09:50.195 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-11T06:09:50.195 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-11T06:09:50.196 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-11T06:09:50.196 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-11T06:09:50.196 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-11T06:09:50.196 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-11T06:09:50.196 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-11T06:09:50.197 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-11T06:09:52.233 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-11T06:09:52.234 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:52.234 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2023-10-11T06:09:52.234 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:52.234 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:52.235 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:52.235 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-11T06:09:52.235 INFO:teuthology.orchestra.run.smithi084.stdout: acl-2.3.1-4.el9.x86_64 2023-10-11T06:09:52.235 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-11T06:09:52.235 INFO:teuthology.orchestra.run.smithi084.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-11T06:09:52.236 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.236 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.236 INFO:teuthology.orchestra.run.smithi084.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-11T06:09:52.236 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2023-10-11T06:09:52.236 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.2-8.el9.noarch 2023-10-11T06:09:52.237 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.7.4-5.el9.x86_64 2023-10-11T06:09:52.237 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-11T06:09:52.237 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.237 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.237 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.238 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.238 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.238 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.238 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.238 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.239 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.239 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.239 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.239 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.239 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.240 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.240 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.240 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.240 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.240 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.240 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.241 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.241 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.241 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.241 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:18.2.0-662.g9a774796.el9.noarch 2023-10-11T06:09:52.241 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-11T06:09:52.242 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.4-9.el9.x86_64 2023-10-11T06:09:52.242 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-11T06:09:52.242 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-11T06:09:52.242 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-11T06:09:52.242 INFO:teuthology.orchestra.run.smithi084.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-11T06:09:52.242 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-11T06:09:52.243 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-11T06:09:52.243 INFO:teuthology.orchestra.run.smithi084.stdout: jq-1.6-15.el9.x86_64 2023-10-11T06:09:52.243 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-11T06:09:52.243 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-11T06:09:52.243 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-11T06:09:52.244 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.244 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.244 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.244 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-11T06:09:52.244 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-11T06:09:52.245 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-11T06:09:52.245 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-11T06:09:52.245 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-11T06:09:52.245 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.245 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.246 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-11T06:09:52.246 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.246 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-11T06:09:52.246 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-11T06:09:52.246 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-11T06:09:52.247 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-11T06:09:52.247 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-11T06:09:52.247 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-11T06:09:52.247 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-11T06:09:52.247 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-11T06:09:52.248 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-11T06:09:52.248 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-11T06:09:52.248 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-13.11-1.el9.x86_64 2023-10-11T06:09:52.248 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-11T06:09:52.248 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-11T06:09:52.248 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-11T06:09:52.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-11T06:09:52.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-11T06:09:52.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-11T06:09:52.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-11T06:09:52.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-11T06:09:52.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-11T06:09:52.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-11T06:09:52.251 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-11T06:09:52.251 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-11T06:09:52.251 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-11T06:09:52.251 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-11T06:09:52.251 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-11T06:09:52.252 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-11T06:09:52.252 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-11T06:09:52.252 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-11T06:09:52.252 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-11T06:09:52.252 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-11T06:09:52.253 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-11T06:09:52.253 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-11T06:09:52.253 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-11T06:09:52.253 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-11T06:09:52.253 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-11T06:09:52.254 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-11T06:09:52.254 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-11T06:09:52.254 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-11T06:09:52.254 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-11T06:09:52.254 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-11T06:09:52.255 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-11T06:09:52.255 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-11T06:09:52.255 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-11T06:09:52.255 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-11T06:09:52.255 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2023-10-11T06:09:52.256 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-11T06:09:52.256 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-11T06:09:52.256 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-11T06:09:52.256 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-11T06:09:52.256 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-11T06:09:52.257 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-11T06:09:52.257 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-11T06:09:52.257 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.257 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.257 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-11T06:09:52.258 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-11T06:09:52.258 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-11T06:09:52.258 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.258 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-11T06:09:52.258 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-11T06:09:52.259 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-11T06:09:52.259 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-11T06:09:52.259 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-11T06:09:52.259 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-11T06:09:52.259 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-11T06:09:52.260 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-11T06:09:52.260 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-11T06:09:52.260 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-11T06:09:52.260 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-11T06:09:52.260 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.261 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.261 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:18.2.0-662.g9a774796.el9.x86_64 2023-10-11T06:09:52.261 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-3.el9.x86_64 2023-10-11T06:09:52.261 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-11T06:09:52.261 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-11T06:09:52.262 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-11T06:09:52.262 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-11T06:09:52.262 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-11T06:09:52.263 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-11T06:09:52.263 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:52.263 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2023-10-11T06:09:52.750 DEBUG:teuthology.parallel:result is None 2023-10-11T06:09:52.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T06:09:52.895 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-11T06:09:52.959 INFO:teuthology.orchestra.run.smithi084.stdout:18.2.0-662.g9a774796.el9 2023-10-11T06:09:52.960 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-662.g9a774796.el9 2023-10-11T06:09:52.960 INFO:teuthology.task.install:The correct ceph version 18.2.0-662.g9a774796 is installed. 2023-10-11T06:09:52.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a774796f0ecdd11a4b771de396e627aa1a187af 2023-10-11T06:09:53.096 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-11T06:09:53.170 INFO:teuthology.orchestra.run.smithi099.stdout:18.2.0-662.g9a774796.el9 2023-10-11T06:09:53.170 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-662.g9a774796.el9 2023-10-11T06:09:53.171 INFO:teuthology.task.install:The correct ceph version 18.2.0-662.g9a774796 is installed. 2023-10-11T06:09:53.171 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-11T06:09:53.172 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:53.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-11T06:09:53.253 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:53.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-11T06:09:53.361 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-11T06:09:53.362 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:53.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2023-10-11T06:09:53.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-11T06:09:53.559 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:53.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2023-10-11T06:09:53.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-11T06:09:53.764 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-11T06:09:53.765 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:53.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-11T06:09:53.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-11T06:09:53.986 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:53.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-11T06:09:54.088 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-11T06:09:54.232 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-11T06:09:54.232 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:54.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2023-10-11T06:09:54.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-11T06:09:54.415 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:54.415 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2023-10-11T06:09:54.516 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-11T06:09:54.615 INFO:teuthology.run_tasks:Running task ceph... 2023-10-11T06:09:54.675 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-11T06:09:54.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /var/log/ceph 2023-10-11T06:09:54.682 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2023-10-11T06:09:54.772 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-11T06:09:54.773 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-11T06:09:54.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-11T06:09:54.865 INFO:tasks.ceph:Creating extra log directories... 2023-10-11T06:09:54.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-11T06:09:54.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-11T06:09:54.973 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-11T06:09:54.974 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-11T06:09:54.974 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-11T06:09:54.975 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:54.975 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-11T06:09:55.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-11T06:09:55.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-11T06:09:55.312 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:55.312 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-11T06:09:55.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-11T06:09:55.650 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-11T06:09:55.760 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-11T06:09:55.760 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-11T06:09:55.760 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719', 'branch': 'wip-vshankar-testing-reef-20231009.131610', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7418719', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-reef-20231009.131610'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-reef-20231009.131610', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1116, 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af', 'sleep_before_teardown': 0, 'subset': '593/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-reef-20231009.131610', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi084.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIz07+vwo6enfYjYT3bgN9oAW/+WdhG0ul6F9y0AySoW1NTZ85wpDnIsN/oxucbA12mY5FlIxagz7oq58PLVr2s=', 'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO4Fkjn9MgszQq+O3mVjZjzL214/drNn9oJ1GIkUSVYEf036TMD8K0z7blUuOud3nbFZcubNP/Ios9UYGazh49c='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a774796f0ecdd11a4b771de396e627aa1a187af'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-09_16:48:27', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-11T06:09:55.761 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-11T06:09:55.770 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-11T06:09:55.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/run/ceph 2023-10-11T06:09:55.833 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2023-10-11T06:09:55.931 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:55.931 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2023-10-11T06:09:55.981 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-11T06:09:55.982 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2023-10-11T06:09:56.068 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-11T06:09:56.068 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:56.068 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-10-11T06:09:56.069 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:56.069 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:56.069 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-11 06:09:46.189449845 +0000 2023-10-11T06:09:56.069 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-11 06:07:22.903631688 +0000 2023-10-11T06:09:56.070 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-11 06:07:22.903631688 +0000 2023-10-11T06:09:56.070 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-11 06:07:22.903631688 +0000 2023-10-11T06:09:56.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-11T06:09:56.181 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-11T06:09:56.181 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-11T06:09:56.181 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000324857 s, 1.6 MB/s 2023-10-11T06:09:56.183 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-11T06:09:56.226 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2023-10-11T06:09:56.310 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-11T06:09:56.310 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:56.310 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-10-11T06:09:56.311 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:56.311 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:56.311 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-11 06:09:46.190449826 +0000 2023-10-11T06:09:56.311 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-11 06:07:23.903613254 +0000 2023-10-11T06:09:56.311 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-11 06:07:23.903613254 +0000 2023-10-11T06:09:56.311 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-11 06:07:23.903613254 +0000 2023-10-11T06:09:56.312 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-11T06:09:56.424 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-11T06:09:56.424 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-11T06:09:56.424 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000350731 s, 1.5 MB/s 2023-10-11T06:09:56.427 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-11T06:09:56.472 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2023-10-11T06:09:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-11T06:09:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:56.571 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-10-11T06:09:56.571 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:56.571 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:56.572 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-11 06:09:46.191449808 +0000 2023-10-11T06:09:56.572 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-11 06:07:24.943594082 +0000 2023-10-11T06:09:56.572 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-11 06:07:24.943594082 +0000 2023-10-11T06:09:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-11 06:07:24.943594082 +0000 2023-10-11T06:09:56.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-11T06:09:56.656 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-11T06:09:56.656 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-11T06:09:56.657 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000377554 s, 1.4 MB/s 2023-10-11T06:09:56.659 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-11T06:09:56.709 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2023-10-11T06:09:56.756 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-11T06:09:56.756 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:56.756 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-10-11T06:09:56.757 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:56.757 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:56.757 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-10-11 06:09:46.191449808 +0000 2023-10-11T06:09:56.757 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-10-11 06:07:25.986574855 +0000 2023-10-11T06:09:56.757 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-10-11 06:07:25.986574855 +0000 2023-10-11T06:09:56.757 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-10-11 06:07:25.986574855 +0000 2023-10-11T06:09:56.758 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-11T06:09:56.892 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-10-11T06:09:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-10-11T06:09:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000354006 s, 1.4 MB/s 2023-10-11T06:09:56.896 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-11T06:09:56.950 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T06:09:56.950 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:56.950 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2023-10-11T06:09:57.010 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-11T06:09:57.011 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2023-10-11T06:09:57.065 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-11T06:09:57.065 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:57.065 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-10-11T06:09:57.065 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:57.066 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:57.066 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-11 06:09:41.861810866 +0000 2023-10-11T06:09:57.066 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-11 06:07:22.899263691 +0000 2023-10-11T06:09:57.066 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-11 06:07:22.899263691 +0000 2023-10-11T06:09:57.067 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-10-11 06:07:22.899263691 +0000 2023-10-11T06:09:57.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-11T06:09:57.157 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-11T06:09:57.157 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-11T06:09:57.157 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000343141 s, 1.5 MB/s 2023-10-11T06:09:57.160 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-11T06:09:57.219 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2023-10-11T06:09:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-11T06:09:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:57.270 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-10-11T06:09:57.270 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:57.270 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:57.270 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-11 06:09:41.862810843 +0000 2023-10-11T06:09:57.270 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-11 06:07:23.875241276 +0000 2023-10-11T06:09:57.271 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-11 06:07:23.875241276 +0000 2023-10-11T06:09:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-10-11 06:07:23.875241276 +0000 2023-10-11T06:09:57.272 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-11T06:09:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-11T06:09:57.364 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-11T06:09:57.364 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000377931 s, 1.4 MB/s 2023-10-11T06:09:57.367 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-11T06:09:57.422 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2023-10-11T06:09:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-11T06:09:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:57.472 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 671 Links: 1 2023-10-11T06:09:57.472 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:57.472 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:57.472 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-11 06:09:41.862810843 +0000 2023-10-11T06:09:57.473 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-11 06:07:24.914217413 +0000 2023-10-11T06:09:57.473 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-11 06:07:24.914217413 +0000 2023-10-11T06:09:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-10-11 06:07:24.914217413 +0000 2023-10-11T06:09:57.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-11T06:09:57.558 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-11T06:09:57.558 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-11T06:09:57.558 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000316434 s, 1.6 MB/s 2023-10-11T06:09:57.561 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-11T06:09:57.616 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2023-10-11T06:09:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-11T06:09:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-11T06:09:57.661 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-10-11T06:09:57.662 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-11T06:09:57.662 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2023-10-11T06:09:57.662 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2023-10-11 06:09:41.864810797 +0000 2023-10-11T06:09:57.662 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2023-10-11 06:07:25.929194102 +0000 2023-10-11T06:09:57.663 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2023-10-11 06:07:25.929194102 +0000 2023-10-11T06:09:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2023-10-11 06:07:25.929194102 +0000 2023-10-11T06:09:57.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-11T06:09:57.788 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2023-10-11T06:09:57.789 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2023-10-11T06:09:57.789 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000326013 s, 1.6 MB/s 2023-10-11T06:09:57.792 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-11T06:09:57.846 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T06:09:57.846 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-11T06:09:57.846 INFO:tasks.ceph:Generating config... 2023-10-11T06:09:57.848 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-11T06:09:57.848 INFO:tasks.ceph:[client] debug client = 20 2023-10-11T06:09:57.848 INFO:tasks.ceph:[client] debug ms = 1 2023-10-11T06:09:57.848 INFO:tasks.ceph:[client] pid file = 2023-10-11T06:09:57.849 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-11T06:09:57.849 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-11T06:09:57.849 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-11T06:09:57.849 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-11T06:09:57.850 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-11T06:09:57.850 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-11T06:09:57.850 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-11T06:09:57.850 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-11T06:09:57.851 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-11T06:09:57.851 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-11T06:09:57.851 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-11T06:09:57.851 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-11T06:09:57.852 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-11T06:09:57.852 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-11T06:09:57.852 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-11T06:09:57.852 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-11T06:09:57.853 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-11T06:09:57.853 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-11T06:09:57.853 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-11T06:09:57.853 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-11T06:09:57.853 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-11T06:09:57.854 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-11T06:09:57.854 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-11T06:09:57.854 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-11T06:09:57.854 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-11T06:09:57.855 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-11T06:09:57.855 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-11T06:09:57.855 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-11T06:09:57.855 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-11T06:09:57.856 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-11T06:09:57.856 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-11T06:09:57.856 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-11T06:09:57.856 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-11T06:09:57.857 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-11T06:09:57.857 INFO:tasks.ceph:Setting up mon.a... 2023-10-11T06:09:57.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-11T06:09:58.011 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.keyring 2023-10-11T06:09:58.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-11T06:09:58.155 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-11T06:09:58.244 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.84'), ('mon.b', '172.21.15.99'), ('mon.c', '[v2:172.21.15.99:3301,v1:172.21.15.99:6790]')] 2023-10-11T06:09:58.244 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.84,172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-11T06:09:58.245 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:58.246 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-11T06:09:58.298 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.84 --add b 172.21.15.99 --addv c '[v2:172.21.15.99:3301,v1:172.21.15.99:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:09:58.396 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:09:58.396 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: generated fsid c84dc637-9184-44da-bf67-6cb4224adc5a 2023-10-11T06:09:58.396 INFO:teuthology.orchestra.run.smithi084.stdout:setting min_mon_release = pacific 2023-10-11T06:09:58.397 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 0 2023-10-11T06:09:58.397 INFO:teuthology.orchestra.run.smithi084.stdout:fsid c84dc637-9184-44da-bf67-6cb4224adc5a 2023-10-11T06:09:58.397 INFO:teuthology.orchestra.run.smithi084.stdout:last_changed 2023-10-11T06:09:58.395767+0000 2023-10-11T06:09:58.397 INFO:teuthology.orchestra.run.smithi084.stdout:created 2023-10-11T06:09:58.395767+0000 2023-10-11T06:09:58.397 INFO:teuthology.orchestra.run.smithi084.stdout:min_mon_release 16 (pacific) 2023-10-11T06:09:58.398 INFO:teuthology.orchestra.run.smithi084.stdout:election_strategy: 1 2023-10-11T06:09:58.398 INFO:teuthology.orchestra.run.smithi084.stdout:0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2023-10-11T06:09:58.398 INFO:teuthology.orchestra.run.smithi084.stdout:1: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2023-10-11T06:09:58.398 INFO:teuthology.orchestra.run.smithi084.stdout:2: [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] mon.c 2023-10-11T06:09:58.398 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-11T06:09:58.400 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-11T06:09:58.452 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c84dc637-9184-44da-bf67-6cb4224adc5a... 2023-10-11T06:09:58.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T06:09:58.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-11T06:09:58.618 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2023-10-11T06:09:58.619 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2023-10-11T06:09:58.619 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T06:09:58.619 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2023-10-11T06:09:58.620 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.620 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2023-10-11T06:09:58.620 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.620 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2023-10-11T06:09:58.620 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.621 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2023-10-11T06:09:58.621 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2023-10-11T06:09:58.621 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.621 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2023-10-11T06:09:58.621 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2023-10-11T06:09:58.622 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.622 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T06:09:58.622 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2023-10-11T06:09:58.622 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.622 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T06:09:58.623 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T06:09:58.623 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.623 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2023-10-11T06:09:58.623 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.623 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2023-10-11T06:09:58.624 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2023-10-11T06:09:58.624 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2023-10-11T06:09:58.624 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2023-10-11T06:09:58.624 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2023-10-11T06:09:58.624 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2023-10-11T06:09:58.625 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2023-10-11T06:09:58.625 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T06:09:58.625 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T06:09:58.625 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2023-10-11T06:09:58.625 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.625 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T06:09:58.626 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.626 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2023-10-11T06:09:58.626 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.626 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2023-10-11T06:09:58.626 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.627 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2023-10-11T06:09:58.627 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2023-10-11T06:09:58.627 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2023-10-11T06:09:58.627 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.84,172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790]" 2023-10-11T06:09:58.627 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = c84dc637-9184-44da-bf67-6cb4224adc5a 2023-10-11T06:09:58.628 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.628 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2023-10-11T06:09:58.628 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2023-10-11T06:09:58.628 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.628 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2023-10-11T06:09:58.628 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2023-10-11T06:09:58.629 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2023-10-11T06:09:58.629 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.629 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2023-10-11T06:09:58.629 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2023-10-11T06:09:58.629 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.630 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2023-10-11T06:09:58.630 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2023-10-11T06:09:58.630 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2023-10-11T06:09:58.630 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.631 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2023-10-11T06:09:58.631 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2023-10-11T06:09:58.631 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.631 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2023-10-11T06:09:58.631 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.631 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2023-10-11T06:09:58.632 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.632 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2023-10-11T06:09:58.632 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.632 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2023-10-11T06:09:58.632 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2023-10-11T06:09:58.633 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.633 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2023-10-11T06:09:58.633 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2023-10-11T06:09:58.633 INFO:teuthology.orchestra.run.smithi099.stdout: bdev async discard = True 2023-10-11T06:09:58.633 INFO:teuthology.orchestra.run.smithi099.stdout: bdev enable discard = True 2023-10-11T06:09:58.633 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore allocator = bitmap 2023-10-11T06:09:58.634 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2023-10-11T06:09:58.634 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2023-10-11T06:09:58.634 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2023-10-11T06:09:58.634 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2023-10-11T06:09:58.634 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-11T06:09:58.635 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2023-10-11T06:09:58.635 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2023-10-11T06:09:58.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T06:09:58.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2023-10-11T06:09:58.635 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2023-10-11T06:09:58.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2023-10-11T06:09:58.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2023-10-11T06:09:58.636 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2023-10-11T06:09:58.636 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.636 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2023-10-11T06:09:58.636 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-11T06:09:58.637 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2023-10-11T06:09:58.637 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2023-10-11T06:09:58.637 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2023-10-11T06:09:58.637 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2023-10-11T06:09:58.637 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2023-10-11T06:09:58.637 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2023-10-11T06:09:58.638 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.638 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2023-10-11T06:09:58.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-11T06:09:58.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2023-10-11T06:09:58.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2023-10-11T06:09:58.638 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2023-10-11T06:09:58.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2023-10-11T06:09:58.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2023-10-11T06:09:58.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2023-10-11T06:09:58.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2023-10-11T06:09:58.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2023-10-11T06:09:58.639 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2023-10-11T06:09:58.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.640 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T06:09:58.640 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T06:09:58.640 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T06:09:58.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.641 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T06:09:58.641 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T06:09:58.641 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T06:09:58.641 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.641 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2023-10-11T06:09:58.642 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2023-10-11T06:09:58.642 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.642 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2023-10-11T06:09:58.642 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2023-10-11T06:09:58.642 INFO:teuthology.orchestra.run.smithi099.stdout: 2023-10-11T06:09:58.643 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2023-10-11T06:09:58.643 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2023-10-11T06:09:58.643 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2023-10-11T06:09:58.643 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2023-10-11T06:09:58.643 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T06:09:58.644 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T06:09:58.644 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2023-10-11T06:09:58.644 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2023-10-11T06:09:58.644 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-11T06:09:58.644 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = "" 2023-10-11T06:09:58.644 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2023-10-11T06:09:58.645 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2023-10-11T06:09:58.645 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2023-10-11T06:09:58.645 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2023-10-11T06:09:58.645 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2023-10-11T06:09:58.645 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2023-10-11T06:09:58.645 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2023-10-11T06:09:58.646 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2023-10-11T06:09:58.646 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2023-10-11T06:09:58.646 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2023-10-11T06:09:58.646 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2023-10-11T06:09:58.646 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds balancer = 20 2023-10-11T06:09:58.647 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2023-10-11T06:09:58.647 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2023-10-11T06:09:58.647 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2023-10-11T06:09:58.647 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2023-10-11T06:09:58.647 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2023-10-11T06:09:58.648 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2023-10-11T06:09:58.648 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2023-10-11T06:09:58.648 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2023-10-11T06:09:58.648 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2023-10-11T06:09:58.649 INFO:teuthology.orchestra.run.smithi084.stdout: chdir = "" 2023-10-11T06:09:58.649 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-11T06:09:58.649 INFO:teuthology.orchestra.run.smithi084.stdout: auth supported = cephx 2023-10-11T06:09:58.649 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.650 INFO:teuthology.orchestra.run.smithi084.stdout: filestore xattr use omap = true 2023-10-11T06:09:58.650 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.650 INFO:teuthology.orchestra.run.smithi084.stdout: mon clock drift allowed = 1.000 2023-10-11T06:09:58.650 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.650 INFO:teuthology.orchestra.run.smithi084.stdout: osd crush chooseleaf type = 0 2023-10-11T06:09:58.651 INFO:teuthology.orchestra.run.smithi084.stdout: auth debug = true 2023-10-11T06:09:58.651 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.651 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on old message = true 2023-10-11T06:09:58.651 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on bug = true 2023-10-11T06:09:58.651 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.652 INFO:teuthology.orchestra.run.smithi084.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-11T06:09:58.652 INFO:teuthology.orchestra.run.smithi084.stdout: mon pg warn max object skew = 0 2023-10-11T06:09:58.652 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.652 INFO:teuthology.orchestra.run.smithi084.stdout: # disable pg_autoscaler by default for new pools 2023-10-11T06:09:58.652 INFO:teuthology.orchestra.run.smithi084.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-11T06:09:58.653 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.653 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default size = 2 2023-10-11T06:09:58.653 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.653 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow primary affinity = true 2023-10-11T06:09:58.653 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow pg remap = true 2023-10-11T06:09:58.653 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on legacy crush tunables = false 2023-10-11T06:09:58.654 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on crush straw calc version zero = false 2023-10-11T06:09:58.654 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on no sortbitwise = false 2023-10-11T06:09:58.654 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on osd down out interval zero = false 2023-10-11T06:09:58.654 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on too few osds = false 2023-10-11T06:09:58.655 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-11T06:09:58.655 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-11T06:09:58.655 INFO:teuthology.orchestra.run.smithi084.stdout: mon_allow_pool_size_one = true 2023-10-11T06:09:58.655 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.655 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-11T06:09:58.655 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.656 INFO:teuthology.orchestra.run.smithi084.stdout: osd default data pool replay window = 5 2023-10-11T06:09:58.656 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.656 INFO:teuthology.orchestra.run.smithi084.stdout: mon allow pool delete = true 2023-10-11T06:09:58.656 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.656 INFO:teuthology.orchestra.run.smithi084.stdout: mon cluster log file level = debug 2023-10-11T06:09:58.657 INFO:teuthology.orchestra.run.smithi084.stdout: debug asserts on shutdown = true 2023-10-11T06:09:58.657 INFO:teuthology.orchestra.run.smithi084.stdout: mon health detail to clog = false 2023-10-11T06:09:58.657 INFO:teuthology.orchestra.run.smithi084.stdout: mon host = "172.21.15.84,172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790]" 2023-10-11T06:09:58.657 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = c84dc637-9184-44da-bf67-6cb4224adc5a 2023-10-11T06:09:58.657 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.657 INFO:teuthology.orchestra.run.smithi084.stdout:[osd] 2023-10-11T06:09:58.658 INFO:teuthology.orchestra.run.smithi084.stdout: osd journal size = 100 2023-10-11T06:09:58.658 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.658 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub load threshold = 5.0 2023-10-11T06:09:58.658 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub max interval = 600 2023-10-11T06:09:58.658 INFO:teuthology.orchestra.run.smithi084.stdout: osd mclock profile = high_recovery_ops 2023-10-11T06:09:58.659 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.659 INFO:teuthology.orchestra.run.smithi084.stdout: osd recover clone overlap = true 2023-10-11T06:09:58.659 INFO:teuthology.orchestra.run.smithi084.stdout: osd recovery max chunk = 1048576 2023-10-11T06:09:58.659 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.659 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug shutdown = true 2023-10-11T06:09:58.659 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug op order = true 2023-10-11T06:09:58.660 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify stray on activate = true 2023-10-11T06:09:58.660 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.660 INFO:teuthology.orchestra.run.smithi084.stdout: osd open classes on start = true 2023-10-11T06:09:58.660 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug pg log writeout = true 2023-10-11T06:09:58.660 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.661 INFO:teuthology.orchestra.run.smithi084.stdout: osd deep scrub update digest min age = 30 2023-10-11T06:09:58.661 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.661 INFO:teuthology.orchestra.run.smithi084.stdout: osd map max advance = 10 2023-10-11T06:09:58.661 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.661 INFO:teuthology.orchestra.run.smithi084.stdout: journal zero on create = true 2023-10-11T06:09:58.662 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.662 INFO:teuthology.orchestra.run.smithi084.stdout: filestore ondisk finisher threads = 3 2023-10-11T06:09:58.662 INFO:teuthology.orchestra.run.smithi084.stdout: filestore apply finisher threads = 3 2023-10-11T06:09:58.662 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.662 INFO:teuthology.orchestra.run.smithi084.stdout: bdev debug aio = true 2023-10-11T06:09:58.663 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug misdirected ops = true 2023-10-11T06:09:58.665 INFO:teuthology.orchestra.run.smithi084.stdout: bdev async discard = True 2023-10-11T06:09:58.665 INFO:teuthology.orchestra.run.smithi084.stdout: bdev enable discard = True 2023-10-11T06:09:58.666 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore allocator = bitmap 2023-10-11T06:09:58.666 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore block size = 96636764160 2023-10-11T06:09:58.666 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore fsck on mount = True 2023-10-11T06:09:58.666 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluefs = 1/20 2023-10-11T06:09:58.667 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluestore = 1/20 2023-10-11T06:09:58.667 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-11T06:09:58.667 INFO:teuthology.orchestra.run.smithi084.stdout: debug osd = 20 2023-10-11T06:09:58.667 INFO:teuthology.orchestra.run.smithi084.stdout: debug rocksdb = 4/10 2023-10-11T06:09:58.667 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd backfillfull_ratio = 0.85 2023-10-11T06:09:58.668 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd full ratio = 0.9 2023-10-11T06:09:58.668 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd nearfull ratio = 0.8 2023-10-11T06:09:58.668 INFO:teuthology.orchestra.run.smithi084.stdout: osd failsafe full ratio = 0.95 2023-10-11T06:09:58.668 INFO:teuthology.orchestra.run.smithi084.stdout: osd objectstore = bluestore 2023-10-11T06:09:58.668 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2023-10-11T06:09:58.668 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.669 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr] 2023-10-11T06:09:58.669 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-11T06:09:58.669 INFO:teuthology.orchestra.run.smithi084.stdout: debug mgr = 20 2023-10-11T06:09:58.669 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2023-10-11T06:09:58.669 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2023-10-11T06:09:58.670 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2023-10-11T06:09:58.670 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2023-10-11T06:09:58.670 INFO:teuthology.orchestra.run.smithi084.stdout: mgr/telemetry/nag = false 2023-10-11T06:09:58.670 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.670 INFO:teuthology.orchestra.run.smithi084.stdout:[mon] 2023-10-11T06:09:58.671 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-11T06:09:58.671 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2023-10-11T06:09:58.671 INFO:teuthology.orchestra.run.smithi084.stdout: debug paxos = 20 2023-10-11T06:09:58.671 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2023-10-11T06:09:58.671 INFO:teuthology.orchestra.run.smithi084.stdout: mon data avail warn = 5 2023-10-11T06:09:58.672 INFO:teuthology.orchestra.run.smithi084.stdout: mon mgr mkfs grace = 240 2023-10-11T06:09:58.672 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2023-10-11T06:09:58.672 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd reporter subtree level = osd 2023-10-11T06:09:58.672 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd prime pg temp = true 2023-10-11T06:09:58.672 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2023-10-11T06:09:58.673 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.673 INFO:teuthology.orchestra.run.smithi084.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-11T06:09:58.673 INFO:teuthology.orchestra.run.smithi084.stdout: auth mon ticket ttl = 660 # 11m 2023-10-11T06:09:58.673 INFO:teuthology.orchestra.run.smithi084.stdout: auth service ticket ttl = 240 # 4m 2023-10-11T06:09:58.673 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.673 INFO:teuthology.orchestra.run.smithi084.stdout: # don't complain about insecure global_id in the test suite 2023-10-11T06:09:58.674 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-11T06:09:58.674 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-11T06:09:58.674 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.674 INFO:teuthology.orchestra.run.smithi084.stdout: # 1m isn't quite enough 2023-10-11T06:09:58.674 INFO:teuthology.orchestra.run.smithi084.stdout: mon_down_mkfs_grace = 2m 2023-10-11T06:09:58.675 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.675 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_filestore_osds = false 2023-10-11T06:09:58.675 INFO:teuthology.orchestra.run.smithi084.stdout: mon op complaint time = 120 2023-10-11T06:09:58.675 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:09:58.675 INFO:teuthology.orchestra.run.smithi084.stdout:[client] 2023-10-11T06:09:58.676 INFO:teuthology.orchestra.run.smithi084.stdout: rgw cache enabled = true 2023-10-11T06:09:58.676 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable ops log = true 2023-10-11T06:09:58.676 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable usage log = true 2023-10-11T06:09:58.676 INFO:teuthology.orchestra.run.smithi084.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-11T06:09:58.676 INFO:teuthology.orchestra.run.smithi084.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-11T06:09:58.676 INFO:teuthology.orchestra.run.smithi084.stdout: client mount timeout = 600 2023-10-11T06:09:58.677 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2023-10-11T06:09:58.677 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-11T06:09:58.677 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = "" 2023-10-11T06:09:58.677 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2023-10-11T06:09:58.677 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2023-10-11T06:09:58.678 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.a] 2023-10-11T06:09:58.678 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.b] 2023-10-11T06:09:58.678 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.c] 2023-10-11T06:09:58.678 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2023-10-11T06:09:58.678 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2023-10-11T06:09:58.679 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2023-10-11T06:09:58.679 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2023-10-11T06:09:58.679 INFO:teuthology.orchestra.run.smithi084.stdout:[mds] 2023-10-11T06:09:58.679 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds = 20 2023-10-11T06:09:58.679 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds balancer = 20 2023-10-11T06:09:58.680 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-10-11T06:09:58.680 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug frag = True 2023-10-11T06:09:58.680 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug scatterstat = True 2023-10-11T06:09:58.680 INFO:teuthology.orchestra.run.smithi084.stdout: mds op complaint time = 180 2023-10-11T06:09:58.680 INFO:teuthology.orchestra.run.smithi084.stdout: mds verify scatter = True 2023-10-11T06:09:58.680 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2023-10-11T06:09:58.681 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2023-10-11T06:09:58.681 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2023-10-11T06:09:58.682 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-11T06:09:58.683 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-11T06:09:58.836 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-11T06:09:58.836 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:58.836 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-11T06:09:58.889 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:58.890 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-11T06:09:58.946 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:09:58.946 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:58.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-11T06:09:58.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-11T06:09:59.094 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:09:59.094 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:09:59.191 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:09:59.192 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:59.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-11T06:09:59.193 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-11T06:09:59.347 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:09:59.348 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:09:59.448 INFO:tasks.ceph:Setting up mon nodes... 2023-10-11T06:09:59.449 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-11T06:09:59.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-11T06:09:59.644 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-11T06:09:59.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-11T06:09:59.857 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-11T06:09:59.868 INFO:tasks.ceph:Setting up mds nodes... 2023-10-11T06:09:59.869 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-11T06:10:00.069 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-11T06:10:00.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-11T06:10:00.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-11T06:10:00.337 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-11T06:10:00.346 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-11T06:10:00.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-11T06:10:00.637 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-11T06:10:00.646 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-11T06:10:00.741 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-11T06:10:00.923 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-11T06:10:00.929 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-11T06:10:01.028 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-11T06:10:01.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-11T06:10:01.163 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-11T06:10:01.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-11T06:10:01.335 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-11T06:10:01.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-11T06:10:01.521 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-11T06:10:01.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-11T06:10:01.702 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-11T06:10:01.760 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-11T06:10:01.760 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-11T06:10:01.761 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-11T06:10:01.851 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:01.852 INFO:tasks.ceph:role: osd.0 2023-10-11T06:10:01.852 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:10:01.852 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-11T06:10:01.946 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:01.950 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:01.950 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:01.950 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:01.951 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:02.444 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-11T06:10:02.449 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-11T06:10:02.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-11T06:10:02.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-11T06:10:02.677 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-11T06:10:02.762 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:02.763 INFO:tasks.ceph:role: osd.1 2023-10-11T06:10:02.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:10:02.764 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-11T06:10:02.852 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:02.852 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:02.853 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:02.853 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:02.853 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:02.853 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:02.853 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:02.854 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:02.854 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:02.854 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:03.312 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-11T06:10:03.319 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-11T06:10:03.319 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-11T06:10:03.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-11T06:10:03.539 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-11T06:10:03.633 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:03.634 INFO:tasks.ceph:role: osd.2 2023-10-11T06:10:03.634 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:10:03.634 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-11T06:10:03.738 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:03.739 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:03.739 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:03.739 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:03.740 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:03.740 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:03.740 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:03.740 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:03.740 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:03.741 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:04.237 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-11T06:10:04.243 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-11T06:10:04.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-11T06:10:04.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-11T06:10:04.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-11T06:10:04.591 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:04.592 INFO:tasks.ceph:role: osd.3 2023-10-11T06:10:04.592 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:10:04.593 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-11T06:10:04.694 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:04.695 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:04.695 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:04.695 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:04.695 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:04.695 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:04.696 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:04.696 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:04.696 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:04.696 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:05.147 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-10-11T06:10:05.153 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-10-11T06:10:05.153 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-11T06:10:05.299 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-11T06:10:05.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:05.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:05.548+0000 7f451d80c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-11T06:10:05.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:05.551+0000 7f451d80c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-11T06:10:05.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:05.551+0000 7f451d80c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-11T06:10:05.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:05.551+0000 7f451d80c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-11T06:10:05.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:05.551+0000 7f451d80c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-11T06:10:05.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:05.560+0000 7f451d80c500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-11T06:10:08.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-11T06:10:08.595 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:08.738+0000 7f30d7fbe500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-11T06:10:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:08.738+0000 7f30d7fbe500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-11T06:10:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:08.739+0000 7f30d7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-11T06:10:08.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:08.739+0000 7f30d7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-11T06:10:08.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:08.739+0000 7f30d7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-11T06:10:08.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:08.741+0000 7f30d7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-11T06:10:11.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-11T06:10:11.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:11.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:11.891+0000 7f9a9d80f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-11T06:10:11.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:11.891+0000 7f9a9d80f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-11T06:10:11.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:11.892+0000 7f9a9d80f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-11T06:10:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:11.892+0000 7f9a9d80f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-11T06:10:11.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:11.892+0000 7f9a9d80f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-11T06:10:11.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:11.894+0000 7f9a9d80f500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-11T06:10:14.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-11T06:10:14.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:15.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:15.073+0000 7f88158b6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-11T06:10:15.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:15.074+0000 7f88158b6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-11T06:10:15.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:15.074+0000 7f88158b6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-11T06:10:15.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:15.074+0000 7f88158b6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-11T06:10:15.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:15.074+0000 7f88158b6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-11T06:10:15.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:15.076+0000 7f88158b6500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-11T06:10:17.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-11T06:10:18.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-11T06:10:18.185 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:18.185 INFO:tasks.ceph:role: osd.4 2023-10-11T06:10:18.186 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:10:18.186 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-11T06:10:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:18.287 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:18.287 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:18.288 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:18.288 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:18.806 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-11T06:10:18.813 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-11T06:10:18.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-11T06:10:19.016 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-11T06:10:19.124 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-11T06:10:19.215 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:19.216 INFO:tasks.ceph:role: osd.5 2023-10-11T06:10:19.216 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:10:19.216 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-11T06:10:19.319 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:19.320 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:19.320 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:19.320 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:19.321 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:19.321 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:19.321 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:19.321 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:19.322 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:19.322 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:19.813 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-11T06:10:19.819 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-11T06:10:19.820 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-11T06:10:19.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-11T06:10:20.079 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-11T06:10:20.168 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:20.169 INFO:tasks.ceph:role: osd.6 2023-10-11T06:10:20.169 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:10:20.169 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-11T06:10:20.267 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:20.268 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:20.268 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:20.268 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:20.269 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:20.785 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-11T06:10:20.791 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-11T06:10:20.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-11T06:10:20.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-11T06:10:21.090 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-11T06:10:21.184 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-11T06:10:21.184 INFO:tasks.ceph:role: osd.7 2023-10-11T06:10:21.184 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:10:21.185 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-11T06:10:21.283 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-11T06:10:21.283 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-11T06:10:21.283 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-11T06:10:21.283 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-11T06:10:21.284 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-11T06:10:21.284 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2023-10-11T06:10:21.284 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-11T06:10:21.284 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-11T06:10:21.284 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-11T06:10:21.284 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-11T06:10:21.742 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2023-10-11T06:10:21.749 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2023-10-11T06:10:21.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-11T06:10:21.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-11T06:10:21.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:22.138 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:22.136+0000 7f79eedad500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-11T06:10:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:22.137+0000 7f79eedad500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-11T06:10:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:22.138+0000 7f79eedad500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-11T06:10:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:22.138+0000 7f79eedad500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-11T06:10:22.140 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:22.138+0000 7f79eedad500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-11T06:10:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:22.141+0000 7f79eedad500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-11T06:10:25.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-11T06:10:25.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:25.315 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:25.313+0000 7fc14320f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-11T06:10:25.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:25.314+0000 7fc14320f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-11T06:10:25.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:25.314+0000 7fc14320f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-11T06:10:25.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:25.314+0000 7fc14320f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-11T06:10:25.316 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:25.314+0000 7fc14320f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-11T06:10:25.317 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:25.316+0000 7fc14320f500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-11T06:10:25.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:10:25.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:10:28.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-11T06:10:28.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:28.485 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:28.484+0000 7f348b9fe500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-11T06:10:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:28.484+0000 7f348b9fe500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-11T06:10:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:28.484+0000 7f348b9fe500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-11T06:10:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:28.484+0000 7f348b9fe500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-11T06:10:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:28.484+0000 7f348b9fe500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-11T06:10:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:28.486+0000 7f348b9fe500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-11T06:10:31.671 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-11T06:10:31.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:31.897 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:31.896+0000 7f7bbb238500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-11T06:10:31.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:31.896+0000 7f7bbb238500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-11T06:10:31.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:31.896+0000 7f7bbb238500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-11T06:10:31.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:31.896+0000 7f7bbb238500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-11T06:10:31.898 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:31.896+0000 7f7bbb238500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-11T06:10:31.899 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:31.898+0000 7f7bbb238500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-11T06:10:34.824 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-11T06:10:34.919 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-11T06:10:34.920 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:34.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-11T06:10:35.025 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:35.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-11T06:10:35.119 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:35.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-11T06:10:35.215 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:35.215 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-11T06:10:35.309 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:35.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-11T06:10:35.396 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:35.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-11T06:10:35.485 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:35.485 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-11T06:10:35.627 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:35.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-11T06:10:35.731 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:35.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-11T06:10:35.832 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:35.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-11T06:10:35.929 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:35.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-11T06:10:36.027 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:36.027 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-11T06:10:36.122 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:36.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-11T06:10:36.215 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:36.216 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-11T06:10:36.307 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:36.307 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-11T06:10:36.368 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:36.369 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-11T06:10:36.423 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:36.423 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-11T06:10:36.482 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:36.483 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-11T06:10:36.533 INFO:tasks.ceph:Adding keys to all mons... 2023-10-11T06:10:36.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-11T06:10:36.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-11T06:10:36.627 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2023-10-11T06:10:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA3PCZlcQtlJhAAyb6WWJKJgbNLkVkNL36ttg== 2023-10-11T06:10:36.628 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2023-10-11T06:10:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA4PCZl2TIZBBAAHTDKquuoZy+2HB2lk815Fg== 2023-10-11T06:10:36.628 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2023-10-11T06:10:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA4PCZlj+QWFBAAw6XDEihH6iXYbQsLht/VsQ== 2023-10-11T06:10:36.629 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2023-10-11T06:10:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA9PCZljp29IBAAFJgwhWwcw3OWMIDeeXR2sw== 2023-10-11T06:10:36.629 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2023-10-11T06:10:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBAPCZlBCAQLBAAeWQVnN/FYimDHxX1nYzVKA== 2023-10-11T06:10:36.629 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2023-10-11T06:10:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBDPCZlu1o+NRAAjAauNPRb88x0p37gT88vBw== 2023-10-11T06:10:36.630 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2023-10-11T06:10:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBHPCZlcy98BBAALniVjO3ojVM/+q9fw6tNsw== 2023-10-11T06:10:36.630 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2023-10-11T06:10:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA3PCZlsakeMxAA52n80muv9FKTIvML6txp5w== 2023-10-11T06:10:36.631 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2023-10-11T06:10:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA4PCZlaZYAJhAAzmJNe7HYcmHJu+7Fbfv6dw== 2023-10-11T06:10:36.631 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2023-10-11T06:10:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA4PCZlgvsCNxAAJL7HKUMYQA6zG6useZDoQw== 2023-10-11T06:10:36.631 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2023-10-11T06:10:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBOPCZlmds+CBAAb0nHqugOzMsqLZCUabaHGQ== 2023-10-11T06:10:36.632 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2023-10-11T06:10:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBRPCZlPObDEhAA1yMeENf3iV3lQf6f6WM8UQ== 2023-10-11T06:10:36.632 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2023-10-11T06:10:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBUPCZlZQvrHBAAc/hx1KwJuGfHv5M+Tobkjg== 2023-10-11T06:10:36.633 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2023-10-11T06:10:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBXPCZlKKt5NRAAkM8aLRyZPYaG0n1AH2tDIw== 2023-10-11T06:10:36.633 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2023-10-11T06:10:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA5PCZlP/K+CRAAYT5I5tAyH8nHrsBReUAL9A== 2023-10-11T06:10:36.633 INFO:teuthology.orchestra.run.smithi099.stdout:[client.2] 2023-10-11T06:10:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA5PCZli1j2ExAA77o/7VayFJb+C/TfdZrxRg== 2023-10-11T06:10:36.634 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2023-10-11T06:10:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA5PCZlzhkWHxAAMV2BVukY7a/l7AXFNPCMhg== 2023-10-11T06:10:36.634 INFO:teuthology.orchestra.run.smithi099.stdout:[client.3] 2023-10-11T06:10:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA5PCZljnzfKRAA2s7Yg5NkKNp/LP1eWIBEVA== 2023-10-11T06:10:36.636 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.y] 2023-10-11T06:10:36.636 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA3PCZlcQtlJhAAyb6WWJKJgbNLkVkNL36ttg== 2023-10-11T06:10:36.636 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2023-10-11T06:10:36.636 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA4PCZl2TIZBBAAHTDKquuoZy+2HB2lk815Fg== 2023-10-11T06:10:36.637 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2023-10-11T06:10:36.637 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA4PCZlj+QWFBAAw6XDEihH6iXYbQsLht/VsQ== 2023-10-11T06:10:36.637 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.0] 2023-10-11T06:10:36.637 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA9PCZljp29IBAAFJgwhWwcw3OWMIDeeXR2sw== 2023-10-11T06:10:36.637 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.1] 2023-10-11T06:10:36.637 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBAPCZlBCAQLBAAeWQVnN/FYimDHxX1nYzVKA== 2023-10-11T06:10:36.638 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.2] 2023-10-11T06:10:36.638 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBDPCZlu1o+NRAAjAauNPRb88x0p37gT88vBw== 2023-10-11T06:10:36.638 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.3] 2023-10-11T06:10:36.638 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBHPCZlcy98BBAALniVjO3ojVM/+q9fw6tNsw== 2023-10-11T06:10:36.638 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.x] 2023-10-11T06:10:36.639 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA3PCZlsakeMxAA52n80muv9FKTIvML6txp5w== 2023-10-11T06:10:36.639 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2023-10-11T06:10:36.639 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA4PCZlaZYAJhAAzmJNe7HYcmHJu+7Fbfv6dw== 2023-10-11T06:10:36.639 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2023-10-11T06:10:36.639 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA4PCZlgvsCNxAAJL7HKUMYQA6zG6useZDoQw== 2023-10-11T06:10:36.640 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.4] 2023-10-11T06:10:36.640 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBOPCZlmds+CBAAb0nHqugOzMsqLZCUabaHGQ== 2023-10-11T06:10:36.640 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.5] 2023-10-11T06:10:36.640 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBRPCZlPObDEhAA1yMeENf3iV3lQf6f6WM8UQ== 2023-10-11T06:10:36.640 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.6] 2023-10-11T06:10:36.640 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBUPCZlZQvrHBAAc/hx1KwJuGfHv5M+Tobkjg== 2023-10-11T06:10:36.641 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.7] 2023-10-11T06:10:36.641 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBXPCZlKKt5NRAAkM8aLRyZPYaG0n1AH2tDIw== 2023-10-11T06:10:36.641 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-10-11T06:10:36.641 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA5PCZlP/K+CRAAYT5I5tAyH8nHrsBReUAL9A== 2023-10-11T06:10:36.641 INFO:teuthology.orchestra.run.smithi084.stdout:[client.2] 2023-10-11T06:10:36.642 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA5PCZli1j2ExAA77o/7VayFJb+C/TfdZrxRg== 2023-10-11T06:10:36.642 INFO:teuthology.orchestra.run.smithi084.stdout:[client.1] 2023-10-11T06:10:36.642 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA5PCZlzhkWHxAAMV2BVukY7a/l7AXFNPCMhg== 2023-10-11T06:10:36.642 INFO:teuthology.orchestra.run.smithi084.stdout:[client.3] 2023-10-11T06:10:36.642 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQA5PCZljnzfKRAA2s7Yg5NkKNp/LP1eWIBEVA== 2023-10-11T06:10:36.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T06:10:36.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T06:10:36.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:36.835 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:36.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:36.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:37.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.140 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.294 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:37.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T06:10:37.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-11T06:10:37.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:37.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:38.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:38.049 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-11T06:10:38.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.488 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.633 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.640 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-11T06:10:38.784 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:38.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:38.931 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:38.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:39.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:39.090 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:39.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:39.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-11T06:10:39.395 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-11T06:10:39.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-11T06:10:39.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-11T06:10:39.651 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-11T06:10:39.723 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-11T06:10:39.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-11T06:10:39.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-11T06:10:40.069 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-11T06:10:40.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-11T06:10:40.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-11T06:10:40.389 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:40.398 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-11T06:10:40.449 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-11T06:10:40.450 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-11T06:10:40.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-11T06:10:40.461 INFO:tasks.ceph.mon.a:Started 2023-10-11T06:10:40.461 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-11T06:10:40.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-11T06:10:40.467 INFO:tasks.ceph.mon.b:Started 2023-10-11T06:10:40.467 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-11T06:10:40.468 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-11T06:10:40.473 INFO:tasks.ceph.mon.c:Started 2023-10-11T06:10:40.473 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-11T06:10:40.474 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-11T06:10:40.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-11T06:10:40.482 INFO:tasks.ceph.mgr.y:Started 2023-10-11T06:10:40.483 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-11T06:10:40.483 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-11T06:10:40.488 INFO:tasks.ceph.mgr.x:Started 2023-10-11T06:10:40.489 INFO:tasks.ceph:Setting crush tunables to default 2023-10-11T06:10:40.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd crush tunables default 2023-10-11T06:10:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.665+0000 7f73ec2f7640 1 Processor -- start 2023-10-11T06:10:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.665+0000 7f73ec2f7640 1 -- start start 2023-10-11T06:10:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.666+0000 7f73ec2f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.666+0000 7f73ec2f7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e40af930 con 0x7f73e415c290 2023-10-11T06:10:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.666+0000 7f73ec2f7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e406cd10 con 0x7f73e40d0c20 2023-10-11T06:10:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.666+0000 7f73ec2f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40afb60 con 0x7f73e40af4d0 2023-10-11T06:10:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.667+0000 7f73e9892640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f73e40d0c20 0x7f73e40b0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:49910/0 (socket says 172.21.15.84:49910) 2023-10-11T06:10:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.667+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 learned_addr learned my addr 172.21.15.84:0/1123769884 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:10:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.667+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.667+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T06:10:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.667+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.667+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.667+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-11T06:10:40.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.868+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:40.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.869+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T06:10:40.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.869+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:40.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.869+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:40.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:40.869+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-11T06:10:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:41.270+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:41.270+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T06:10:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:41.270+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:41.270+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:41.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:41.271+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-11T06:10:42.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:42.072+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:42.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:42.073+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-11T06:10:42.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:42.073+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:42.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:42.073+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:42.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:42.073+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-11T06:10:43.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 msgr2=0x7f73e40d25e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T06:10:43.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73e40af4d0 0x7f73e40d25e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:43.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 >> v1:172.21.15.99:6789/0 conn(0x7f73e40d0c20 legacy=0x7f73e40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:43.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 >> v1:172.21.15.84:6789/0 conn(0x7f73e415c290 legacy=0x7f73e40d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:43.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:43.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:43.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e40afb60 con 0x7f73c80036f0 2023-10-11T06:10:43.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c8008d20 con 0x7f73c80061b0 2023-10-11T06:10:43.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.667+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:43.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:43.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T06:10:43.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:43.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:43.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T06:10:43.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-11T06:10:43.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:43.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:43.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.668+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-11T06:10:43.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.869+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:43.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.869+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:43.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.869+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T06:10:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.869+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.869+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.869+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-11T06:10:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.870+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T06:10:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.870+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:43.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.870+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:43.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:43.870+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-11T06:10:44.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.271+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:44.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.271+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:44.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T06:10:44.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:44.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:44.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-11T06:10:44.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T06:10:44.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:44.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:44.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:44.272+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-11T06:10:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.073+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:45.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-11T06:10:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73ea093640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73ea093640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-11T06:10:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-11T06:10:45.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73e9892640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-11T06:10:45.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-11T06:10:45.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:45.074+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-11T06:10:46.482 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:46.481+0000 7f07bc43c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-11T06:10:46.494 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:46.493+0000 7f325ea32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-11T06:10:46.541 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:46.539+0000 7f07bc43c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-11T06:10:46.552 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:46.551+0000 7f325ea32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-11T06:10:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.667+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 msgr2=0x7f73c8008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T06:10:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.667+0000 7f73dbfff640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80061b0 0x7f73c8008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.667+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 msgr2=0x7f73c8005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T06:10:46.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.667+0000 7f73dbfff640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c80036f0 0x7f73c8005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.667+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 reap_dead start 2023-10-11T06:10:46.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.667+0000 7f73dbfff640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c8007d50 0x7f73c8008180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:46.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.668+0000 7f73dbfff640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80086c0 0x7f73c800d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.668+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e40afe40 con 0x7f73c8003b50 2023-10-11T06:10:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.668+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e40afb60 con 0x7f73c8007d50 2023-10-11T06:10:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.668+0000 7f73dbfff640 1 -- 172.21.15.84:0/1123769884 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c8008d20 con 0x7f73c80086c0 2023-10-11T06:10:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.668+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.99:3300/0 conn(0x7f73c80086c0 0x7f73c800d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.668+0000 7f73e9892640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c8007d50 0x7f73c8008180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.669+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c8007d50 msgr2=0x7f73c8008180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.669+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c8007d50 0x7f73c8008180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.669+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 >> v1:172.21.15.99:6789/0 conn(0x7f73c8003b50 legacy=0x7f73c8006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.669+0000 7f73ea894640 1 -- 172.21.15.84:0/1123769884 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c800e900 con 0x7f73c80086c0 2023-10-11T06:10:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.672+0000 7f73ea894640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c80086c0 0x7f73c800d0f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73e40b5530 tx=0x7f73dc02c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5609caa1e186d7ae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:46.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.673+0000 7f73e9091640 1 -- 172.21.15.84:0/1123769884 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc02cce0 con 0x7f73c80086c0 2023-10-11T06:10:46.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.673+0000 7f73e9091640 1 -- 172.21.15.84:0/1123769884 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc030070 con 0x7f73c80086c0 2023-10-11T06:10:46.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.673+0000 7f73e9091640 1 -- 172.21.15.84:0/1123769884 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc036640 con 0x7f73c80086c0 2023-10-11T06:10:46.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c80086c0 msgr2=0x7f73c800d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c80086c0 0x7f73c800d0f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73e40b5530 tx=0x7f73dc02c300 comp rx=0 tx=0).stop 2023-10-11T06:10:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1123769884 shutdown_connections 2023-10-11T06:10:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 --2- 172.21.15.84:0/1123769884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c80086c0 0x7f73c800d0f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 --2- 172.21.15.84:0/1123769884 >> v2:172.21.15.84:3300/0 conn(0x7f73c8007d50 0x7f73c8008180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1123769884 >> 172.21.15.84:0/1123769884 conn(0x7f73e4071cc0 msgr2=0x7f73e409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1123769884 shutdown_connections 2023-10-11T06:10:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.674+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1123769884 wait complete. 2023-10-11T06:10:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.676+0000 7f73ec2f7640 1 Processor -- start 2023-10-11T06:10:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.677+0000 7f73ec2f7640 1 -- start start 2023-10-11T06:10:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ec2f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73c8003b50 0x7f73e4158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ec2f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c8007d50 0x7f73e409f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ec2f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c80086c0 0x7f73e409f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ec2f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40a0630 con 0x7f73c8003b50 2023-10-11T06:10:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ec2f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e409fd90 con 0x7f73c8007d50 2023-10-11T06:10:46.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ec2f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40a0070 con 0x7f73c80086c0 2023-10-11T06:10:46.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ea093640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73c8003b50 0x7f73e4158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:46.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ea093640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73c8003b50 0x7f73e4158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47728/0 (socket says 172.21.15.84:47728) 2023-10-11T06:10:46.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.678+0000 7f73ea093640 1 -- 172.21.15.84:0/1539406246 learned_addr learned my addr 172.21.15.84:0/1539406246 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:10:46.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ea093640 1 -- 172.21.15.84:0/1539406246 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c80086c0 msgr2=0x7f73e409f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:46.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ea093640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c80086c0 0x7f73e409f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73e9892640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c8007d50 0x7f73e409f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:46.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ea093640 1 -- 172.21.15.84:0/1539406246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c8007d50 msgr2=0x7f73e409f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:46.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ea093640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c8007d50 0x7f73e409f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ea093640 1 -- 172.21.15.84:0/1539406246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc003e10 con 0x7f73c8003b50 2023-10-11T06:10:46.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ea093640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73c8003b50 0x7f73e4158250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73e0003f00 tx=0x7f73e0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:46.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73daffd640 1 -- 172.21.15.84:0/1539406246 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0001c00 con 0x7f73c8003b50 2023-10-11T06:10:46.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73daffd640 1 -- 172.21.15.84:0/1539406246 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0016450 con 0x7f73c8003b50 2023-10-11T06:10:46.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73daffd640 1 -- 172.21.15.84:0/1539406246 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0001d80 con 0x7f73c8003b50 2023-10-11T06:10:46.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e40a0350 con 0x7f73c8003b50 2023-10-11T06:10:46.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.679+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e4158790 con 0x7f73c8003b50 2023-10-11T06:10:46.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.680+0000 7f73daffd640 1 -- 172.21.15.84:0/1539406246 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f73e0016800 con 0x7f73c8003b50 2023-10-11T06:10:46.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.680+0000 7f73daffd640 1 -- 172.21.15.84:0/1539406246 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f73e001f970 con 0x7f73c8003b50 2023-10-11T06:10:46.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.680+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e406f890 con 0x7f73c8003b50 2023-10-11T06:10:46.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.685+0000 7f73daffd640 1 -- 172.21.15.84:0/1539406246 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f73e406f890 con 0x7f73c8003b50 2023-10-11T06:10:46.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.755+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f73e4073ed0 con 0x7f73c8003b50 2023-10-11T06:10:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.759+0000 7f73daffd640 1 -- 172.21.15.84:0/1539406246 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f73e002c020 con 0x7f73c8003b50 2023-10-11T06:10:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:adjusted tunables profile to default 2023-10-11T06:10:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73c8003b50 msgr2=0x7f73e4158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73c8003b50 0x7f73e4158250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73e0003f00 tx=0x7f73e0007e00 comp rx=0 tx=0).stop 2023-10-11T06:10:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 shutdown_connections 2023-10-11T06:10:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c80086c0 0x7f73e409f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c8007d50 0x7f73e409f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 --2- 172.21.15.84:0/1539406246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73c8003b50 0x7f73e4158250 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:46.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 >> 172.21.15.84:0/1539406246 conn(0x7f73e4071cc0 msgr2=0x7f73e40b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:46.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 shutdown_connections 2023-10-11T06:10:46.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:46.760+0000 7f73ec2f7640 1 -- 172.21.15.84:0/1539406246 wait complete. 2023-10-11T06:10:46.780 INFO:tasks.ceph:check_enable_crimson: False 2023-10-11T06:10:46.780 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-11T06:10:46.780 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:46.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-11T06:10:46.867 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:46.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-11T06:10:46.997 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:46.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-11T06:10:47.078 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:10:47.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-11T06:10:47.163 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.161+0000 7f325ea32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-11T06:10:47.165 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.164+0000 7f07bc43c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-11T06:10:47.207 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:47.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-11T06:10:47.224 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.223+0000 7f325ea32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-11T06:10:47.230 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.229+0000 7f07bc43c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-11T06:10:47.277 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.276+0000 7f325ea32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-11T06:10:47.281 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.280+0000 7f07bc43c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-11T06:10:47.297 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:47.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-11T06:10:47.343 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.342+0000 7f07bc43c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-11T06:10:47.346 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.341+0000 7f325ea32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-11T06:10:47.392 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.391+0000 7f07bc43c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-11T06:10:47.394 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.393+0000 7f325ea32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-11T06:10:47.427 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:47.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-11T06:10:47.503 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.501+0000 7f07bc43c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-11T06:10:47.512 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.511+0000 7f325ea32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-11T06:10:47.558 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:10:47.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-11T06:10:47.581 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.580+0000 7f07bc43c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-11T06:10:47.596 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.595+0000 7f325ea32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-11T06:10:47.634 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.633+0000 7f07bc43c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-11T06:10:47.652 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.651+0000 7f325ea32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-11T06:10:47.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new f04c5f33-5c2b-4fe0-ac57-f6deda5f70df 0 2023-10-11T06:10:47.693 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.692+0000 7f07bc43c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-11T06:10:47.715 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.714+0000 7f325ea32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-11T06:10:47.748 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:47.746+0000 7f07bc43c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-11T06:10:47.773 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:47.771+0000 7f325ea32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-11T06:10:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.910+0000 7fc19e911640 1 Processor -- start 2023-10-11T06:10:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.910+0000 7fc19e911640 1 -- start start 2023-10-11T06:10:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.911+0000 7fc19e911640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc19806a380 0x7fc1980cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.911+0000 7fc19e911640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19806e7d0 con 0x7fc198071c50 2023-10-11T06:10:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.911+0000 7fc19e911640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19806ea70 con 0x7fc198071300 2023-10-11T06:10:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.911+0000 7fc19e911640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19806ed50 con 0x7fc19806a380 2023-10-11T06:10:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.911+0000 7fc19e110640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc198071300 0x7fc1980ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52784/0 (socket says 172.21.15.99:52784) 2023-10-11T06:10:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.911+0000 7fc19e110640 1 -- 172.21.15.99:0/2177823593 learned_addr learned my addr 172.21.15.99:0/2177823593 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-11T06:10:47.914 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19d10e640 1 --2- 172.21.15.99:0/2177823593 >> v2:172.21.15.84:3300/0 conn(0x7fc19806a380 0x7fc1980cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:47.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506580986 0 0) 0x7fc19806ea70 con 0x7fc198071300 2023-10-11T06:10:47.915 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc184003520 con 0x7fc198071300 2023-10-11T06:10:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 <== mon.1 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608030080 0 0) 0x7fc184003520 con 0x7fc198071300 2023-10-11T06:10:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 >> v2:172.21.15.84:3300/0 conn(0x7fc19806a380 msgr2=0x7fc1980cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 --2- 172.21.15.99:0/2177823593 >> v2:172.21.15.84:3300/0 conn(0x7fc19806a380 0x7fc1980cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 >> v1:172.21.15.84:6789/0 conn(0x7fc198071c50 legacy=0x7fc198069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19806f030 con 0x7fc198071300 2023-10-11T06:10:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 <== mon.1 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fc190001a40 con 0x7fc198071300 2023-10-11T06:10:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 >> v1:172.21.15.99:6789/0 conn(0x7fc198071300 legacy=0x7fc1980ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184003970 0x7fc184003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1840047c0 0x7fc184004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc184005170 0x7fc18400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19806ed50 con 0x7fc1840047c0 2023-10-11T06:10:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18400c100 con 0x7fc184003970 2023-10-11T06:10:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.912+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18400c2c0 con 0x7fc184005170 2023-10-11T06:10:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19d90f640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184003970 0x7fc184003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19e110640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc184005170 0x7fc18400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19d10e640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1840047c0 0x7fc184004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19d90f640 1 -- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc184005170 msgr2=0x7fc18400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:47.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19d90f640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc184005170 0x7fc18400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:47.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19d90f640 1 -- 172.21.15.99:0/2177823593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1840047c0 msgr2=0x7fc184004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:47.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19d90f640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1840047c0 0x7fc184004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:47.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19d90f640 1 -- 172.21.15.99:0/2177823593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc184003520 con 0x7fc184003970 2023-10-11T06:10:47.919 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.913+0000 7fc19e110640 1 -- 172.21.15.99:0/2177823593 reap_dead start 2023-10-11T06:10:47.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.915+0000 7fc19d90f640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184003970 0x7fc184003d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc194001720 tx=0x7fc1840057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a9dec7ce1aba645 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:47.922 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.915+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940052d0 con 0x7fc184003970 2023-10-11T06:10:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.915+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19402e050 con 0x7fc184003970 2023-10-11T06:10:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.915+0000 7fc19c90d640 1 -- 172.21.15.99:0/2177823593 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194003830 con 0x7fc184003970 2023-10-11T06:10:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.915+0000 7fc19e911640 1 -- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184003970 msgr2=0x7fc184003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.915+0000 7fc19e911640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184003970 0x7fc184003d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc194001720 tx=0x7fc1840057f0 comp rx=0 tx=0).stop 2023-10-11T06:10:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.916+0000 7fc19e911640 1 -- 172.21.15.99:0/2177823593 shutdown_connections 2023-10-11T06:10:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.916+0000 7fc19e911640 1 --2- 172.21.15.99:0/2177823593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184003970 0x7fc184003d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.916+0000 7fc19e911640 1 -- 172.21.15.99:0/2177823593 >> 172.21.15.99:0/2177823593 conn(0x7fc19806ad80 msgr2=0x7fc19806b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.916+0000 7fc19e911640 1 -- 172.21.15.99:0/2177823593 shutdown_connections 2023-10-11T06:10:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.916+0000 7fc19e911640 1 -- 172.21.15.99:0/2177823593 wait complete. 2023-10-11T06:10:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.917+0000 7fc19e911640 1 Processor -- start 2023-10-11T06:10:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.917+0000 7fc19e911640 1 -- start start 2023-10-11T06:10:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.918+0000 7fc19e911640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc184003970 0x7fc198154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.918+0000 7fc19e911640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1981f50b0 0x7fc198150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.918+0000 7fc19e911640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981f2330 0x7fc198158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.918+0000 7fc19e911640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981f58e0 con 0x7fc184003970 2023-10-11T06:10:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.918+0000 7fc19e911640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981f5bc0 con 0x7fc1981f2330 2023-10-11T06:10:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.918+0000 7fc19e911640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981f5ea0 con 0x7fc1981f50b0 2023-10-11T06:10:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19d10e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1981f50b0 0x7fc198150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981f2330 0x7fc198158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981f2330 0x7fc198158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45956/0 (socket says 172.21.15.99:45956) 2023-10-11T06:10:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19d10e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1981f50b0 0x7fc198150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:37082/0 (socket says 172.21.15.99:37082) 2023-10-11T06:10:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 -- 172.21.15.99:0/3019134264 learned_addr learned my addr 172.21.15.99:0/3019134264 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19d90f640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc184003970 0x7fc198154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 -- 172.21.15.99:0/3019134264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1981f50b0 msgr2=0x7fc198150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1981f50b0 0x7fc198150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 -- 172.21.15.99:0/3019134264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc184003970 msgr2=0x7fc198154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc184003970 0x7fc198154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 -- 172.21.15.99:0/3019134264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1981f1e10 con 0x7fc1981f2330 2023-10-11T06:10:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.919+0000 7fc19e110640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981f2330 0x7fc198158fc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc190000f80 tx=0x7fc190007a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.920+0000 7fc18e7fc640 1 -- 172.21.15.99:0/3019134264 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900136c0 con 0x7fc1981f2330 2023-10-11T06:10:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.920+0000 7fc18e7fc640 1 -- 172.21.15.99:0/3019134264 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc190013840 con 0x7fc1981f2330 2023-10-11T06:10:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.920+0000 7fc18e7fc640 1 -- 172.21.15.99:0/3019134264 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190013bf0 con 0x7fc1981f2330 2023-10-11T06:10:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.920+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1981f1ac0 con 0x7fc1981f2330 2023-10-11T06:10:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.920+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1981f5520 con 0x7fc1981f2330 2023-10-11T06:10:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.920+0000 7fc18e7fc640 1 -- 172.21.15.99:0/3019134264 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc190013d50 con 0x7fc1981f2330 2023-10-11T06:10:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.920+0000 7fc18e7fc640 1 -- 172.21.15.99:0/3019134264 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc19001d620 con 0x7fc1981f2330 2023-10-11T06:10:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.921+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19804e890 con 0x7fc1981f2330 2023-10-11T06:10:47.930 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:47.924+0000 7fc18e7fc640 1 -- 172.21.15.99:0/3019134264 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc19001e8f0 con 0x7fc1981f2330 2023-10-11T06:10:48.003 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.002+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f04c5f33-5c2b-4fe0-ac57-f6deda5f70df", "id": 0} v 0) v1 -- 0x7fc1980cc920 con 0x7fc1981f2330 2023-10-11T06:10:48.010 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.009+0000 7fc18e7fc640 1 -- 172.21.15.99:0/3019134264 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f04c5f33-5c2b-4fe0-ac57-f6deda5f70df", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc19001ea90 con 0x7fc1981f2330 2023-10-11T06:10:48.011 INFO:teuthology.orchestra.run.smithi099.stdout:0 2023-10-11T06:10:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981f2330 msgr2=0x7fc198158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981f2330 0x7fc198158fc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc190000f80 tx=0x7fc190007a30 comp rx=0 tx=0).stop 2023-10-11T06:10:48.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 shutdown_connections 2023-10-11T06:10:48.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981f2330 0x7fc198158fc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1981f50b0 0x7fc198150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 --2- 172.21.15.99:0/3019134264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc184003970 0x7fc198154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.012 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 >> 172.21.15.99:0/3019134264 conn(0x7fc19806ad80 msgr2=0x7fc19806b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:48.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 shutdown_connections 2023-10-11T06:10:48.013 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.010+0000 7fc19e911640 1 -- 172.21.15.99:0/3019134264 wait complete. 2023-10-11T06:10:48.028 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 713cb503-ae2a-45af-b3bd-5133ee34f7cd 1 2023-10-11T06:10:48.198 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.197+0000 7f07bc43c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-11T06:10:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.219+0000 7f19dcdbb640 1 Processor -- start 2023-10-11T06:10:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.219+0000 7f19dcdbb640 1 -- start start 2023-10-11T06:10:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.220+0000 7f19dcdbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.220+0000 7f19dcdbb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d815a530 con 0x7f19d80cad10 2023-10-11T06:10:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.220+0000 7f19dcdbb640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d80b0cf0 con 0x7f19d80afce0 2023-10-11T06:10:48.222 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.220+0000 7f19dcdbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80b1590 con 0x7f19d815dc40 2023-10-11T06:10:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d659c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f19d80afce0 0x7f19d80cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52794/0 (socket says 172.21.15.99:52794) 2023-10-11T06:10:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:37096/0 (socket says 172.21.15.99:37096) 2023-10-11T06:10:48.223 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d659c640 1 -- 172.21.15.99:0/4104636844 learned_addr learned my addr 172.21.15.99:0/4104636844 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-11T06:10:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d559a640 1 -- 172.21.15.99:0/4104636844 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258528350 0 0) 0x7f19d80b0cf0 con 0x7f19d80afce0 2023-10-11T06:10:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d5d9b640 1 -- 172.21.15.99:0/4104636844 >> v1:172.21.15.99:6789/0 conn(0x7f19d80afce0 legacy=0x7f19d80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d5d9b640 1 -- 172.21.15.99:0/4104636844 >> v1:172.21.15.84:6789/0 conn(0x7f19d80cad10 legacy=0x7f19d80ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.221+0000 7f19d5d9b640 1 -- 172.21.15.99:0/4104636844 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80b0fd0 con 0x7f19d815dc40 2023-10-11T06:10:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.222+0000 7f19d5d9b640 1 --2- 172.21.15.99:0/4104636844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b05f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097d0 tx=0x7f19cc02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16dfdab529a07215 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19d559a640 1 -- 172.21.15.99:0/4104636844 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc031800 con 0x7f19d815dc40 2023-10-11T06:10:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19d559a640 1 -- 172.21.15.99:0/4104636844 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc031960 con 0x7f19d815dc40 2023-10-11T06:10:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19d559a640 1 -- 172.21.15.99:0/4104636844 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc031cf0 con 0x7f19d815dc40 2023-10-11T06:10:48.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19dcdbb640 1 -- 172.21.15.99:0/4104636844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 msgr2=0x7f19d80b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19dcdbb640 1 --2- 172.21.15.99:0/4104636844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b05f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097d0 tx=0x7f19cc02b8a0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.226 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19dcdbb640 1 -- 172.21.15.99:0/4104636844 shutdown_connections 2023-10-11T06:10:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19dcdbb640 1 --2- 172.21.15.99:0/4104636844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b05f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19dcdbb640 1 -- 172.21.15.99:0/4104636844 >> 172.21.15.99:0/4104636844 conn(0x7f19d806a590 msgr2=0x7f19d8099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.223+0000 7f19dcdbb640 1 -- 172.21.15.99:0/4104636844 shutdown_connections 2023-10-11T06:10:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.224+0000 7f19dcdbb640 1 -- 172.21.15.99:0/4104636844 wait complete. 2023-10-11T06:10:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.226+0000 7f19dcdbb640 1 Processor -- start 2023-10-11T06:10:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.226+0000 7f19dcdbb640 1 -- start start 2023-10-11T06:10:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19dcdbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19d80afce0 0x7f19d80c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19dcdbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19d80cad10 0x7f19d80cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19dcdbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19dcdbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81518e0 con 0x7f19d80afce0 2023-10-11T06:10:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19dcdbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80c4cb0 con 0x7f19d80cad10 2023-10-11T06:10:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19dcdbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80b3e20 con 0x7f19d815dc40 2023-10-11T06:10:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19d80cad10 0x7f19d80cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19d6d9d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19d5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19d80cad10 0x7f19d80cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45972/0 (socket says 172.21.15.99:45972) 2023-10-11T06:10:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.228+0000 7f19d5d9b640 1 -- 172.21.15.99:0/1439124371 learned_addr learned my addr 172.21.15.99:0/1439124371 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d659c640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19d80afce0 0x7f19d80c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d6d9d640 1 -- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19d80cad10 msgr2=0x7f19d80cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d6d9d640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19d80cad10 0x7f19d80cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d6d9d640 1 -- 172.21.15.99:0/1439124371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19d80afce0 msgr2=0x7f19d80c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d6d9d640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19d80afce0 0x7f19d80c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d6d9d640 1 -- 172.21.15.99:0/1439124371 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc009400 con 0x7f19d815dc40 2023-10-11T06:10:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d659c640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19d80afce0 0x7f19d80c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:10:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d5d9b640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19d80cad10 0x7f19d80cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:10:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19d6d9d640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b5960 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c730 tx=0x7f19d000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19c6ffd640 1 -- 172.21.15.99:0/1439124371 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0003b70 con 0x7f19d815dc40 2023-10-11T06:10:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19c6ffd640 1 -- 172.21.15.99:0/1439124371 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d0003cd0 con 0x7f19d815dc40 2023-10-11T06:10:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19c6ffd640 1 -- 172.21.15.99:0/1439124371 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0005310 con 0x7f19d815dc40 2023-10-11T06:10:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.229+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d80b4100 con 0x7f19d815dc40 2023-10-11T06:10:48.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.230+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d80b43e0 con 0x7f19d815dc40 2023-10-11T06:10:48.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.230+0000 7f19c6ffd640 1 -- 172.21.15.99:0/1439124371 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f19d000a640 con 0x7f19d815dc40 2023-10-11T06:10:48.237 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.230+0000 7f19c6ffd640 1 -- 172.21.15.99:0/1439124371 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f19d00125e0 con 0x7f19d815dc40 2023-10-11T06:10:48.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.231+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d80c4e70 con 0x7f19d815dc40 2023-10-11T06:10:48.238 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.233+0000 7f19c6ffd640 1 -- 172.21.15.99:0/1439124371 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f19d000aa80 con 0x7f19d815dc40 2023-10-11T06:10:48.252 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:48.251+0000 7f325ea32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-11T06:10:48.313 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.312+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd new", "uuid": "713cb503-ae2a-45af-b3bd-5133ee34f7cd", "id": 1} v 0) v1 -- 0x7f19d8067090 con 0x7f19d815dc40 2023-10-11T06:10:48.323 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.322+0000 7f19c6ffd640 1 -- 172.21.15.99:0/1439124371 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "713cb503-ae2a-45af-b3bd-5133ee34f7cd", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f19d000ac20 con 0x7f19d815dc40 2023-10-11T06:10:48.323 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-11T06:10:48.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.323+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 msgr2=0x7f19d80b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.323+0000 7f19dcdbb640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b5960 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f19d000c730 tx=0x7f19d000cbf0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.323+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 shutdown_connections 2023-10-11T06:10:48.325 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.323+0000 7f19dcdbb640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19d815dc40 0x7f19d80b5960 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.323+0000 7f19dcdbb640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19d80cad10 0x7f19d80cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.323+0000 7f19dcdbb640 1 --2- 172.21.15.99:0/1439124371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19d80afce0 0x7f19d80c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.323+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 >> 172.21.15.99:0/1439124371 conn(0x7f19d806a590 msgr2=0x7f19d8099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:48.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.324+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 shutdown_connections 2023-10-11T06:10:48.326 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.324+0000 7f19dcdbb640 1 -- 172.21.15.99:0/1439124371 wait complete. 2023-10-11T06:10:48.341 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new faeec611-1e2f-466e-addc-fc7e02102e37 2 2023-10-11T06:10:48.357 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.356+0000 7f07bc43c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-11T06:10:48.421 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:48.419+0000 7f325ea32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-11T06:10:48.494 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.493+0000 7f07bc43c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-11T06:10:48.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.526+0000 7ff25a500640 1 Processor -- start 2023-10-11T06:10:48.527 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.526+0000 7ff25a500640 1 -- start start 2023-10-11T06:10:48.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.527+0000 7ff25a500640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540ce0d0 0x7ff2540c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.529 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.527+0000 7ff25a500640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2540adb90 0x7ff2540caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.527+0000 7ff25a500640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2541593f0 con 0x7ff2540cd300 2023-10-11T06:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.527+0000 7ff25a500640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25415a340 con 0x7ff2540adb90 2023-10-11T06:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.527+0000 7ff25a500640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540ce4a0 con 0x7ff2540ce0d0 2023-10-11T06:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff253fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540ce0d0 0x7ff2540c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff253fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540ce0d0 0x7ff2540c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:37114/0 (socket says 172.21.15.99:37114) 2023-10-11T06:10:48.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff253fff640 1 -- 172.21.15.99:0/656003557 learned_addr learned my addr 172.21.15.99:0/656003557 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:48.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff24bfff640 1 --2- 172.21.15.99:0/656003557 >> v2:172.21.15.99:3300/0 conn(0x7ff2540adb90 0x7ff2540caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff253fff640 1 -- 172.21.15.99:0/656003557 >> v2:172.21.15.99:3300/0 conn(0x7ff2540adb90 msgr2=0x7ff2540caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:48.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff253fff640 1 --2- 172.21.15.99:0/656003557 >> v2:172.21.15.99:3300/0 conn(0x7ff2540adb90 0x7ff2540caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.531 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff253fff640 1 -- 172.21.15.99:0/656003557 >> v1:172.21.15.99:6789/0 conn(0x7ff2540cd300 legacy=0x7ff2540b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.532 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.528+0000 7ff253fff640 1 -- 172.21.15.99:0/656003557 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2541596c0 con 0x7ff2540ce0d0 2023-10-11T06:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.529+0000 7ff253fff640 1 --2- 172.21.15.99:0/656003557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540ce0d0 0x7ff2540c8cc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff244009850 tx=0x7ff24402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a036e5dc67f8bb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.529+0000 7ff2537fe640 1 -- 172.21.15.99:0/656003557 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244004230 con 0x7ff2540ce0d0 2023-10-11T06:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.529+0000 7ff2537fe640 1 -- 172.21.15.99:0/656003557 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244004390 con 0x7ff2540ce0d0 2023-10-11T06:10:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.529+0000 7ff2537fe640 1 -- 172.21.15.99:0/656003557 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244003810 con 0x7ff2540ce0d0 2023-10-11T06:10:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 -- 172.21.15.99:0/656003557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540ce0d0 msgr2=0x7ff2540c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 --2- 172.21.15.99:0/656003557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540ce0d0 0x7ff2540c8cc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff244009850 tx=0x7ff24402b960 comp rx=0 tx=0).stop 2023-10-11T06:10:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 -- 172.21.15.99:0/656003557 shutdown_connections 2023-10-11T06:10:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 --2- 172.21.15.99:0/656003557 >> v2:172.21.15.99:3300/0 conn(0x7ff2540adb90 0x7ff2540caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 --2- 172.21.15.99:0/656003557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540ce0d0 0x7ff2540c8cc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 -- 172.21.15.99:0/656003557 >> 172.21.15.99:0/656003557 conn(0x7ff254069c70 msgr2=0x7ff25406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:48.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 -- 172.21.15.99:0/656003557 shutdown_connections 2023-10-11T06:10:48.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.530+0000 7ff25a500640 1 -- 172.21.15.99:0/656003557 wait complete. 2023-10-11T06:10:48.535 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.532+0000 7ff25a500640 1 Processor -- start 2023-10-11T06:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.532+0000 7ff25a500640 1 -- start start 2023-10-11T06:10:48.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff25a500640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2540adb90 0x7ff2540c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff25a500640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540cd300 0x7ff25414bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff25a500640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540ce0d0 0x7ff2540b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.538 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff25a500640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540c7190 con 0x7ff2540adb90 2023-10-11T06:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff25a500640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540c5bb0 con 0x7ff2540ce0d0 2023-10-11T06:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff25a500640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540c5e90 con 0x7ff2540cd300 2023-10-11T06:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540cd300 0x7ff25414bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff258a9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540ce0d0 0x7ff2540b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff24bfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540cd300 0x7ff25414bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:37122/0 (socket says 172.21.15.99:37122) 2023-10-11T06:10:48.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.533+0000 7ff24bfff640 1 -- 172.21.15.99:0/2067646153 learned_addr learned my addr 172.21.15.99:0/2067646153 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:48.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff253fff640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2540adb90 0x7ff2540c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff258a9d640 1 -- 172.21.15.99:0/2067646153 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540cd300 msgr2=0x7ff25414bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff258a9d640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540cd300 0x7ff25414bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff258a9d640 1 -- 172.21.15.99:0/2067646153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2540adb90 msgr2=0x7ff2540c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.540 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff258a9d640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2540adb90 0x7ff2540c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff258a9d640 1 -- 172.21.15.99:0/2067646153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff244009460 con 0x7ff2540ce0d0 2023-10-11T06:10:48.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff258a9d640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540ce0d0 0x7ff2540b1df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c700 tx=0x7ff24c00cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:48.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff2517fa640 1 -- 172.21.15.99:0/2067646153 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c003b70 con 0x7ff2540ce0d0 2023-10-11T06:10:48.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff2517fa640 1 -- 172.21.15.99:0/2067646153 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c003cd0 con 0x7ff2540ce0d0 2023-10-11T06:10:48.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff2517fa640 1 -- 172.21.15.99:0/2067646153 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c005310 con 0x7ff2540ce0d0 2023-10-11T06:10:48.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2540c6170 con 0x7ff2540ce0d0 2023-10-11T06:10:48.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.534+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2540c6730 con 0x7ff2540ce0d0 2023-10-11T06:10:48.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.535+0000 7ff2517fa640 1 -- 172.21.15.99:0/2067646153 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff24c00a640 con 0x7ff2540ce0d0 2023-10-11T06:10:48.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.535+0000 7ff2517fa640 1 -- 172.21.15.99:0/2067646153 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7ff24c00ad10 con 0x7ff2540ce0d0 2023-10-11T06:10:48.542 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.535+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff254159f10 con 0x7ff2540ce0d0 2023-10-11T06:10:48.543 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.538+0000 7ff2517fa640 1 -- 172.21.15.99:0/2067646153 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff24c00a8f0 con 0x7ff2540ce0d0 2023-10-11T06:10:48.546 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.545+0000 7f07bc43c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-11T06:10:48.564 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:48.563+0000 7f325ea32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-11T06:10:48.616 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.615+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "faeec611-1e2f-466e-addc-fc7e02102e37", "id": 2} v 0) v1 -- 0x7ff254073ed0 con 0x7ff2540ce0d0 2023-10-11T06:10:48.619 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:48.618+0000 7f325ea32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-11T06:10:48.626 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.625+0000 7ff2517fa640 1 -- 172.21.15.99:0/2067646153 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "faeec611-1e2f-466e-addc-fc7e02102e37", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff24c025480 con 0x7ff2540ce0d0 2023-10-11T06:10:48.626 INFO:teuthology.orchestra.run.smithi099.stdout:2 2023-10-11T06:10:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540ce0d0 msgr2=0x7ff2540b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540ce0d0 0x7ff2540b1df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c700 tx=0x7ff24c00cbc0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 shutdown_connections 2023-10-11T06:10:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540ce0d0 0x7ff2540b1df0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2540cd300 0x7ff25414bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 --2- 172.21.15.99:0/2067646153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2540adb90 0x7ff2540c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 >> 172.21.15.99:0/2067646153 conn(0x7ff254069c70 msgr2=0x7ff2540ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:48.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 shutdown_connections 2023-10-11T06:10:48.629 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.626+0000 7ff25a500640 1 -- 172.21.15.99:0/2067646153 wait complete. 2023-10-11T06:10:48.643 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 7485b46e-6701-4a5d-825f-66b3a7841ab8 3 2023-10-11T06:10:48.654 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.653+0000 7f07bc43c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-11T06:10:48.736 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:48.735+0000 7f325ea32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-11T06:10:48.763 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.762+0000 7f07bc43c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-11T06:10:48.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.832+0000 7fd0663dc640 1 Processor -- start 2023-10-11T06:10:48.833 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.832+0000 7fd0663dc640 1 -- start start 2023-10-11T06:10:48.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.833+0000 7fd0663dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd06015c290 0x7fd0600af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.835 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd0663dc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0600d0c20 0x7fd0600b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd0663dc640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600d2a60 con 0x7fd0600d25e0 2023-10-11T06:10:48.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd0663dc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06006cd10 con 0x7fd06015c290 2023-10-11T06:10:48.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd0663dc640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600afb60 con 0x7fd0600d0c20 2023-10-11T06:10:48.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd064979640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0600d0c20 0x7fd0600b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.836 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd05f7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd0600d25e0 0x7fd0600d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52808/0 (socket says 172.21.15.99:52808) 2023-10-11T06:10:48.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd05f7fe640 1 -- 172.21.15.99:0/3071922846 learned_addr learned my addr 172.21.15.99:0/3071922846 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-11T06:10:48.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd064979640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0600d0c20 0x7fd0600b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45998/0 (socket says 172.21.15.99:45998) 2023-10-11T06:10:48.837 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd05ffff640 1 --2- 172.21.15.99:0/3071922846 >> v2:172.21.15.84:3300/0 conn(0x7fd06015c290 0x7fd0600af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd064979640 1 -- 172.21.15.99:0/3071922846 >> v2:172.21.15.84:3300/0 conn(0x7fd06015c290 msgr2=0x7fd0600af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:48.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd064979640 1 --2- 172.21.15.99:0/3071922846 >> v2:172.21.15.84:3300/0 conn(0x7fd06015c290 0x7fd0600af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd064979640 1 -- 172.21.15.99:0/3071922846 >> v1:172.21.15.99:6789/0 conn(0x7fd0600d25e0 legacy=0x7fd0600d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.834+0000 7fd064979640 1 -- 172.21.15.99:0/3071922846 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600afe40 con 0x7fd0600d0c20 2023-10-11T06:10:48.839 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.835+0000 7fd064979640 1 --2- 172.21.15.99:0/3071922846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0600d0c20 0x7fd0600b0c00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0600d2150 tx=0x7fd05802be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a22117cfcd8f26 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:48.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd05effd640 1 -- 172.21.15.99:0/3071922846 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05802cbe0 con 0x7fd0600d0c20 2023-10-11T06:10:48.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd05effd640 1 -- 172.21.15.99:0/3071922846 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05802cd40 con 0x7fd0600d0c20 2023-10-11T06:10:48.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd05effd640 1 -- 172.21.15.99:0/3071922846 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058034660 con 0x7fd0600d0c20 2023-10-11T06:10:48.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd0663dc640 1 -- 172.21.15.99:0/3071922846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0600d0c20 msgr2=0x7fd0600b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.840 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd0663dc640 1 --2- 172.21.15.99:0/3071922846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0600d0c20 0x7fd0600b0c00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0600d2150 tx=0x7fd05802be30 comp rx=0 tx=0).stop 2023-10-11T06:10:48.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd0663dc640 1 -- 172.21.15.99:0/3071922846 shutdown_connections 2023-10-11T06:10:48.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd0663dc640 1 --2- 172.21.15.99:0/3071922846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0600d0c20 0x7fd0600b0c00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.836+0000 7fd0663dc640 1 --2- 172.21.15.99:0/3071922846 >> v2:172.21.15.84:3300/0 conn(0x7fd06015c290 0x7fd0600af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.837+0000 7fd0663dc640 1 -- 172.21.15.99:0/3071922846 >> 172.21.15.99:0/3071922846 conn(0x7fd060071cc0 msgr2=0x7fd06009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:48.841 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.837+0000 7fd0663dc640 1 -- 172.21.15.99:0/3071922846 shutdown_connections 2023-10-11T06:10:48.842 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.837+0000 7fd0663dc640 1 -- 172.21.15.99:0/3071922846 wait complete. 2023-10-11T06:10:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.838+0000 7fd0663dc640 1 Processor -- start 2023-10-11T06:10:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.838+0000 7fd0663dc640 1 -- start start 2023-10-11T06:10:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.839+0000 7fd0663dc640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0600d0c20 0x7fd0600c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.844 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.839+0000 7fd0663dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0600d25e0 0x7fd0600c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.839+0000 7fd0663dc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06015c290 0x7fd0600cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:48.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.839+0000 7fd0663dc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600ceda0 con 0x7fd0600d25e0 2023-10-11T06:10:48.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.839+0000 7fd0663dc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600cf890 con 0x7fd06015c290 2023-10-11T06:10:48.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.839+0000 7fd0663dc640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06009da90 con 0x7fd0600d0c20 2023-10-11T06:10:48.845 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.839+0000 7fd064979640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06015c290 0x7fd0600cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd05ffff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0600d0c20 0x7fd0600c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06015c290 0x7fd0600cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46008/0 (socket says 172.21.15.99:46008) 2023-10-11T06:10:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 -- 172.21.15.99:0/2693073814 learned_addr learned my addr 172.21.15.99:0/2693073814 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd05f7fe640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0600d25e0 0x7fd0600c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:48.846 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 -- 172.21.15.99:0/2693073814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0600d0c20 msgr2=0x7fd0600c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0600d0c20 0x7fd0600c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 -- 172.21.15.99:0/2693073814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0600d25e0 msgr2=0x7fd0600c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0600d25e0 0x7fd0600c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 -- 172.21.15.99:0/2693073814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009a70 con 0x7fd06015c290 2023-10-11T06:10:48.847 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd064979640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06015c290 0x7fd0600cc270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd058009e80 tx=0x7fd058004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:48.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.840+0000 7fd05cff9640 1 -- 172.21.15.99:0/2693073814 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058040650 con 0x7fd06015c290 2023-10-11T06:10:48.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.841+0000 7fd05cff9640 1 -- 172.21.15.99:0/2693073814 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0580407d0 con 0x7fd06015c290 2023-10-11T06:10:48.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.841+0000 7fd05cff9640 1 -- 172.21.15.99:0/2693073814 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058040b80 con 0x7fd06015c290 2023-10-11T06:10:48.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.841+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600c9f50 con 0x7fd06015c290 2023-10-11T06:10:48.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.841+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0600cf2d0 con 0x7fd06015c290 2023-10-11T06:10:48.848 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.841+0000 7fd05cff9640 1 -- 172.21.15.99:0/2693073814 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd058049400 con 0x7fd06015c290 2023-10-11T06:10:48.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.841+0000 7fd05cff9640 1 -- 172.21.15.99:0/2693073814 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd058049b40 con 0x7fd06015c290 2023-10-11T06:10:48.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.842+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd060073ed0 con 0x7fd06015c290 2023-10-11T06:10:48.849 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.844+0000 7fd05cff9640 1 -- 172.21.15.99:0/2693073814 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd058068b00 con 0x7fd06015c290 2023-10-11T06:10:48.854 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:48.852+0000 7f325ea32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-11T06:10:48.922 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.920+0000 7f07bc43c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-11T06:10:48.924 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.923+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7485b46e-6701-4a5d-825f-66b3a7841ab8", "id": 3} v 0) v1 -- 0x7fd06006c720 con 0x7fd06015c290 2023-10-11T06:10:48.934 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.933+0000 7fd05cff9640 1 -- 172.21.15.99:0/2693073814 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7485b46e-6701-4a5d-825f-66b3a7841ab8", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd058057030 con 0x7fd06015c290 2023-10-11T06:10:48.934 INFO:teuthology.orchestra.run.smithi099.stdout:3 2023-10-11T06:10:48.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06015c290 msgr2=0x7fd0600cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:48.935 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06015c290 0x7fd0600cc270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd058009e80 tx=0x7fd058004a40 comp rx=0 tx=0).stop 2023-10-11T06:10:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 shutdown_connections 2023-10-11T06:10:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd06015c290 0x7fd0600cc270 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0600d25e0 0x7fd0600c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 --2- 172.21.15.99:0/2693073814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0600d0c20 0x7fd0600c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 >> 172.21.15.99:0/2693073814 conn(0x7fd060071cc0 msgr2=0x7fd0600b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:48.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 shutdown_connections 2023-10-11T06:10:48.937 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:48.934+0000 7fd0663dc640 1 -- 172.21.15.99:0/2693073814 wait complete. 2023-10-11T06:10:48.952 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8 4 2023-10-11T06:10:48.971 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:48.970+0000 7f07bc43c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-11T06:10:49.024 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:49.022+0000 7f325ea32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-11T06:10:49.077 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:49.076+0000 7f325ea32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-11T06:10:49.127 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:49.126+0000 7f07bc43c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-11T06:10:49.148 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.146+0000 7fb77643b640 1 Processor -- start 2023-10-11T06:10:49.149 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.147+0000 7fb77643b640 1 -- start start 2023-10-11T06:10:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb77643b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb770071c50 0x7fb7700cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb77643b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb770071300 0x7fb770069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb77643b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77006e7d0 con 0x7fb77006a1b0 2023-10-11T06:10:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb77643b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77006a5a0 con 0x7fb770071c50 2023-10-11T06:10:49.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb77643b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77006ea70 con 0x7fb770071300 2023-10-11T06:10:49.154 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb774c38640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb770071300 0x7fb770069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb774c38640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb770071300 0x7fb770069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46024/0 (socket says 172.21.15.99:46024) 2023-10-11T06:10:49.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb774c38640 1 -- 172.21.15.99:0/3239095642 learned_addr learned my addr 172.21.15.99:0/3239095642 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:49.155 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb775439640 1 --2- 172.21.15.99:0/3239095642 >> v2:172.21.15.84:3300/0 conn(0x7fb770071c50 0x7fb7700cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.148+0000 7fb774c38640 1 -- 172.21.15.99:0/3239095642 >> v2:172.21.15.84:3300/0 conn(0x7fb770071c50 msgr2=0x7fb7700cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:49.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.149+0000 7fb774c38640 1 --2- 172.21.15.99:0/3239095642 >> v2:172.21.15.84:3300/0 conn(0x7fb770071c50 0x7fb7700cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.156 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.149+0000 7fb774c38640 1 -- 172.21.15.99:0/3239095642 >> v1:172.21.15.84:6789/0 conn(0x7fb77006a1b0 legacy=0x7fb77015fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.149+0000 7fb774c38640 1 -- 172.21.15.99:0/3239095642 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77006ed50 con 0x7fb770071300 2023-10-11T06:10:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.150+0000 7fb774c38640 1 --2- 172.21.15.99:0/3239095642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb770069c70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7700708f0 tx=0x7fb76002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6287bd8bf4a5c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:49.157 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.150+0000 7fb767fff640 1 -- 172.21.15.99:0/3239095642 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760031870 con 0x7fb770071300 2023-10-11T06:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.150+0000 7fb767fff640 1 -- 172.21.15.99:0/3239095642 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7600319d0 con 0x7fb770071300 2023-10-11T06:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.150+0000 7fb767fff640 1 -- 172.21.15.99:0/3239095642 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760031d60 con 0x7fb770071300 2023-10-11T06:10:49.158 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.150+0000 7fb77643b640 1 -- 172.21.15.99:0/3239095642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 msgr2=0x7fb770069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.150+0000 7fb77643b640 1 --2- 172.21.15.99:0/3239095642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb770069c70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7700708f0 tx=0x7fb76002b910 comp rx=0 tx=0).stop 2023-10-11T06:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.151+0000 7fb77643b640 1 -- 172.21.15.99:0/3239095642 shutdown_connections 2023-10-11T06:10:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.151+0000 7fb77643b640 1 --2- 172.21.15.99:0/3239095642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb770069c70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.151+0000 7fb77643b640 1 --2- 172.21.15.99:0/3239095642 >> v2:172.21.15.84:3300/0 conn(0x7fb770071c50 0x7fb7700cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.151+0000 7fb77643b640 1 -- 172.21.15.99:0/3239095642 >> 172.21.15.99:0/3239095642 conn(0x7fb77006ad80 msgr2=0x7fb77006b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:49.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.151+0000 7fb77643b640 1 -- 172.21.15.99:0/3239095642 shutdown_connections 2023-10-11T06:10:49.161 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.151+0000 7fb77643b640 1 -- 172.21.15.99:0/3239095642 wait complete. 2023-10-11T06:10:49.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.152+0000 7fb77643b640 1 Processor -- start 2023-10-11T06:10:49.165 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.152+0000 7fb77643b640 1 -- start start 2023-10-11T06:10:49.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.153+0000 7fb77643b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77006a1b0 0x7fb77014e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.166 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.153+0000 7fb77643b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb77015f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.153+0000 7fb77643b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb770071c50 0x7fb77015fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.153+0000 7fb77643b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7700c7f60 con 0x7fb77006a1b0 2023-10-11T06:10:49.167 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.153+0000 7fb77643b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7700c76a0 con 0x7fb770071300 2023-10-11T06:10:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.153+0000 7fb77643b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7700c7910 con 0x7fb770071c50 2023-10-11T06:10:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb77015f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb77015f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46032/0 (socket says 172.21.15.99:46032) 2023-10-11T06:10:49.168 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 -- 172.21.15.99:0/3469181437 learned_addr learned my addr 172.21.15.99:0/3469181437 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb775c3a640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb770071c50 0x7fb77015fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb775439640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77006a1b0 0x7fb77014e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.169 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 -- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb770071c50 msgr2=0x7fb77015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb770071c50 0x7fb77015fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 -- 172.21.15.99:0/3469181437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77006a1b0 msgr2=0x7fb77014e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77006a1b0 0x7fb77014e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 -- 172.21.15.99:0/3469181437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7600094d0 con 0x7fb770071300 2023-10-11T06:10:49.171 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.154+0000 7fb774c38640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb77015f850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb760000c00 tx=0x7fb7600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:49.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.155+0000 7fb765ffb640 1 -- 172.21.15.99:0/3469181437 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76003abe0 con 0x7fb770071300 2023-10-11T06:10:49.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.155+0000 7fb765ffb640 1 -- 172.21.15.99:0/3469181437 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76003ad40 con 0x7fb770071300 2023-10-11T06:10:49.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.155+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7700c7bf0 con 0x7fb770071300 2023-10-11T06:10:49.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.155+0000 7fb765ffb640 1 -- 172.21.15.99:0/3469181437 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760044610 con 0x7fb770071300 2023-10-11T06:10:49.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.155+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77014ed80 con 0x7fb770071300 2023-10-11T06:10:49.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.155+0000 7fb765ffb640 1 -- 172.21.15.99:0/3469181437 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb760044a70 con 0x7fb770071300 2023-10-11T06:10:49.174 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.156+0000 7fb765ffb640 1 -- 172.21.15.99:0/3469181437 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fb76002d040 con 0x7fb770071300 2023-10-11T06:10:49.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.156+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77004e890 con 0x7fb770071300 2023-10-11T06:10:49.175 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.158+0000 7fb765ffb640 1 -- 172.21.15.99:0/3469181437 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb760052060 con 0x7fb770071300 2023-10-11T06:10:49.236 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.235+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8", "id": 4} v 0) v1 -- 0x7fb7700cc920 con 0x7fb770071300 2023-10-11T06:10:49.242 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:49.240+0000 7f325ea32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-11T06:10:49.246 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.245+0000 7fb765ffb640 1 -- 172.21.15.99:0/3469181437 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb760035020 con 0x7fb770071300 2023-10-11T06:10:49.247 INFO:teuthology.orchestra.run.smithi099.stdout:4 2023-10-11T06:10:49.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.246+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 msgr2=0x7fb77015f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.246+0000 7fb77643b640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb77015f850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb760000c00 tx=0x7fb7600046c0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.248 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.246+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 shutdown_connections 2023-10-11T06:10:49.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.246+0000 7fb77643b640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb770071c50 0x7fb77015fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.246+0000 7fb77643b640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb770071300 0x7fb77015f850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.246+0000 7fb77643b640 1 --2- 172.21.15.99:0/3469181437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb77006a1b0 0x7fb77014e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.246+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 >> 172.21.15.99:0/3469181437 conn(0x7fb77006ad80 msgr2=0x7fb7700c2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:49.249 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.247+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 shutdown_connections 2023-10-11T06:10:49.250 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.247+0000 7fb77643b640 1 -- 172.21.15.99:0/3469181437 wait complete. 2023-10-11T06:10:49.265 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 62ae3f3f-d323-46cd-8f29-289419f9f189 5 2023-10-11T06:10:49.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.450+0000 7fb8f6a15640 1 Processor -- start 2023-10-11T06:10:49.452 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.450+0000 7fb8f6a15640 1 -- start start 2023-10-11T06:10:49.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.452+0000 7fb8f6a15640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f00c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.452+0000 7fb8f6a15640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f00c4760 con 0x7fb8f0159070 2023-10-11T06:10:49.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.452+0000 7fb8f6a15640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f00cadc0 con 0x7fb8f00affd0 2023-10-11T06:10:49.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.452+0000 7fb8f6a15640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00cbd40 con 0x7fb8f00cb8a0 2023-10-11T06:10:49.454 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.452+0000 7fb8f5a13640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb8f00affd0 0x7fb8f00b03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52820/0 (socket says 172.21.15.99:52820) 2023-10-11T06:10:49.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.452+0000 7fb8f5a13640 1 -- 172.21.15.99:0/3956401755 learned_addr learned my addr 172.21.15.99:0/3956401755 (peer_addr_for_me v1:172.21.15.99:0/0) 2023-10-11T06:10:49.455 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.453+0000 7fb8f6214640 1 --2- 172.21.15.99:0/3956401755 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f00c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.456 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.453+0000 7fb8f6214640 1 -- 172.21.15.99:0/3956401755 >> v1:172.21.15.99:6789/0 conn(0x7fb8f00affd0 legacy=0x7fb8f00b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.453+0000 7fb8f6214640 1 -- 172.21.15.99:0/3956401755 >> v1:172.21.15.84:6789/0 conn(0x7fb8f0159070 legacy=0x7fb8f00c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.453+0000 7fb8f4a11640 1 -- 172.21.15.99:0/3956401755 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304891552 0 0) 0x7fb8f00cadc0 con 0x7fb8f00affd0 2023-10-11T06:10:49.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.453+0000 7fb8f6214640 1 -- 172.21.15.99:0/3956401755 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00cb0a0 con 0x7fb8f00cb8a0 2023-10-11T06:10:49.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.454+0000 7fb8f6214640 1 --2- 172.21.15.99:0/3956401755 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f00c9460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009dc0 tx=0x7fb8e802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc6758ac5843bc67 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:49.457 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.454+0000 7fb8f4a11640 1 -- 172.21.15.99:0/3956401755 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e802cc40 con 0x7fb8f00cb8a0 2023-10-11T06:10:49.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.454+0000 7fb8f4a11640 1 -- 172.21.15.99:0/3956401755 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e802cda0 con 0x7fb8f00cb8a0 2023-10-11T06:10:49.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.454+0000 7fb8f4a11640 1 -- 172.21.15.99:0/3956401755 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8034660 con 0x7fb8f00cb8a0 2023-10-11T06:10:49.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.455+0000 7fb8f6a15640 1 -- 172.21.15.99:0/3956401755 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 msgr2=0x7fb8f00c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.455+0000 7fb8f6a15640 1 --2- 172.21.15.99:0/3956401755 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f00c9460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009dc0 tx=0x7fb8e802be90 comp rx=0 tx=0).stop 2023-10-11T06:10:49.458 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.455+0000 7fb8f6a15640 1 -- 172.21.15.99:0/3956401755 shutdown_connections 2023-10-11T06:10:49.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.455+0000 7fb8f6a15640 1 --2- 172.21.15.99:0/3956401755 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f00c9460 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.455+0000 7fb8f6a15640 1 -- 172.21.15.99:0/3956401755 >> 172.21.15.99:0/3956401755 conn(0x7fb8f0069c70 msgr2=0x7fb8f006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:49.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.455+0000 7fb8f6a15640 1 -- 172.21.15.99:0/3956401755 shutdown_connections 2023-10-11T06:10:49.459 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.455+0000 7fb8f6a15640 1 -- 172.21.15.99:0/3956401755 wait complete. 2023-10-11T06:10:49.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.457+0000 7fb8f6a15640 1 Processor -- start 2023-10-11T06:10:49.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.457+0000 7fb8f6a15640 1 -- start start 2023-10-11T06:10:49.462 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6a15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f00affd0 0x7fb8f00af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6a15640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6a15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f0159070 0x7fb8f00ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6a15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00afdb0 con 0x7fb8f00affd0 2023-10-11T06:10:49.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6a15640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f015ab80 con 0x7fb8f0159070 2023-10-11T06:10:49.463 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6a15640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f015ae60 con 0x7fb8f00cb8a0 2023-10-11T06:10:49.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f0159070 0x7fb8f00ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f5212640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f0159070 0x7fb8f00ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46042/0 (socket says 172.21.15.99:46042) 2023-10-11T06:10:49.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 -- 172.21.15.99:0/1615994438 learned_addr learned my addr 172.21.15.99:0/1615994438 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:49.464 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f5a13640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f00affd0 0x7fb8f00af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 -- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 msgr2=0x7fb8f01570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f01570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 -- 172.21.15.99:0/1615994438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f00affd0 msgr2=0x7fb8f00af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f00affd0 0x7fb8f00af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f6214640 1 -- 172.21.15.99:0/1615994438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8009a70 con 0x7fb8f0159070 2023-10-11T06:10:49.465 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.458+0000 7fb8f5212640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f01570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:10:49.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.459+0000 7fb8f6214640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f0159070 0x7fb8f00ae790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8037040 tx=0x7fb8e8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:49.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.459+0000 7fb8e67fc640 1 -- 172.21.15.99:0/1615994438 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8040650 con 0x7fb8f0159070 2023-10-11T06:10:49.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.459+0000 7fb8e67fc640 1 -- 172.21.15.99:0/1615994438 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e80407b0 con 0x7fb8f0159070 2023-10-11T06:10:49.466 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.459+0000 7fb8e67fc640 1 -- 172.21.15.99:0/1615994438 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8040b60 con 0x7fb8f0159070 2023-10-11T06:10:49.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.459+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f015b140 con 0x7fb8f0159070 2023-10-11T06:10:49.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.459+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f00ae270 con 0x7fb8f0159070 2023-10-11T06:10:49.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.459+0000 7fb8e67fc640 1 -- 172.21.15.99:0/1615994438 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb8e8049400 con 0x7fb8f0159070 2023-10-11T06:10:49.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.460+0000 7fb8e67fc640 1 -- 172.21.15.99:0/1615994438 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fb8e8049c30 con 0x7fb8f0159070 2023-10-11T06:10:49.467 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.460+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f004e890 con 0x7fb8f0159070 2023-10-11T06:10:49.468 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.463+0000 7fb8e67fc640 1 -- 172.21.15.99:0/1615994438 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb8e8067b00 con 0x7fb8f0159070 2023-10-11T06:10:49.541 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.539+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62ae3f3f-d323-46cd-8f29-289419f9f189", "id": 5} v 0) v1 -- 0x7fb8f00b41e0 con 0x7fb8f0159070 2023-10-11T06:10:49.548 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.547+0000 7fb8e67fc640 1 -- 172.21.15.99:0/1615994438 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62ae3f3f-d323-46cd-8f29-289419f9f189", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb8e8056030 con 0x7fb8f0159070 2023-10-11T06:10:49.549 INFO:teuthology.orchestra.run.smithi099.stdout:5 2023-10-11T06:10:49.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f0159070 msgr2=0x7fb8f00ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f0159070 0x7fb8f00ae790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8037040 tx=0x7fb8e8004a40 comp rx=0 tx=0).stop 2023-10-11T06:10:49.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 shutdown_connections 2023-10-11T06:10:49.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8f0159070 0x7fb8f00ae790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.551 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8f00cb8a0 0x7fb8f01570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 --2- 172.21.15.99:0/1615994438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f00affd0 0x7fb8f00af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 >> 172.21.15.99:0/1615994438 conn(0x7fb8f0069c70 msgr2=0x7fb8f00cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:49.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 shutdown_connections 2023-10-11T06:10:49.552 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.549+0000 7fb8f6a15640 1 -- 172.21.15.99:0/1615994438 wait complete. 2023-10-11T06:10:49.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 73564779-506c-4fd7-9fa7-d75e37a4be36 6 2023-10-11T06:10:49.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.761+0000 7f65cd6f7640 1 Processor -- start 2023-10-11T06:10:49.763 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.761+0000 7f65cd6f7640 1 -- start start 2023-10-11T06:10:49.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65cd6f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65c80afce0 0x7f65c80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65cd6f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c815dc40 0x7f65c80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.764 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65cd6f7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f65c80cdd60 0x7f65c80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65cd6f7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80b1590 con 0x7f65c80afce0 2023-10-11T06:10:49.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65cd6f7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80667c0 con 0x7f65c80cdd60 2023-10-11T06:10:49.765 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65cd6f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815a530 con 0x7f65c815dc40 2023-10-11T06:10:49.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65c67fc640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c815dc40 0x7f65c80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.767 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65c77fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f65c80cdd60 0x7f65c80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65c67fc640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c815dc40 0x7f65c80cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:37162/0 (socket says 172.21.15.99:37162) 2023-10-11T06:10:49.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65c67fc640 1 -- 172.21.15.99:0/1328687919 learned_addr learned my addr 172.21.15.99:0/1328687919 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:49.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.763+0000 7f65c6ffd640 1 --2- 172.21.15.99:0/1328687919 >> v2:172.21.15.84:3300/0 conn(0x7f65c80afce0 0x7f65c80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.764+0000 7f65c77fe640 1 -- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c815dc40 msgr2=0x7f65c80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:49.768 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.764+0000 7f65c77fe640 1 --2- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c815dc40 0x7f65c80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.764+0000 7f65c77fe640 1 -- 172.21.15.99:0/1328687919 >> v2:172.21.15.84:3300/0 conn(0x7f65c80afce0 msgr2=0x7f65c80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:49.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.764+0000 7f65c77fe640 1 --2- 172.21.15.99:0/1328687919 >> v2:172.21.15.84:3300/0 conn(0x7f65c80afce0 0x7f65c80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.764+0000 7f65c77fe640 1 -- 172.21.15.99:0/1328687919 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80b0cf0 con 0x7f65c80cdd60 2023-10-11T06:10:49.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.764+0000 7f65c67fc640 1 --2- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c815dc40 0x7f65c80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:10:49.769 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.765+0000 7f65c77fe640 1 --2- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80cdd60 0x7f65c80cad10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f65c80b3250 tx=0x7f65c002bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2462fb772db9f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:49.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.765+0000 7f65c5ffb640 1 -- 172.21.15.99:0/1328687919 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c002cb40 con 0x7f65c80cdd60 2023-10-11T06:10:49.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.765+0000 7f65c5ffb640 1 -- 172.21.15.99:0/1328687919 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c002cca0 con 0x7f65c80cdd60 2023-10-11T06:10:49.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.765+0000 7f65c5ffb640 1 -- 172.21.15.99:0/1328687919 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0034660 con 0x7f65c80cdd60 2023-10-11T06:10:49.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 -- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80cdd60 msgr2=0x7f65c80cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.770 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80cdd60 0x7f65c80cad10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f65c80b3250 tx=0x7f65c002bd90 comp rx=0 tx=0).stop 2023-10-11T06:10:49.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 -- 172.21.15.99:0/1328687919 shutdown_connections 2023-10-11T06:10:49.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80cdd60 0x7f65c80cad10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/1328687919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c815dc40 0x7f65c80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/1328687919 >> v2:172.21.15.84:3300/0 conn(0x7f65c80afce0 0x7f65c80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.771 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 -- 172.21.15.99:0/1328687919 >> 172.21.15.99:0/1328687919 conn(0x7f65c806a590 msgr2=0x7f65c8099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:49.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 -- 172.21.15.99:0/1328687919 shutdown_connections 2023-10-11T06:10:49.772 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.766+0000 7f65cd6f7640 1 -- 172.21.15.99:0/1328687919 wait complete. 2023-10-11T06:10:49.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.768+0000 7f65cd6f7640 1 Processor -- start 2023-10-11T06:10:49.774 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.768+0000 7f65cd6f7640 1 -- start start 2023-10-11T06:10:49.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65cd6f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80afce0 0x7f65c815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65cd6f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c80cdd60 0x7f65c80d0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65cd6f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c815dc40 0x7f65c80d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:49.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65cd6f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80c9250 con 0x7f65c815dc40 2023-10-11T06:10:49.775 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65cd6f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80c85d0 con 0x7f65c80afce0 2023-10-11T06:10:49.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65cd6f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80c88b0 con 0x7f65c80cdd60 2023-10-11T06:10:49.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80afce0 0x7f65c815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c67fc640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c80cdd60 0x7f65c80d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80afce0 0x7f65c815ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46058/0 (socket says 172.21.15.99:46058) 2023-10-11T06:10:49.776 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 -- 172.21.15.99:0/2860867659 learned_addr learned my addr 172.21.15.99:0/2860867659 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:49.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c77fe640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c815dc40 0x7f65c80d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:49.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 -- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c80cdd60 msgr2=0x7f65c80d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c80cdd60 0x7f65c80d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 -- 172.21.15.99:0/2860867659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c815dc40 msgr2=0x7f65c80d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.777 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c815dc40 0x7f65c80d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 -- 172.21.15.99:0/2860867659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0094d0 con 0x7f65c80afce0 2023-10-11T06:10:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c67fc640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c80cdd60 0x7f65c80d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:10:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.769+0000 7f65c6ffd640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80afce0 0x7f65c815ab60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d800 tx=0x7f65b800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.770+0000 7f65af7fe640 1 -- 172.21.15.99:0/2860867659 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80096a0 con 0x7f65c80afce0 2023-10-11T06:10:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.770+0000 7f65af7fe640 1 -- 172.21.15.99:0/2860867659 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b8009800 con 0x7f65c80afce0 2023-10-11T06:10:49.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.770+0000 7f65af7fe640 1 -- 172.21.15.99:0/2860867659 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8009b90 con 0x7f65c80afce0 2023-10-11T06:10:49.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.770+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c0009a70 con 0x7f65c80afce0 2023-10-11T06:10:49.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.770+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c80c8e70 con 0x7f65c80afce0 2023-10-11T06:10:49.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.770+0000 7f65af7fe640 1 -- 172.21.15.99:0/2860867659 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f65b801e460 con 0x7f65c80afce0 2023-10-11T06:10:49.779 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.770+0000 7f65af7fe640 1 -- 172.21.15.99:0/2860867659 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f65b801ed00 con 0x7f65c80afce0 2023-10-11T06:10:49.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.771+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c8067090 con 0x7f65c80afce0 2023-10-11T06:10:49.780 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.773+0000 7f65af7fe640 1 -- 172.21.15.99:0/2860867659 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f65b801e710 con 0x7f65c80afce0 2023-10-11T06:10:49.852 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.851+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "73564779-506c-4fd7-9fa7-d75e37a4be36", "id": 6} v 0) v1 -- 0x7f65c815e3f0 con 0x7f65c80afce0 2023-10-11T06:10:49.862 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.861+0000 7f65af7fe640 1 -- 172.21.15.99:0/2860867659 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73564779-506c-4fd7-9fa7-d75e37a4be36", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f65b801e980 con 0x7f65c80afce0 2023-10-11T06:10:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:6 2023-10-11T06:10:49.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80afce0 msgr2=0x7f65c815ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:49.864 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80afce0 0x7f65c815ab60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d800 tx=0x7f65b800dcc0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 shutdown_connections 2023-10-11T06:10:49.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65c815dc40 0x7f65c80d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.865 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65c80cdd60 0x7f65c80d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 --2- 172.21.15.99:0/2860867659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c80afce0 0x7f65c815ab60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:49.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 >> 172.21.15.99:0/2860867659 conn(0x7f65c806a590 msgr2=0x7f65c8159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:49.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 shutdown_connections 2023-10-11T06:10:49.866 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:49.862+0000 7f65cd6f7640 1 -- 172.21.15.99:0/2860867659 wait complete. 2023-10-11T06:10:49.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b 7 2023-10-11T06:10:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.069+0000 7fc6affff640 1 Processor -- start 2023-10-11T06:10:50.071 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.070+0000 7fc6affff640 1 -- start start 2023-10-11T06:10:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.071+0000 7fc6affff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6b00d0c10 0x7fc6b00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.073 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.071+0000 7fc6affff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b006b490 0x7fc6b006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.071+0000 7fc6affff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b015c6b0 con 0x7fc6b00cc020 2023-10-11T06:10:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.071+0000 7fc6affff640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00cc470 con 0x7fc6b00d0c10 2023-10-11T06:10:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.071+0000 7fc6affff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b006dc60 con 0x7fc6b006b490 2023-10-11T06:10:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6aeffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6b00d0c10 0x7fc6b00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.074 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6aeffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6b00d0c10 0x7fc6b00cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46068/0 (socket says 172.21.15.99:46068) 2023-10-11T06:10:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6aeffd640 1 -- 172.21.15.99:0/2675545615 learned_addr learned my addr 172.21.15.99:0/2675545615 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6af7fe640 1 --2- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b006b490 0x7fc6b006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6aeffd640 1 -- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b006b490 msgr2=0x7fc6b006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6aeffd640 1 --2- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b006b490 0x7fc6b006e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.075 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6aeffd640 1 -- 172.21.15.99:0/2675545615 >> v1:172.21.15.84:6789/0 conn(0x7fc6b00cc020 legacy=0x7fc6b006ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6aeffd640 1 -- 172.21.15.99:0/2675545615 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00cc680 con 0x7fc6b00d0c10 2023-10-11T06:10:50.076 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.072+0000 7fc6af7fe640 1 --2- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b006b490 0x7fc6b006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:10:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.073+0000 7fc6aeffd640 1 --2- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b00d0c10 0x7fc6b00cbae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4cedfddc05e68e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:50.077 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.073+0000 7fc6adffb640 1 -- 172.21.15.99:0/2675545615 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0031640 con 0x7fc6b00d0c10 2023-10-11T06:10:50.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.073+0000 7fc6adffb640 1 -- 172.21.15.99:0/2675545615 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a00317a0 con 0x7fc6b00d0c10 2023-10-11T06:10:50.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.073+0000 7fc6adffb640 1 -- 172.21.15.99:0/2675545615 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0031b30 con 0x7fc6b00d0c10 2023-10-11T06:10:50.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 -- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b00d0c10 msgr2=0x7fc6b00cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 --2- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b00d0c10 0x7fc6b00cbae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002b8f0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.078 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 -- 172.21.15.99:0/2675545615 shutdown_connections 2023-10-11T06:10:50.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 --2- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b006b490 0x7fc6b006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 --2- 172.21.15.99:0/2675545615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b00d0c10 0x7fc6b00cbae0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 -- 172.21.15.99:0/2675545615 >> 172.21.15.99:0/2675545615 conn(0x7fc6b006a590 msgr2=0x7fc6b00998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:50.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 -- 172.21.15.99:0/2675545615 shutdown_connections 2023-10-11T06:10:50.079 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.074+0000 7fc6affff640 1 -- 172.21.15.99:0/2675545615 wait complete. 2023-10-11T06:10:50.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.076+0000 7fc6affff640 1 Processor -- start 2023-10-11T06:10:50.080 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.076+0000 7fc6affff640 1 -- start start 2023-10-11T06:10:50.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b006b490 0x7fc6b0158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6affff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b00cc020 0x7fc6b01f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b00d0c10 0x7fc6b00ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.082 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6affff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b01f2a40 con 0x7fc6b00d0c10 2023-10-11T06:10:50.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6affff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b01f21a0 con 0x7fc6b006b490 2023-10-11T06:10:50.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6affff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b01f2480 con 0x7fc6b00cc020 2023-10-11T06:10:50.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6aeffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b006b490 0x7fc6b0158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6ae7fc640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b00cc020 0x7fc6b01f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.083 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6aeffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b006b490 0x7fc6b0158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46078/0 (socket says 172.21.15.99:46078) 2023-10-11T06:10:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.077+0000 7fc6aeffd640 1 -- 172.21.15.99:0/3107756813 learned_addr learned my addr 172.21.15.99:0/3107756813 (peer_addr_for_me v2:172.21.15.99:0/0) 2023-10-11T06:10:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6af7fe640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b00d0c10 0x7fc6b00ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6aeffd640 1 -- 172.21.15.99:0/3107756813 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b00cc020 msgr2=0x7fc6b01f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6aeffd640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b00cc020 0x7fc6b01f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6aeffd640 1 -- 172.21.15.99:0/3107756813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b00d0c10 msgr2=0x7fc6b00ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.084 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6aeffd640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b00d0c10 0x7fc6b00ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6aeffd640 1 -- 172.21.15.99:0/3107756813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00094d0 con 0x7fc6b006b490 2023-10-11T06:10:50.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6aeffd640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b006b490 0x7fc6b0158a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:50.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc697fff640 1 -- 172.21.15.99:0/3107756813 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a003ac60 con 0x7fc6b006b490 2023-10-11T06:10:50.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc697fff640 1 -- 172.21.15.99:0/3107756813 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a003adc0 con 0x7fc6b006b490 2023-10-11T06:10:50.085 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc697fff640 1 -- 172.21.15.99:0/3107756813 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0044630 con 0x7fc6b006b490 2023-10-11T06:10:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b01f2760 con 0x7fc6b006b490 2023-10-11T06:10:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.078+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b01f1e90 con 0x7fc6b006b490 2023-10-11T06:10:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.079+0000 7fc697fff640 1 -- 172.21.15.99:0/3107756813 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc6a002f050 con 0x7fc6b006b490 2023-10-11T06:10:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.079+0000 7fc697fff640 1 -- 172.21.15.99:0/3107756813 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fc6a0044a90 con 0x7fc6b006b490 2023-10-11T06:10:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.080+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b00b4cd0 con 0x7fc6b006b490 2023-10-11T06:10:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.082+0000 7fc697fff640 1 -- 172.21.15.99:0/3107756813 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc6b00b4cd0 con 0x7fc6b006b490 2023-10-11T06:10:50.160 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.159+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b", "id": 7} v 0) v1 -- 0x7fc6b004e890 con 0x7fc6b006b490 2023-10-11T06:10:50.170 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.169+0000 7fc697fff640 1 -- 172.21.15.99:0/3107756813 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc6a0035020 con 0x7fc6b006b490 2023-10-11T06:10:50.170 INFO:teuthology.orchestra.run.smithi099.stdout:7 2023-10-11T06:10:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.170+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b006b490 msgr2=0x7fc6b0158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.170+0000 7fc6affff640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b006b490 0x7fc6b0158a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a00046c0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.170+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 shutdown_connections 2023-10-11T06:10:50.172 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.170+0000 7fc6affff640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b00d0c10 0x7fc6b00ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.170+0000 7fc6affff640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6b00cc020 0x7fc6b01f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.170+0000 7fc6affff640 1 --2- 172.21.15.99:0/3107756813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6b006b490 0x7fc6b0158a30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.170+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 >> 172.21.15.99:0/3107756813 conn(0x7fc6b006a590 msgr2=0x7fc6b00cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:50.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.171+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 shutdown_connections 2023-10-11T06:10:50.173 INFO:teuthology.orchestra.run.smithi099.stderr:2023-10-11T06:10:50.171+0000 7fc6affff640 1 -- 172.21.15.99:0/3107756813 wait complete. 2023-10-11T06:10:50.186 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:50.184+0000 7f07bc43c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-11T06:10:50.189 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-11T06:10:50.189 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-11T06:10:50.198 INFO:tasks.ceph.osd.0:Started 2023-10-11T06:10:50.199 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-11T06:10:50.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-11T06:10:50.207 INFO:tasks.ceph.osd.1:Started 2023-10-11T06:10:50.207 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-11T06:10:50.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-11T06:10:50.213 INFO:tasks.ceph.osd.2:Started 2023-10-11T06:10:50.213 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-11T06:10:50.214 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-11T06:10:50.219 INFO:tasks.ceph.osd.3:Started 2023-10-11T06:10:50.220 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-11T06:10:50.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-11T06:10:50.227 INFO:tasks.ceph.osd.4:Started 2023-10-11T06:10:50.228 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-11T06:10:50.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-11T06:10:50.236 INFO:tasks.ceph.osd.5:Started 2023-10-11T06:10:50.236 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-11T06:10:50.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-11T06:10:50.242 INFO:tasks.ceph.osd.6:Started 2023-10-11T06:10:50.243 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-11T06:10:50.243 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-11T06:10:50.248 INFO:tasks.ceph.osd.7:Started 2023-10-11T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:10:50.307 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:50.306+0000 7f325ea32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-11T06:10:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.468+0000 7faeabaa4640 1 Processor -- start 2023-10-11T06:10:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.469+0000 7faeabaa4640 1 -- start start 2023-10-11T06:10:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faeabaa4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faea415c290 0x7faea40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faeabaa4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea40d2a60 con 0x7faea40d25e0 2023-10-11T06:10:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faeabaa4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea406cd10 con 0x7faea40d0c20 2023-10-11T06:10:50.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faeabaa4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea40afb60 con 0x7faea415c290 2023-10-11T06:10:50.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faea9840640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faea415c290 0x7faea40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faea903f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7faea40d25e0 0x7faea40d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39230/0 (socket says 172.21.15.84:39230) 2023-10-11T06:10:50.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faea903f640 1 -- 172.21.15.84:0/2539857739 learned_addr learned my addr 172.21.15.84:0/2539857739 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:10:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faea9840640 1 -- 172.21.15.84:0/2539857739 >> v1:172.21.15.99:6789/0 conn(0x7faea40d0c20 legacy=0x7faea40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faea9840640 1 -- 172.21.15.84:0/2539857739 >> v1:172.21.15.84:6789/0 conn(0x7faea40d25e0 legacy=0x7faea40d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.471+0000 7faea9840640 1 -- 172.21.15.84:0/2539857739 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea40afe40 con 0x7faea415c290 2023-10-11T06:10:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.470+0000 7faea883e640 1 -- 172.21.15.84:0/2539857739 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664947847 0 0) 0x7faea406cd10 con 0x7faea40d0c20 2023-10-11T06:10:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faea9840640 1 --2- 172.21.15.84:0/2539857739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40af4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faea00097a0 tx=0x7faea002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8176723108d72ba server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faea883e640 1 -- 172.21.15.84:0/2539857739 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0030710 con 0x7faea415c290 2023-10-11T06:10:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faea883e640 1 -- 172.21.15.84:0/2539857739 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea002e050 con 0x7faea415c290 2023-10-11T06:10:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faea883e640 1 -- 172.21.15.84:0/2539857739 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0030a90 con 0x7faea415c290 2023-10-11T06:10:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faeabaa4640 1 -- 172.21.15.84:0/2539857739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 msgr2=0x7faea40af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faeabaa4640 1 --2- 172.21.15.84:0/2539857739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40af4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faea00097a0 tx=0x7faea002b710 comp rx=0 tx=0).stop 2023-10-11T06:10:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faeabaa4640 1 -- 172.21.15.84:0/2539857739 shutdown_connections 2023-10-11T06:10:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faeabaa4640 1 --2- 172.21.15.84:0/2539857739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40af4d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faeabaa4640 1 -- 172.21.15.84:0/2539857739 >> 172.21.15.84:0/2539857739 conn(0x7faea4071cc0 msgr2=0x7faea409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.472+0000 7faeabaa4640 1 -- 172.21.15.84:0/2539857739 shutdown_connections 2023-10-11T06:10:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.473+0000 7faeabaa4640 1 -- 172.21.15.84:0/2539857739 wait complete. 2023-10-11T06:10:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.474+0000 7faeabaa4640 1 Processor -- start 2023-10-11T06:10:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.474+0000 7faeabaa4640 1 -- start start 2023-10-11T06:10:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeabaa4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faea40d0c20 0x7faea41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeabaa4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea40d25e0 0x7faea41f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeabaa4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeabaa4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea4152a30 con 0x7faea415c290 2023-10-11T06:10:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeabaa4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea40b5770 con 0x7faea40d25e0 2023-10-11T06:10:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeabaa4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea40b59c0 con 0x7faea40d0c20 2023-10-11T06:10:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeaa041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faea903f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea40d25e0 0x7faea41f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.475+0000 7faeaa041640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40b7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42098/0 (socket says 172.21.15.84:42098) 2023-10-11T06:10:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeaa041640 1 -- 172.21.15.84:0/1397132229 learned_addr learned my addr 172.21.15.84:0/1397132229 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:10:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faea9840640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faea40d0c20 0x7faea41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeaa041640 1 -- 172.21.15.84:0/1397132229 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faea40d0c20 msgr2=0x7faea41524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeaa041640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faea40d0c20 0x7faea41524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeaa041640 1 -- 172.21.15.84:0/1397132229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea40d25e0 msgr2=0x7faea41f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeaa041640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea40d25e0 0x7faea41f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeaa041640 1 -- 172.21.15.84:0/1397132229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea0009400 con 0x7faea415c290 2023-10-11T06:10:50.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeaa041640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40b7e60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c7c0 tx=0x7fae9c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:50.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7fae9a7fc640 1 -- 172.21.15.84:0/1397132229 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c003da0 con 0x7faea415c290 2023-10-11T06:10:50.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7fae9a7fc640 1 -- 172.21.15.84:0/1397132229 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c005070 con 0x7faea415c290 2023-10-11T06:10:50.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea40b5ca0 con 0x7faea415c290 2023-10-11T06:10:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.476+0000 7fae9a7fc640 1 -- 172.21.15.84:0/1397132229 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c005420 con 0x7faea415c290 2023-10-11T06:10:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.477+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea40b6260 con 0x7faea415c290 2023-10-11T06:10:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.477+0000 7fae9a7fc640 1 -- 172.21.15.84:0/1397132229 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fae9c00a640 con 0x7faea415c290 2023-10-11T06:10:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.477+0000 7fae9a7fc640 1 -- 172.21.15.84:0/1397132229 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fae9c012400 con 0x7faea415c290 2023-10-11T06:10:50.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.478+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea40b0690 con 0x7faea415c290 2023-10-11T06:10:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.480+0000 7fae9a7fc640 1 -- 172.21.15.84:0/1397132229 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fae9c017070 con 0x7faea415c290 2023-10-11T06:10:50.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.542+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faea4073ed0 con 0x7faea415c290 2023-10-11T06:10:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.543+0000 7fae9a7fc640 1 -- 172.21.15.84:0/1397132229 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fae9c00a8f0 con 0x7faea415c290 2023-10-11T06:10:50.545 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:10:50.545 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":10,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:10:50.161835+0000","last_up_change":"0.000000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:10:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.544+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 msgr2=0x7faea40b7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.544+0000 7faeabaa4640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40b7e60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00c7c0 tx=0x7fae9c00cc80 comp rx=0 tx=0).stop 2023-10-11T06:10:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.544+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 shutdown_connections 2023-10-11T06:10:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.544+0000 7faeabaa4640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faea415c290 0x7faea40b7e60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.544+0000 7faeabaa4640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faea40d25e0 0x7faea41f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.544+0000 7faeabaa4640 1 --2- 172.21.15.84:0/1397132229 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faea40d0c20 0x7faea41524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.544+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 >> 172.21.15.84:0/1397132229 conn(0x7faea4071cc0 msgr2=0x7faea40b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:50.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.545+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 shutdown_connections 2023-10-11T06:10:50.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:10:50.545+0000 7faeabaa4640 1 -- 172.21.15.84:0/1397132229 wait complete. 2023-10-11T06:10:50.562 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-11T06:10:50.562 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-11T06:10:50.568 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:50.567+0000 7f07bc43c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-11T06:10:50.629 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:50.627+0000 7f07bc43c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-11T06:10:50.632 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:50.631+0000 7f325ea32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-11T06:10:50.693 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:50.692+0000 7f325ea32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-11T06:10:50.743 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:50.742+0000 7f07bc43c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-11T06:10:50.814 INFO:tasks.ceph.mgr.x.smithi099.stderr:2023-10-11T06:10:50.813+0000 7f325ea32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-11T06:10:50.897 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-11T06:10:50.896+0000 7f447dbe2500 -1 Falling back to public interface 2023-10-11T06:10:50.915 INFO:tasks.ceph.osd.4.smithi099.stderr:2023-10-11T06:10:50.914+0000 7f3a3f010500 -1 Falling back to public interface 2023-10-11T06:10:50.921 INFO:tasks.ceph.osd.5.smithi099.stderr:2023-10-11T06:10:50.920+0000 7f0445424500 -1 Falling back to public interface 2023-10-11T06:10:50.941 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-11T06:10:50.940+0000 7ff62220f500 -1 Falling back to public interface 2023-10-11T06:10:50.946 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-11T06:10:50.945+0000 7eff02a14500 -1 Falling back to public interface 2023-10-11T06:10:50.946 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-11T06:10:50.945+0000 7f52d4802500 -1 Falling back to public interface 2023-10-11T06:10:50.947 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-11T06:10:50.946+0000 7eff01488640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6815/0: (98) Address already in use 2023-10-11T06:10:50.950 INFO:tasks.ceph.osd.7.smithi099.stderr:2023-10-11T06:10:50.949+0000 7f271b40c500 -1 Falling back to public interface 2023-10-11T06:10:50.954 INFO:tasks.ceph.osd.6.smithi099.stderr:2023-10-11T06:10:50.953+0000 7f6449cfa500 -1 Falling back to public interface 2023-10-11T06:10:51.368 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-11T06:10:51.546 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.545+0000 7fa36277b640 1 Processor -- start 2023-10-11T06:10:51.546 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.545+0000 7fa36277b640 1 -- start start 2023-10-11T06:10:51.547 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.546+0000 7fa36277b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa35c0b0c20 0x7fa35c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:51.548 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.546+0000 7fa36277b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0b0080 con 0x7fa35c0d27f0 2023-10-11T06:10:51.548 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.546+0000 7fa36277b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0b15a0 con 0x7fa35c0b19b0 2023-10-11T06:10:51.548 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.546+0000 7fa36277b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c0d0f90 con 0x7fa35c0b0c20 2023-10-11T06:10:51.548 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.546+0000 7fa361f7a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa35c0b0c20 0x7fa35c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:51.548 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.546+0000 7fa361f7a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa35c0b0c20 0x7fa35c0b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42164/0 (socket says 172.21.15.84:42164) 2023-10-11T06:10:51.549 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.546+0000 7fa361f7a640 1 -- 172.21.15.84:0/1118172923 learned_addr learned my addr 172.21.15.84:0/1118172923 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:10:51.550 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.547+0000 7fa361f7a640 1 -- 172.21.15.84:0/1118172923 >> v1:172.21.15.99:6789/0 conn(0x7fa35c0b19b0 legacy=0x7fa35c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:51.550 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.547+0000 7fa361f7a640 1 -- 172.21.15.84:0/1118172923 >> v1:172.21.15.84:6789/0 conn(0x7fa35c0d27f0 legacy=0x7fa35c0d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:51.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.547+0000 7fa361f7a640 1 -- 172.21.15.84:0/1118172923 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0af470 con 0x7fa35c0b0c20 2023-10-11T06:10:51.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.547+0000 7fa353fff640 1 -- 172.21.15.84:0/1118172923 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640217109 0 0) 0x7fa35c0b0080 con 0x7fa35c0d27f0 2023-10-11T06:10:51.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.548+0000 7fa361f7a640 1 --2- 172.21.15.84:0/1118172923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0b0c20 0x7fa35c0b1060 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa354009e50 tx=0x7fa35402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff5fe7a1e7f330 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:51.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.548+0000 7fa353fff640 1 -- 172.21.15.84:0/1118172923 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35402cbb0 con 0x7fa35c0b0c20 2023-10-11T06:10:51.551 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.548+0000 7fa353fff640 1 -- 172.21.15.84:0/1118172923 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35402cd10 con 0x7fa35c0b0c20 2023-10-11T06:10:51.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.548+0000 7fa353fff640 1 -- 172.21.15.84:0/1118172923 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354034610 con 0x7fa35c0b0c20 2023-10-11T06:10:51.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.549+0000 7fa36277b640 1 -- 172.21.15.84:0/1118172923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0b0c20 msgr2=0x7fa35c0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:51.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.549+0000 7fa36277b640 1 --2- 172.21.15.84:0/1118172923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0b0c20 0x7fa35c0b1060 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa354009e50 tx=0x7fa35402c010 comp rx=0 tx=0).stop 2023-10-11T06:10:51.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.549+0000 7fa36277b640 1 -- 172.21.15.84:0/1118172923 shutdown_connections 2023-10-11T06:10:51.552 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.549+0000 7fa36277b640 1 --2- 172.21.15.84:0/1118172923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0b0c20 0x7fa35c0b1060 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:51.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.549+0000 7fa36277b640 1 -- 172.21.15.84:0/1118172923 >> 172.21.15.84:0/1118172923 conn(0x7fa35c09bbe0 msgr2=0x7fa35c089ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:51.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.549+0000 7fa36277b640 1 -- 172.21.15.84:0/1118172923 shutdown_connections 2023-10-11T06:10:51.553 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.549+0000 7fa36277b640 1 -- 172.21.15.84:0/1118172923 wait complete. 2023-10-11T06:10:51.556 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.551+0000 7fa36277b640 1 Processor -- start 2023-10-11T06:10:51.556 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.551+0000 7fa36277b640 1 -- start start 2023-10-11T06:10:51.557 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa36277b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa35c0b0c20 0x7fa35c0b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:51.557 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa36277b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa35c0b19b0 0x7fa35c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:51.557 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa36277b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0d27f0 0x7fa35c0b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:51.557 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa36277b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0ce840 con 0x7fa35c0d27f0 2023-10-11T06:10:51.558 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa36277b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0c70d0 con 0x7fa35c0b19b0 2023-10-11T06:10:51.558 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa36277b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0ce180 con 0x7fa35c0b0c20 2023-10-11T06:10:51.559 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0d27f0 0x7fa35c0b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:51.559 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0d27f0 0x7fa35c0b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42168/0 (socket says 172.21.15.84:42168) 2023-10-11T06:10:51.559 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 -- 172.21.15.84:0/4193453983 learned_addr learned my addr 172.21.15.84:0/4193453983 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:10:51.559 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361779640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa35c0b0c20 0x7fa35c0b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:51.560 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa360f78640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa35c0b19b0 0x7fa35c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:51.560 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 -- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa35c0b0c20 msgr2=0x7fa35c0b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:51.560 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa35c0b0c20 0x7fa35c0b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:51.561 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 -- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa35c0b19b0 msgr2=0x7fa35c159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:51.561 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa35c0b19b0 0x7fa35c159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:51.561 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361f7a640 1 -- 172.21.15.84:0/4193453983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009b30 con 0x7fa35c0d27f0 2023-10-11T06:10:51.561 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.552+0000 7fa361779640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa35c0b0c20 0x7fa35c0b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:10:51.561 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.553+0000 7fa361f7a640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0d27f0 0x7fa35c0b6ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa354036040 tx=0x7fa354004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:51.562 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.553+0000 7fa351ffb640 1 -- 172.21.15.84:0/4193453983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354009510 con 0x7fa35c0d27f0 2023-10-11T06:10:51.562 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.553+0000 7fa351ffb640 1 -- 172.21.15.84:0/4193453983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35403e450 con 0x7fa35c0d27f0 2023-10-11T06:10:51.562 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.553+0000 7fa351ffb640 1 -- 172.21.15.84:0/4193453983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35403e7e0 con 0x7fa35c0d27f0 2023-10-11T06:10:51.562 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.553+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c0ce460 con 0x7fa35c0d27f0 2023-10-11T06:10:51.562 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.553+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c0b1ee0 con 0x7fa35c0d27f0 2023-10-11T06:10:51.563 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.554+0000 7fa351ffb640 1 -- 172.21.15.84:0/4193453983 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 2) v1 ==== 44409+0+0 (secure 0 0 0) 0x7fa35403e940 con 0x7fa35c0d27f0 2023-10-11T06:10:51.563 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.554+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c0ce980 con 0x7fa35c0d27f0 2023-10-11T06:10:51.563 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.554+0000 7fa351ffb640 1 -- 172.21.15.84:0/4193453983 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa35406bdc0 con 0x7fa35c0d27f0 2023-10-11T06:10:51.563 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.557+0000 7fa351ffb640 1 -- 172.21.15.84:0/4193453983 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa35c0ce980 con 0x7fa35c0d27f0 2023-10-11T06:10:51.618 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.617+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa35c0ce980 con 0x7fa35c0d27f0 2023-10-11T06:10:51.620 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.619+0000 7fa351ffb640 1 -- 172.21.15.84:0/4193453983 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fa35c0ce980 con 0x7fa35c0d27f0 2023-10-11T06:10:51.620 INFO:teuthology.misc.health.smithi084.stdout: 2023-10-11T06:10:51.621 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":10,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:10:50.161835+0000","last_up_change":"0.000000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:10:51.623 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0d27f0 msgr2=0x7fa35c0b6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:51.623 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0d27f0 0x7fa35c0b6ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa354036040 tx=0x7fa354004a40 comp rx=0 tx=0).stop 2023-10-11T06:10:51.624 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 shutdown_connections 2023-10-11T06:10:51.624 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0d27f0 0x7fa35c0b6ca0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:51.624 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa35c0b19b0 0x7fa35c159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:51.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 --2- 172.21.15.84:0/4193453983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa35c0b0c20 0x7fa35c0b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:51.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 >> 172.21.15.84:0/4193453983 conn(0x7fa35c09bbe0 msgr2=0x7fa35c15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:51.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 shutdown_connections 2023-10-11T06:10:51.626 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:51.620+0000 7fa36277b640 1 -- 172.21.15.84:0/4193453983 wait complete. 2023-10-11T06:10:51.634 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-11T06:10:51.790 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-10-11T06:10:51.789+0000 7f075d830640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-11T06:10:54.485 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-11T06:10:54.484+0000 7ff62220f500 -1 osd.1 0 log_to_monitors true 2023-10-11T06:10:54.492 INFO:tasks.ceph.osd.5.smithi099.stderr:2023-10-11T06:10:54.491+0000 7f0445424500 -1 osd.5 0 log_to_monitors true 2023-10-11T06:10:54.510 INFO:tasks.ceph.osd.7.smithi099.stderr:2023-10-11T06:10:54.509+0000 7f271b40c500 -1 osd.7 0 log_to_monitors true 2023-10-11T06:10:54.520 INFO:tasks.ceph.osd.6.smithi099.stderr:2023-10-11T06:10:54.519+0000 7f6449cfa500 -1 osd.6 0 log_to_monitors true 2023-10-11T06:10:54.526 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-11T06:10:54.525+0000 7eff02a14500 -1 osd.2 0 log_to_monitors true 2023-10-11T06:10:54.530 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-11T06:10:54.529+0000 7f52d4802500 -1 osd.3 0 log_to_monitors true 2023-10-11T06:10:54.666 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-11T06:10:54.665+0000 7f447dbe2500 -1 osd.0 0 log_to_monitors true 2023-10-11T06:10:54.701 INFO:tasks.ceph.osd.4.smithi099.stderr:2023-10-11T06:10:54.700+0000 7f3a3f010500 -1 osd.4 0 log_to_monitors true 2023-10-11T06:10:55.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:10:55.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:10:58.423 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-11T06:10:58.422+0000 7ff61d276640 -1 osd.1 0 waiting for initial osdmap 2023-10-11T06:10:58.430 INFO:tasks.ceph.osd.4.smithi099.stderr:2023-10-11T06:10:58.429+0000 7f3a3b98c640 -1 osd.4 0 waiting for initial osdmap 2023-10-11T06:10:58.436 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-11T06:10:58.435+0000 7f447ab87640 -1 osd.0 0 waiting for initial osdmap 2023-10-11T06:10:58.438 INFO:tasks.ceph.osd.6.smithi099.stderr:2023-10-11T06:10:58.437+0000 7f6446c9d640 -1 osd.6 0 waiting for initial osdmap 2023-10-11T06:10:58.439 INFO:tasks.ceph.osd.5.smithi099.stderr:2023-10-11T06:10:58.438+0000 7f0440542640 -1 osd.5 0 waiting for initial osdmap 2023-10-11T06:10:58.440 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-11T06:10:58.441 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-10-11T06:10:58.440+0000 7ff6158a7640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.442 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-11T06:10:58.441+0000 7efefdc49640 -1 osd.2 0 waiting for initial osdmap 2023-10-11T06:10:58.443 INFO:tasks.ceph.osd.7.smithi099.stderr:2023-10-11T06:10:58.442+0000 7f271650c640 -1 osd.7 0 waiting for initial osdmap 2023-10-11T06:10:58.449 INFO:tasks.ceph.osd.4.smithi099.stderr:2023-10-11T06:10:58.448+0000 7f3a32782640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.453 INFO:tasks.ceph.osd.5.smithi099.stderr:2023-10-11T06:10:58.452+0000 7f0438b73640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.453 INFO:tasks.ceph.osd.6.smithi099.stderr:2023-10-11T06:10:58.452+0000 7f643da93640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.454 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-10-11T06:10:58.453+0000 7f447197d640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.457 INFO:tasks.ceph.osd.7.smithi099.stderr:2023-10-11T06:10:58.456+0000 7f270eb3d640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.461 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-10-11T06:10:58.460+0000 7efef627a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.469 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-11T06:10:58.468+0000 7f52d17a5640 -1 osd.3 0 waiting for initial osdmap 2023-10-11T06:10:58.484 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-10-11T06:10:58.483+0000 7f52c859b640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-11T06:10:58.586 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.585+0000 7f51b1bbb640 1 Processor -- start 2023-10-11T06:10:58.619 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.585+0000 7f51b1bbb640 1 -- start start 2023-10-11T06:10:58.623 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.586+0000 7f51b1bbb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51ac0b4d00 0x7f51ac15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:58.624 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.586+0000 7f51b1bbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac15d2a0 0x7f51ac0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:58.624 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51b1bbb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac15a450 con 0x7f51ac15a080 2023-10-11T06:10:58.624 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51b1bbb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac0cfb50 con 0x7f51ac0b4d00 2023-10-11T06:10:58.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51b1bbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0cf220 con 0x7f51ac15d2a0 2023-10-11T06:10:58.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51a3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51ac0b4d00 0x7f51ac15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:58.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51a3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51ac0b4d00 0x7f51ac15cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42198/0 (socket says 172.21.15.84:42198) 2023-10-11T06:10:58.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51a3fff640 1 -- 172.21.15.84:0/1926645418 learned_addr learned my addr 172.21.15.84:0/1926645418 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:10:58.625 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51a3fff640 1 -- 172.21.15.84:0/1926645418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac15d2a0 msgr2=0x7f51ac0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:10:58.626 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51a3fff640 1 --2- 172.21.15.84:0/1926645418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac15d2a0 0x7f51ac0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.626 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51a3fff640 1 -- 172.21.15.84:0/1926645418 >> v1:172.21.15.84:6789/0 conn(0x7f51ac15a080 legacy=0x7f51ac0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:58.626 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.587+0000 7f51a3fff640 1 -- 172.21.15.84:0/1926645418 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0cf500 con 0x7f51ac0b4d00 2023-10-11T06:10:58.626 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.588+0000 7f51a3fff640 1 --2- 172.21.15.84:0/1926645418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac0b4d00 0x7f51ac15cd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5194009870 tx=0x7f519402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7593596463580566 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:58.626 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.588+0000 7f51aaffd640 1 -- 172.21.15.84:0/1926645418 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194031860 con 0x7f51ac0b4d00 2023-10-11T06:10:58.627 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51aaffd640 1 -- 172.21.15.84:0/1926645418 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940319c0 con 0x7f51ac0b4d00 2023-10-11T06:10:58.627 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51aaffd640 1 -- 172.21.15.84:0/1926645418 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194031d50 con 0x7f51ac0b4d00 2023-10-11T06:10:58.627 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 -- 172.21.15.84:0/1926645418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac0b4d00 msgr2=0x7f51ac15cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:58.627 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/1926645418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac0b4d00 0x7f51ac15cd60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5194009870 tx=0x7f519402b900 comp rx=0 tx=0).stop 2023-10-11T06:10:58.627 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 -- 172.21.15.84:0/1926645418 shutdown_connections 2023-10-11T06:10:58.628 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/1926645418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac15d2a0 0x7f51ac0d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.628 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/1926645418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac0b4d00 0x7f51ac15cd60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.628 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 -- 172.21.15.84:0/1926645418 >> 172.21.15.84:0/1926645418 conn(0x7f51ac06afc0 msgr2=0x7f51ac06b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:58.628 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 -- 172.21.15.84:0/1926645418 shutdown_connections 2023-10-11T06:10:58.628 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.589+0000 7f51b1bbb640 1 -- 172.21.15.84:0/1926645418 wait complete. 2023-10-11T06:10:58.629 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.591+0000 7f51b1bbb640 1 Processor -- start 2023-10-11T06:10:58.629 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.591+0000 7f51b1bbb640 1 -- start start 2023-10-11T06:10:58.629 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51b1bbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac0b4d00 0x7f51ac1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:58.629 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51b1bbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac15a080 0x7f51ac0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:58.629 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51b1bbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d2a0 0x7f51ac0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:58.630 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51b1bbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0b5cd0 con 0x7f51ac15a080 2023-10-11T06:10:58.630 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51b1bbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0b5fb0 con 0x7f51ac15d2a0 2023-10-11T06:10:58.630 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51b1bbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0b6290 con 0x7f51ac0b4d00 2023-10-11T06:10:58.630 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac15a080 0x7f51ac0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:58.630 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac15a080 0x7f51ac0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42212/0 (socket says 172.21.15.84:42212) 2023-10-11T06:10:58.631 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51a3fff640 1 -- 172.21.15.84:0/3206196604 learned_addr learned my addr 172.21.15.84:0/3206196604 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:10:58.631 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51ab7fe640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac0b4d00 0x7f51ac1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:58.631 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51abfff640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d2a0 0x7f51ac0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:58.631 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51a3fff640 1 -- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac0b4d00 msgr2=0x7f51ac1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:58.631 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51a3fff640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac0b4d00 0x7f51ac1583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.631 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51a3fff640 1 -- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d2a0 msgr2=0x7f51ac0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:58.632 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.592+0000 7f51a3fff640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d2a0 0x7f51ac0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.632 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51a3fff640 1 -- 172.21.15.84:0/3206196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940094d0 con 0x7f51ac15a080 2023-10-11T06:10:58.632 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51abfff640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d2a0 0x7f51ac0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:10:58.632 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51a3fff640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac15a080 0x7f51ac0cae60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5194009870 tx=0x7f51940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:58.632 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51a8ff9640 1 -- 172.21.15.84:0/3206196604 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194031810 con 0x7f51ac15a080 2023-10-11T06:10:58.633 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51a8ff9640 1 -- 172.21.15.84:0/3206196604 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940443f0 con 0x7f51ac15a080 2023-10-11T06:10:58.633 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51a8ff9640 1 -- 172.21.15.84:0/3206196604 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940447a0 con 0x7f51ac15a080 2023-10-11T06:10:58.633 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac0b6570 con 0x7f51ac15a080 2023-10-11T06:10:58.633 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.593+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac0b59a0 con 0x7f51ac15a080 2023-10-11T06:10:58.633 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.594+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac0b4b20 con 0x7f51ac15a080 2023-10-11T06:10:58.634 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.595+0000 7f51a8ff9640 1 -- 172.21.15.84:0/3206196604 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7f519404d450 con 0x7f51ac15a080 2023-10-11T06:10:58.634 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.596+0000 7f51a8ff9640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f518c06a7a0 0x7f518c06cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:10:58.634 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.596+0000 7f51a8ff9640 1 -- 172.21.15.84:0/3206196604 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f51940aa800 con 0x7f51ac15a080 2023-10-11T06:10:58.634 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.596+0000 7f51ab7fe640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f518c06a7a0 0x7f518c06cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:10:58.634 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.597+0000 7f51ab7fe640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f518c06a7a0 0x7f518c06cc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f519c000f60 tx=0x7f519c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:10:58.635 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.601+0000 7f51a8ff9640 1 -- 172.21.15.84:0/3206196604 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f51940769b0 con 0x7f51ac15a080 2023-10-11T06:10:58.780 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.779+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51ac0b7190 con 0x7f51ac15a080 2023-10-11T06:10:58.781 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.779+0000 7f51a8ff9640 1 -- 172.21.15.84:0/3206196604 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5194035070 con 0x7f51ac15a080 2023-10-11T06:10:58.781 INFO:teuthology.misc.health.smithi084.stdout: 2023-10-11T06:10:58.781 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":12,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:10:56.094275+0000","last_up_change":"0.000000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:10:58.783 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f518c06a7a0 msgr2=0x7f518c06cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:58.783 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f518c06a7a0 0x7f518c06cc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f519c000f60 tx=0x7f519c009040 comp rx=0 tx=0).stop 2023-10-11T06:10:58.784 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac15a080 msgr2=0x7f51ac0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:10:58.784 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac15a080 0x7f51ac0cae60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5194009870 tx=0x7f51940046c0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.784 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 shutdown_connections 2023-10-11T06:10:58.784 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f518c06a7a0 0x7f518c06cc50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.784 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d2a0 0x7f51ac0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.785 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac15a080 0x7f51ac0cae60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.785 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 --2- 172.21.15.84:0/3206196604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51ac0b4d00 0x7f51ac1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:10:58.785 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 >> 172.21.15.84:0/3206196604 conn(0x7f51ac06afc0 msgr2=0x7f51ac15b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:10:58.785 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 shutdown_connections 2023-10-11T06:10:58.785 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:10:58.782+0000 7f51b1bbb640 1 -- 172.21.15.84:0/3206196604 wait complete. 2023-10-11T06:10:58.797 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-11T06:11:05.603 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:05.769 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.767+0000 7fa1c5004640 1 Processor -- start 2023-10-11T06:11:05.769 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.767+0000 7fa1c5004640 1 -- start start 2023-10-11T06:11:05.770 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.768+0000 7fa1c5004640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa1c01590c0 0x7fa1c00cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:05.771 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.768+0000 7fa1c5004640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006bd30 0x7fa1c00b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:05.771 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.768+0000 7fa1c5004640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c00c9950 con 0x7fa1c006c1b0 2023-10-11T06:11:05.771 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.768+0000 7fa1c5004640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c006cfb0 con 0x7fa1c01590c0 2023-10-11T06:11:05.771 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.768+0000 7fa1c5004640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c006c710 con 0x7fa1c006bd30 2023-10-11T06:11:05.772 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.769+0000 7fa1bf59e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa1c006c1b0 0x7fa1c00ca0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38464/0 (socket says 172.21.15.84:38464) 2023-10-11T06:11:05.772 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.769+0000 7fa1bf59e640 1 -- 172.21.15.84:0/3585143613 learned_addr learned my addr 172.21.15.84:0/3585143613 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:05.773 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.769+0000 7fa1bed9d640 1 --2- 172.21.15.84:0/3585143613 >> v2:172.21.15.99:3300/0 conn(0x7fa1c01590c0 0x7fa1c00cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:05.773 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.769+0000 7fa1b7fff640 1 --2- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006bd30 0x7fa1c00b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:05.773 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.770+0000 7fa1bed9d640 1 -- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006bd30 msgr2=0x7fa1c00b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:05.773 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.770+0000 7fa1bed9d640 1 --2- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006bd30 0x7fa1c00b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:05.773 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.770+0000 7fa1bed9d640 1 -- 172.21.15.84:0/3585143613 >> v1:172.21.15.84:6789/0 conn(0x7fa1c006c1b0 legacy=0x7fa1c00ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:05.774 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.770+0000 7fa1bed9d640 1 -- 172.21.15.84:0/3585143613 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c006c9f0 con 0x7fa1c01590c0 2023-10-11T06:11:05.775 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.771+0000 7fa1bed9d640 1 --2- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 0x7fa1c00cc3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00097a0 tx=0x7fa1b002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caab3b9434ff5528 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:05.775 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.771+0000 7fa1be59c640 1 -- 172.21.15.84:0/3585143613 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0030800 con 0x7fa1c01590c0 2023-10-11T06:11:05.775 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.771+0000 7fa1be59c640 1 -- 172.21.15.84:0/3585143613 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0030960 con 0x7fa1c01590c0 2023-10-11T06:11:05.775 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.771+0000 7fa1be59c640 1 -- 172.21.15.84:0/3585143613 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0030cf0 con 0x7fa1c01590c0 2023-10-11T06:11:05.776 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 -- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 msgr2=0x7fa1c00cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:05.776 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 --2- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 0x7fa1c00cc3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00097a0 tx=0x7fa1b002b940 comp rx=0 tx=0).stop 2023-10-11T06:11:05.776 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 -- 172.21.15.84:0/3585143613 shutdown_connections 2023-10-11T06:11:05.776 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 --2- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006bd30 0x7fa1c00b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:05.776 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 --2- 172.21.15.84:0/3585143613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 0x7fa1c00cc3c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:05.776 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 -- 172.21.15.84:0/3585143613 >> 172.21.15.84:0/3585143613 conn(0x7fa1c00cb270 msgr2=0x7fa1c00cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:05.777 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 -- 172.21.15.84:0/3585143613 shutdown_connections 2023-10-11T06:11:05.777 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.772+0000 7fa1c5004640 1 -- 172.21.15.84:0/3585143613 wait complete. 2023-10-11T06:11:05.778 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.774+0000 7fa1c5004640 1 Processor -- start 2023-10-11T06:11:05.779 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.774+0000 7fa1c5004640 1 -- start start 2023-10-11T06:11:05.779 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1c5004640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1c006bd30 0x7fa1c014baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:05.779 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1c5004640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006c1b0 0x7fa1c0158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:05.779 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1c5004640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 0x7fa1c00c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:05.779 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1c5004640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c014b6d0 con 0x7fa1c006bd30 2023-10-11T06:11:05.780 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1c5004640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c0165830 con 0x7fa1c01590c0 2023-10-11T06:11:05.780 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1c5004640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c014c030 con 0x7fa1c006c1b0 2023-10-11T06:11:05.780 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1bed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1c006bd30 0x7fa1c014baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:05.780 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1bed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1c006bd30 0x7fa1c014baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59672/0 (socket says 172.21.15.84:59672) 2023-10-11T06:11:05.780 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.775+0000 7fa1bed9d640 1 -- 172.21.15.84:0/2213381939 learned_addr learned my addr 172.21.15.84:0/2213381939 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:05.781 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.776+0000 7fa1b7fff640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006c1b0 0x7fa1c0158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:05.781 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.776+0000 7fa1bf59e640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 0x7fa1c00c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:05.781 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.776+0000 7fa1bed9d640 1 -- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006c1b0 msgr2=0x7fa1c0158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:05.781 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.776+0000 7fa1bed9d640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006c1b0 0x7fa1c0158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:05.781 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.776+0000 7fa1bed9d640 1 -- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 msgr2=0x7fa1c00c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:05.781 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.776+0000 7fa1bed9d640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 0x7fa1c00c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:05.782 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.776+0000 7fa1bed9d640 1 -- 172.21.15.84:0/2213381939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0009400 con 0x7fa1c006bd30 2023-10-11T06:11:05.782 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.777+0000 7fa1bed9d640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1c006bd30 0x7fa1c014baf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1b002b980 tx=0x7fa1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:05.820 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.818+0000 7fa1b77fe640 1 -- 172.21.15.84:0/2213381939 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b003e070 con 0x7fa1c006bd30 2023-10-11T06:11:05.820 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.818+0000 7fa1b77fe640 1 -- 172.21.15.84:0/2213381939 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0039a10 con 0x7fa1c006bd30 2023-10-11T06:11:05.821 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.818+0000 7fa1b77fe640 1 -- 172.21.15.84:0/2213381939 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0039dc0 con 0x7fa1c006bd30 2023-10-11T06:11:05.821 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.818+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c014a580 con 0x7fa1c006bd30 2023-10-11T06:11:05.821 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.818+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c014ad30 con 0x7fa1c006bd30 2023-10-11T06:11:05.821 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.819+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c004e040 con 0x7fa1c006bd30 2023-10-11T06:11:05.822 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.820+0000 7fa1b77fe640 1 -- 172.21.15.84:0/2213381939 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 88859+0+0 (secure 0 0 0) 0x7fa1b00423f0 con 0x7fa1c006bd30 2023-10-11T06:11:05.823 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.821+0000 7fa1b77fe640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1a006a7a0 0x7fa1a006cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:05.824 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.822+0000 7fa1b77fe640 1 -- 172.21.15.84:0/2213381939 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa1b00aa670 con 0x7fa1c006bd30 2023-10-11T06:11:05.824 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.822+0000 7fa1b7fff640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1a006a7a0 0x7fa1a006cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:05.824 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.822+0000 7fa1b7fff640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1a006a7a0 0x7fa1a006cc50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00ad3b0 tx=0x7fa1a8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:05.830 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:05.829+0000 7fa1b77fe640 1 -- 172.21.15.84:0/2213381939 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa1b0075f30 con 0x7fa1c006bd30 2023-10-11T06:11:06.020 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.018+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1c014a580 con 0x7fa1c006bd30 2023-10-11T06:11:06.022 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.020+0000 7fa1b77fe640 1 -- 172.21.15.84:0/2213381939 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12334 (secure 0 0 0) 0x7fa1c014a580 con 0x7fa1c006bd30 2023-10-11T06:11:06.023 INFO:teuthology.misc.health.smithi084.stdout: 2023-10-11T06:11:06.023 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":17,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:03.664041+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:06.025 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1a006a7a0 msgr2=0x7fa1a006cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:06.025 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1a006a7a0 0x7fa1a006cc50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00ad3b0 tx=0x7fa1a8009190 comp rx=0 tx=0).stop 2023-10-11T06:11:06.026 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1c006bd30 msgr2=0x7fa1c014baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:06.026 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1c006bd30 0x7fa1c014baf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1b002b980 tx=0x7fa1b00046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.026 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 shutdown_connections 2023-10-11T06:11:06.026 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1a006a7a0 0x7fa1a006cc50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.026 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1c01590c0 0x7fa1c00c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.027 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1c006c1b0 0x7fa1c0158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.027 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 --2- 172.21.15.84:0/2213381939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1c006bd30 0x7fa1c014baf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.027 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 >> 172.21.15.84:0/2213381939 conn(0x7fa1c00cb270 msgr2=0x7fa1c006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:06.027 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 shutdown_connections 2023-10-11T06:11:06.028 INFO:teuthology.misc.health.smithi084.stderr:2023-10-11T06:11:06.023+0000 7fa1c5004640 1 -- 172.21.15.84:0/2213381939 wait complete. 2023-10-11T06:11:06.039 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-11T06:11:06.040 INFO:tasks.ceph:Creating RBD pool 2023-10-11T06:11:06.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-11T06:11:06.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.242+0000 7f2f61a30640 1 Processor -- start 2023-10-11T06:11:06.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.242+0000 7f2f61a30640 1 -- start start 2023-10-11T06:11:06.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.243+0000 7f2f61a30640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 0x7f2f5c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.243+0000 7f2f61a30640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5c071380 con 0x7f2f5c15a0b0 2023-10-11T06:11:06.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.243+0000 7f2f61a30640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5c06de10 con 0x7f2f5c0c8d00 2023-10-11T06:11:06.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.243+0000 7f2f61a30640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c06e0f0 con 0x7f2f5c0ced30 2023-10-11T06:11:06.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f5affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2f5c15a0b0 0x7f2f5c0ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38474/0 (socket says 172.21.15.84:38474) 2023-10-11T06:11:06.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f5affd640 1 -- 172.21.15.84:0/3358366971 learned_addr learned my addr 172.21.15.84:0/3358366971 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338682769 0 0) 0x7f2f5c071380 con 0x7f2f5c15a0b0 2023-10-11T06:11:06.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f44003520 con 0x7f2f5c15a0b0 2023-10-11T06:11:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f5a7fc640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 0x7f2f5c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747099076 0 0) 0x7f2f44003520 con 0x7f2f5c15a0b0 2023-10-11T06:11:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 msgr2=0x7f2f5c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:06.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 0x7f2f5c0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 >> v1:172.21.15.99:6789/0 conn(0x7f2f5c0c8d00 legacy=0x7f2f5c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c070dc0 con 0x7f2f5c15a0b0 2023-10-11T06:11:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.244+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f2f4c001670 con 0x7f2f5c15a0b0 2023-10-11T06:11:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 >> v1:172.21.15.84:6789/0 conn(0x7f2f5c15a0b0 legacy=0x7f2f5c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:06.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f59ffb640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440039a0 0x7f2f44003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f59ffb640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 0x7f2f440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f59ffb640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44004d50 0x7f2f440051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c06e0f0 con 0x7f2f440039a0 2023-10-11T06:11:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4400bbe0 con 0x7f2f44004d50 2023-10-11T06:11:06.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4400bda0 con 0x7f2f5c0ced30 2023-10-11T06:11:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5affd640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440039a0 0x7f2f44003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5a7fc640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 0x7f2f440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5affd640 1 -- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 msgr2=0x7f2f440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5affd640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0ced30 0x7f2f440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5b7fe640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44004d50 0x7f2f440051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5affd640 1 -- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44004d50 msgr2=0x7f2f440051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5affd640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44004d50 0x7f2f440051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.245+0000 7f2f5affd640 1 -- 172.21.15.84:0/3358366971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44003520 con 0x7f2f440039a0 2023-10-11T06:11:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.246+0000 7f2f5affd640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440039a0 0x7f2f44003db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c210 tx=0x7f2f440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82102153d9b9ad97 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c005ad0 con 0x7f2f440039a0 2023-10-11T06:11:06.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c005c30 con 0x7f2f440039a0 2023-10-11T06:11:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f59ffb640 1 -- 172.21.15.84:0/3358366971 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c02c890 con 0x7f2f440039a0 2023-10-11T06:11:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f61a30640 1 -- 172.21.15.84:0/3358366971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440039a0 msgr2=0x7f2f44003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f61a30640 1 --2- 172.21.15.84:0/3358366971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440039a0 0x7f2f44003db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c210 tx=0x7f2f440057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f5b7fe640 1 -- 172.21.15.84:0/3358366971 reap_dead start 2023-10-11T06:11:06.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f61a30640 1 -- 172.21.15.84:0/3358366971 shutdown_connections 2023-10-11T06:11:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f61a30640 1 -- 172.21.15.84:0/3358366971 >> 172.21.15.84:0/3358366971 conn(0x7f2f5c069c70 msgr2=0x7f2f5c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f61a30640 1 -- 172.21.15.84:0/3358366971 shutdown_connections 2023-10-11T06:11:06.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.247+0000 7f2f61a30640 1 -- 172.21.15.84:0/3358366971 wait complete. 2023-10-11T06:11:06.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.249+0000 7f2f61a30640 1 Processor -- start 2023-10-11T06:11:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.249+0000 7f2f61a30640 1 -- start start 2023-10-11T06:11:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f61a30640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f440039a0 0x7f2f5c14d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f61a30640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0c5780 0x7f2f5c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f61a30640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f5c0c6190 0x7f2f5c0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f61a30640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0c6b70 con 0x7f2f5c0c6190 2023-10-11T06:11:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f61a30640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0cca70 con 0x7f2f440039a0 2023-10-11T06:11:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f61a30640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c14e3d0 con 0x7f2f5c0c5780 2023-10-11T06:11:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f5b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f5c0c6190 0x7f2f5c0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f5b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f5c0c6190 0x7f2f5c0c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59686/0 (socket says 172.21.15.84:59686) 2023-10-11T06:11:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.250+0000 7f2f5b7fe640 1 -- 172.21.15.84:0/1370053511 learned_addr learned my addr 172.21.15.84:0/1370053511 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f5affd640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f440039a0 0x7f2f5c14d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f5b7fe640 1 -- 172.21.15.84:0/1370053511 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0c5780 msgr2=0x7f2f5c0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f5b7fe640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0c5780 0x7f2f5c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f5b7fe640 1 -- 172.21.15.84:0/1370053511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f440039a0 msgr2=0x7f2f5c14d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f5b7fe640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f440039a0 0x7f2f5c14d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f5b7fe640 1 -- 172.21.15.84:0/1370053511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0c5c10 con 0x7f2f5c0c6190 2023-10-11T06:11:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f5b7fe640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f5c0c6190 0x7f2f5c0c6630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0cf6c0 tx=0x7f2f5400d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f43fff640 1 -- 172.21.15.84:0/1370053511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400b070 con 0x7f2f5c0c6190 2023-10-11T06:11:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f43fff640 1 -- 172.21.15.84:0/1370053511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f54003970 con 0x7f2f5c0c6190 2023-10-11T06:11:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f43fff640 1 -- 172.21.15.84:0/1370053511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54003d20 con 0x7f2f5c0c6190 2023-10-11T06:11:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.251+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c0cc420 con 0x7f2f5c0c6190 2023-10-11T06:11:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.252+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c14dec0 con 0x7f2f5c0c6190 2023-10-11T06:11:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.253+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5c0cae90 con 0x7f2f5c0c6190 2023-10-11T06:11:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.254+0000 7f2f43fff640 1 -- 172.21.15.84:0/1370053511 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2f54005070 con 0x7f2f5c0c6190 2023-10-11T06:11:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.254+0000 7f2f43fff640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f4806a860 0x7f2f4806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:06.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.255+0000 7f2f43fff640 1 -- 172.21.15.84:0/1370053511 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2f54087d50 con 0x7f2f5c0c6190 2023-10-11T06:11:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.255+0000 7f2f5affd640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f4806a860 0x7f2f4806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.255+0000 7f2f5affd640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f4806a860 0x7f2f4806cd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c034000 tx=0x7f2f4c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:06.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.259+0000 7f2f43fff640 1 -- 172.21.15.84:0/1370053511 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2f5408b030 con 0x7f2f5c0c6190 2023-10-11T06:11:06.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:06.454+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f2f5c0cc280 con 0x7f2f5c0c6190 2023-10-11T06:11:07.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.110+0000 7f2f43fff640 1 -- 172.21.15.84:0/1370053511 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f2f5c0cc280 con 0x7f2f5c0c6190 2023-10-11T06:11:07.113 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'rbd' created 2023-10-11T06:11:07.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.114+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f4806a860 msgr2=0x7f2f4806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:07.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.114+0000 7f2f61a30640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f4806a860 0x7f2f4806cd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c034000 tx=0x7f2f4c009030 comp rx=0 tx=0).stop 2023-10-11T06:11:07.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.114+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f5c0c6190 msgr2=0x7f2f5c0c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:07.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.114+0000 7f2f61a30640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f5c0c6190 0x7f2f5c0c6630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0cf6c0 tx=0x7f2f5400d830 comp rx=0 tx=0).stop 2023-10-11T06:11:07.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 shutdown_connections 2023-10-11T06:11:07.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f4806a860 0x7f2f4806cd10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f5c0c6190 0x7f2f5c0c6630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f5c0c5780 0x7f2f5c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 --2- 172.21.15.84:0/1370053511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f440039a0 0x7f2f5c14d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 >> 172.21.15.84:0/1370053511 conn(0x7f2f5c069c70 msgr2=0x7f2f5c0c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:07.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 shutdown_connections 2023-10-11T06:11:07.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.115+0000 7f2f61a30640 1 -- 172.21.15.84:0/1370053511 wait complete. 2023-10-11T06:11:07.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-11T06:11:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.344+0000 7f563a72e640 1 Processor -- start 2023-10-11T06:11:07.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.345+0000 7f563a72e640 1 -- start start 2023-10-11T06:11:07.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.346+0000 7f563a72e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f56340cb5a0 0x7f56340c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.346+0000 7f563a72e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340c9240 con 0x7f563415a0b0 2023-10-11T06:11:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.346+0000 7f563a72e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5634071380 con 0x7f56340ced30 2023-10-11T06:11:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.346+0000 7f563a72e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563406e0f0 con 0x7f56340cb5a0 2023-10-11T06:11:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f5633fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f563415a0b0 0x7f56340ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38480/0 (socket says 172.21.15.84:38480) 2023-10-11T06:11:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f5633fff640 1 -- 172.21.15.84:0/1596615700 learned_addr learned my addr 172.21.15.84:0/1596615700 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:07.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853379381 0 0) 0x7f5624001110 con 0x7f563415a0b0 2023-10-11T06:11:07.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f5638ccb640 1 --2- 172.21.15.84:0/1596615700 >> v2:172.21.15.99:3300/0 conn(0x7f56340cb5a0 0x7f56340c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5620003520 con 0x7f563415a0b0 2023-10-11T06:11:07.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591766405 0 0) 0x7f5620003520 con 0x7f563415a0b0 2023-10-11T06:11:07.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 >> v2:172.21.15.99:3300/0 conn(0x7f56340cb5a0 msgr2=0x7f56340c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:07.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 --2- 172.21.15.84:0/1596615700 >> v2:172.21.15.99:3300/0 conn(0x7f56340cb5a0 0x7f56340c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 >> v1:172.21.15.99:6789/0 conn(0x7f56340ced30 legacy=0x7f563406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:07.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563406de10 con 0x7f563415a0b0 2023-10-11T06:11:07.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.347+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f5624001800 con 0x7f563415a0b0 2023-10-11T06:11:07.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 >> v1:172.21.15.84:6789/0 conn(0x7f563415a0b0 legacy=0x7f56340ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:07.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f56337fe640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56200039c0 0x7f5620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f56337fe640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56200047c0 0x7f5620004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f56337fe640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620005240 0x7f562000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563406e0f0 con 0x7f5620005240 2023-10-11T06:11:07.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562000c170 con 0x7f56200039c0 2023-10-11T06:11:07.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562000c330 con 0x7f56200047c0 2023-10-11T06:11:07.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f5638ccb640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620005240 0x7f562000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f5633fff640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56200039c0 0x7f5620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f562bfff640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56200047c0 0x7f5620004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.348+0000 7f5638ccb640 1 -- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56200047c0 msgr2=0x7f5620004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:07.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.349+0000 7f5638ccb640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56200047c0 0x7f5620004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.349+0000 7f5638ccb640 1 -- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56200039c0 msgr2=0x7f5620003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:07.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.349+0000 7f5638ccb640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56200039c0 0x7f5620003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.349+0000 7f5638ccb640 1 -- 172.21.15.84:0/1596615700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620003520 con 0x7f5620005240 2023-10-11T06:11:07.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.349+0000 7f5633fff640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56200039c0 0x7f5620003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:11:07.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.350+0000 7f5638ccb640 1 -- 172.21.15.84:0/1596615700 reap_dead start 2023-10-11T06:11:07.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.350+0000 7f5638ccb640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620005240 0x7f562000b980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f562c009fc0 tx=0x7f56200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78af2c72866334a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:07.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.350+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c02cc60 con 0x7f5620005240 2023-10-11T06:11:07.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.350+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c00f040 con 0x7f5620005240 2023-10-11T06:11:07.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.350+0000 7f56337fe640 1 -- 172.21.15.84:0/1596615700 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c003990 con 0x7f5620005240 2023-10-11T06:11:07.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.351+0000 7f563a72e640 1 -- 172.21.15.84:0/1596615700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620005240 msgr2=0x7f562000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:07.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.351+0000 7f563a72e640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620005240 0x7f562000b980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f562c009fc0 tx=0x7f56200057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.351+0000 7f563a72e640 1 -- 172.21.15.84:0/1596615700 shutdown_connections 2023-10-11T06:11:07.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.351+0000 7f563a72e640 1 --2- 172.21.15.84:0/1596615700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620005240 0x7f562000b980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.351+0000 7f563a72e640 1 -- 172.21.15.84:0/1596615700 >> 172.21.15.84:0/1596615700 conn(0x7f5634069c70 msgr2=0x7f563406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:07.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.351+0000 7f563a72e640 1 -- 172.21.15.84:0/1596615700 shutdown_connections 2023-10-11T06:11:07.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.351+0000 7f563a72e640 1 -- 172.21.15.84:0/1596615700 wait complete. 2023-10-11T06:11:07.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.353+0000 7f563a72e640 1 Processor -- start 2023-10-11T06:11:07.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.353+0000 7f563a72e640 1 -- start start 2023-10-11T06:11:07.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f563a72e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5620005240 0x7f5634147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f563a72e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56340c8150 0x7f56341470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f563a72e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563414a920 0x7f563414ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f563a72e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634149b60 con 0x7f56340c8150 2023-10-11T06:11:07.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f563a72e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634149e40 con 0x7f563414a920 2023-10-11T06:11:07.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f563a72e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563414a120 con 0x7f5620005240 2023-10-11T06:11:07.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f562bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56340c8150 0x7f56341470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f562bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56340c8150 0x7f56341470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59696/0 (socket says 172.21.15.84:59696) 2023-10-11T06:11:07.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.354+0000 7f562bfff640 1 -- 172.21.15.84:0/2361362374 learned_addr learned my addr 172.21.15.84:0/2361362374 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:07.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f5638ccb640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563414a920 0x7f563414ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f5633fff640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5620005240 0x7f5634147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f562bfff640 1 -- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5620005240 msgr2=0x7f5634147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:07.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f562bfff640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5620005240 0x7f5634147dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f562bfff640 1 -- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563414a920 msgr2=0x7f563414ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:07.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f562bfff640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563414a920 0x7f563414ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:07.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f562bfff640 1 -- 172.21.15.84:0/2361362374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563414a400 con 0x7f56340c8150 2023-10-11T06:11:07.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f5633fff640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5620005240 0x7f5634147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:07.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f562bfff640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56340c8150 0x7f56341470d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f561c00e440 tx=0x7f561c00e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:07.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f56317fa640 1 -- 172.21.15.84:0/2361362374 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0052e0 con 0x7f56340c8150 2023-10-11T06:11:07.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f56317fa640 1 -- 172.21.15.84:0/2361362374 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c005440 con 0x7f56340c8150 2023-10-11T06:11:07.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56340c85c0 con 0x7f56340c8150 2023-10-11T06:11:07.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f56317fa640 1 -- 172.21.15.84:0/2361362374 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c003810 con 0x7f56340c8150 2023-10-11T06:11:07.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.355+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56341497a0 con 0x7f56340c8150 2023-10-11T06:11:07.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.356+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5634073ed0 con 0x7f56340c8150 2023-10-11T06:11:07.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.357+0000 7f56317fa640 1 -- 172.21.15.84:0/2361362374 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f561c003970 con 0x7f56340c8150 2023-10-11T06:11:07.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.358+0000 7f56317fa640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:07.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.358+0000 7f56317fa640 1 -- 172.21.15.84:0/2361362374 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f561c088460 con 0x7f56340c8150 2023-10-11T06:11:07.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.359+0000 7f5633fff640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:07.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.359+0000 7f5633fff640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5624001390 tx=0x7f5624001300 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:07.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.363+0000 7f56317fa640 1 -- 172.21.15.84:0/2361362374 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f561c053a50 con 0x7f56340c8150 2023-10-11T06:11:07.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:07.554+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5634149ca0 con 0x7f56340c8150 2023-10-11T06:11:08.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.115+0000 7f56317fa640 1 -- 172.21.15.84:0/2361362374 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5634149ca0 con 0x7f56340c8150 2023-10-11T06:11:08.118 INFO:teuthology.orchestra.run.smithi084.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-11T06:11:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.119+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 msgr2=0x7f561406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.119+0000 7f563a72e640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5624001390 tx=0x7f5624001300 comp rx=0 tx=0).stop 2023-10-11T06:11:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.119+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56340c8150 msgr2=0x7f56341470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.119+0000 7f563a72e640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56340c8150 0x7f56341470d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f561c00e440 tx=0x7f561c00e900 comp rx=0 tx=0).stop 2023-10-11T06:11:08.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.121+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 shutdown_connections 2023-10-11T06:11:08.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.121+0000 7f563a72e640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.121+0000 7f563a72e640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563414a920 0x7f563414ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.121+0000 7f563a72e640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56340c8150 0x7f56341470d0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f561c00e440 tx=0x7f561c00e900 comp rx=0 tx=0).stop 2023-10-11T06:11:08.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.121+0000 7f563a72e640 1 --2- 172.21.15.84:0/2361362374 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5620005240 0x7f5634147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.121+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 >> 172.21.15.84:0/2361362374 conn(0x7f5634069c70 msgr2=0x7f56340c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:08.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.122+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 shutdown_connections 2023-10-11T06:11:08.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.122+0000 7f563a72e640 1 -- 172.21.15.84:0/2361362374 wait complete. 2023-10-11T06:11:08.146 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-11T06:11:08.147 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T06:11:08.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T06:11:08.153 INFO:tasks.ceph.mds.a:Started 2023-10-11T06:11:08.153 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T06:11:08.153 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T06:11:08.158 INFO:tasks.ceph.mds.c:Started 2023-10-11T06:11:08.158 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T06:11:08.159 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T06:11:08.168 INFO:tasks.ceph.mds.b:Started 2023-10-11T06:11:08.168 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T06:11:08.168 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T06:11:08.176 INFO:tasks.ceph.mds.d:Started 2023-10-11T06:11:08.177 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-11T06:11:08.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:08.307 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-11T06:11:08.308 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-11T06:11:08.333 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2023-10-11T06:11:08.333 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2023-10-11T06:11:08.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.364+0000 7fbeae6db640 1 Processor -- start 2023-10-11T06:11:08.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.364+0000 7fbeae6db640 1 -- start start 2023-10-11T06:11:08.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.365+0000 7fbeae6db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbea80af4d0 0x7fbea80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.365+0000 7fbeae6db640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea80af930 con 0x7fbea80d0c20 2023-10-11T06:11:08.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.365+0000 7fbeae6db640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea806cd10 con 0x7fbea815c290 2023-10-11T06:11:08.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.365+0000 7fbeae6db640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea80afb60 con 0x7fbea80af4d0 2023-10-11T06:11:08.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.365+0000 7fbeacc78640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbea80af4d0 0x7fbea80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.365+0000 7fbe9ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbea80d0c20 0x7fbea80b0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38498/0 (socket says 172.21.15.84:38498) 2023-10-11T06:11:08.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.365+0000 7fbe9ffff640 1 -- 172.21.15.84:0/4264712734 learned_addr learned my addr 172.21.15.84:0/4264712734 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:08.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.366+0000 7fbea77fe640 1 -- 172.21.15.84:0/4264712734 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172632718 0 0) 0x7fbea80af930 con 0x7fbea80d0c20 2023-10-11T06:11:08.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.366+0000 7fbeacc78640 1 -- 172.21.15.84:0/4264712734 >> v1:172.21.15.99:6789/0 conn(0x7fbea815c290 legacy=0x7fbea80d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.366+0000 7fbeacc78640 1 -- 172.21.15.84:0/4264712734 >> v1:172.21.15.84:6789/0 conn(0x7fbea80d0c20 legacy=0x7fbea80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.366+0000 7fbeacc78640 1 -- 172.21.15.84:0/4264712734 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80afe40 con 0x7fbea80af4d0 2023-10-11T06:11:08.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.367+0000 7fbeacc78640 1 --2- 172.21.15.84:0/4264712734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80af4d0 0x7fbea80d25e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009d90 tx=0x7fbea002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ada8548958b193 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:08.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.367+0000 7fbea77fe640 1 -- 172.21.15.84:0/4264712734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea002cc60 con 0x7fbea80af4d0 2023-10-11T06:11:08.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.367+0000 7fbea77fe640 1 -- 172.21.15.84:0/4264712734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea002cdc0 con 0x7fbea80af4d0 2023-10-11T06:11:08.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.367+0000 7fbea77fe640 1 -- 172.21.15.84:0/4264712734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0034660 con 0x7fbea80af4d0 2023-10-11T06:11:08.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.368+0000 7fbeae6db640 1 -- 172.21.15.84:0/4264712734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80af4d0 msgr2=0x7fbea80d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.368+0000 7fbeae6db640 1 --2- 172.21.15.84:0/4264712734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80af4d0 0x7fbea80d25e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009d90 tx=0x7fbea002beb0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.368+0000 7fbeae6db640 1 -- 172.21.15.84:0/4264712734 shutdown_connections 2023-10-11T06:11:08.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.368+0000 7fbeae6db640 1 --2- 172.21.15.84:0/4264712734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80af4d0 0x7fbea80d25e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.368+0000 7fbeae6db640 1 -- 172.21.15.84:0/4264712734 >> 172.21.15.84:0/4264712734 conn(0x7fbea8071cc0 msgr2=0x7fbea809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:08.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.368+0000 7fbeae6db640 1 -- 172.21.15.84:0/4264712734 shutdown_connections 2023-10-11T06:11:08.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.368+0000 7fbeae6db640 1 -- 172.21.15.84:0/4264712734 wait complete. 2023-10-11T06:11:08.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.370+0000 7fbeae6db640 1 Processor -- start 2023-10-11T06:11:08.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.370+0000 7fbeae6db640 1 -- start start 2023-10-11T06:11:08.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbeae6db640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbea80af4d0 0x7fbea81f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbeae6db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80d0c20 0x7fbea809f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbeae6db640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea815c290 0x7fbea80c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbeae6db640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea809fb80 con 0x7fbea80d0c20 2023-10-11T06:11:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbeae6db640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea809fe60 con 0x7fbea815c290 2023-10-11T06:11:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbeae6db640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80a0140 con 0x7fbea80af4d0 2023-10-11T06:11:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80d0c20 0x7fbea809f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80d0c20 0x7fbea809f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59748/0 (socket says 172.21.15.84:59748) 2023-10-11T06:11:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbe9ffff640 1 -- 172.21.15.84:0/2478397535 learned_addr learned my addr 172.21.15.84:0/2478397535 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbeacc78640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea815c290 0x7fbea80c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbea7fff640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbea80af4d0 0x7fbea81f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbe9ffff640 1 -- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbea80af4d0 msgr2=0x7fbea81f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbe9ffff640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbea80af4d0 0x7fbea81f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbe9ffff640 1 -- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea815c290 msgr2=0x7fbea80c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.371+0000 7fbe9ffff640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea815c290 0x7fbea80c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbe9ffff640 1 -- 172.21.15.84:0/2478397535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea0009a70 con 0x7fbea80d0c20 2023-10-11T06:11:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbeacc78640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea815c290 0x7fbea80c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbe9ffff640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80d0c20 0x7fbea809f0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000c4a0 tx=0x7fbe9000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:08.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbea57fa640 1 -- 172.21.15.84:0/2478397535 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900074e0 con 0x7fbea80d0c20 2023-10-11T06:11:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbea57fa640 1 -- 172.21.15.84:0/2478397535 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe90007640 con 0x7fbea80d0c20 2023-10-11T06:11:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbea57fa640 1 -- 172.21.15.84:0/2478397535 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90004a00 con 0x7fbea80d0c20 2023-10-11T06:11:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea80a0420 con 0x7fbea80d0c20 2023-10-11T06:11:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.372+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea81f4640 con 0x7fbea80d0c20 2023-10-11T06:11:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.373+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea80b7130 con 0x7fbea80d0c20 2023-10-11T06:11:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.374+0000 7fbea57fa640 1 -- 172.21.15.84:0/2478397535 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbe90004b60 con 0x7fbea80d0c20 2023-10-11T06:11:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.375+0000 7fbea57fa640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe8806a830 0x7fbe8806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.375+0000 7fbea57fa640 1 -- 172.21.15.84:0/2478397535 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbe90087d70 con 0x7fbea80d0c20 2023-10-11T06:11:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.376+0000 7fbea7fff640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe8806a830 0x7fbe8806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.376+0000 7fbea7fff640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe8806a830 0x7fbe8806cce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe98000f60 tx=0x7fbe98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.380+0000 7fbea57fa640 1 -- 172.21.15.84:0/2478397535 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fbea80b7130 con 0x7fbea80d0c20 2023-10-11T06:11:08.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.559+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbea80b7130 con 0x7fbea80d0c20 2023-10-11T06:11:08.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.561+0000 7fbea57fa640 1 -- 172.21.15.84:0/2478397535 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14133 (secure 0 0 0) 0x7fbea80b7130 con 0x7fbea80d0c20 2023-10-11T06:11:08.564 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:08.564 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":19,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:08.107655+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:08.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe8806a830 msgr2=0x7fbe8806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe8806a830 0x7fbe8806cce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe98000f60 tx=0x7fbe98009040 comp rx=0 tx=0).stop 2023-10-11T06:11:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80d0c20 msgr2=0x7fbea809f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80d0c20 0x7fbea809f0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000c4a0 tx=0x7fbe9000c960 comp rx=0 tx=0).stop 2023-10-11T06:11:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 shutdown_connections 2023-10-11T06:11:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe8806a830 0x7fbe8806cce0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbea815c290 0x7fbea80c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbea80d0c20 0x7fbea809f0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 --2- 172.21.15.84:0/2478397535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbea80af4d0 0x7fbea81f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 >> 172.21.15.84:0/2478397535 conn(0x7fbea8071cc0 msgr2=0x7fbea809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:08.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.564+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 shutdown_connections 2023-10-11T06:11:08.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.565+0000 7fbeae6db640 1 -- 172.21.15.84:0/2478397535 wait complete. 2023-10-11T06:11:08.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:08.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:08.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.787+0000 7f084d88a640 1 Processor -- start 2023-10-11T06:11:08.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.788+0000 7f084d88a640 1 -- start start 2023-10-11T06:11:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.789+0000 7f084d88a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08480d0b90 0x7f08480d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.789+0000 7f084d88a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f084806bc40 0x7f08480cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.789+0000 7f084d88a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08480ccf00 con 0x7f08480b1d50 2023-10-11T06:11:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.789+0000 7f084d88a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08480b2120 con 0x7f084806bc40 2023-10-11T06:11:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.789+0000 7f084d88a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480d2e90 con 0x7f08480d0b90 2023-10-11T06:11:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f084d089640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f084806bc40 0x7f08480cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f084d089640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f084806bc40 0x7f08480cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59758/0 (socket says 172.21.15.84:59758) 2023-10-11T06:11:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f084d089640 1 -- 172.21.15.84:0/732358531 learned_addr learned my addr 172.21.15.84:0/732358531 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:08.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f0847fff640 1 --2- 172.21.15.84:0/732358531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08480d0b90 0x7f08480d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f084d089640 1 -- 172.21.15.84:0/732358531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08480d0b90 msgr2=0x7f08480d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:08.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f084d089640 1 --2- 172.21.15.84:0/732358531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08480d0b90 0x7f08480d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f084d089640 1 -- 172.21.15.84:0/732358531 >> v1:172.21.15.99:6789/0 conn(0x7f08480b1d50 legacy=0x7f08480d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:08.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.790+0000 7f084d089640 1 -- 172.21.15.84:0/732358531 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480d3170 con 0x7f084806bc40 2023-10-11T06:11:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.791+0000 7f084d089640 1 --2- 172.21.15.84:0/732358531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084806bc40 0x7f08480cd950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0840009870 tx=0x7f084002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5832c68bfa486989 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.791+0000 7f08477fe640 1 -- 172.21.15.84:0/732358531 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840030840 con 0x7f084806bc40 2023-10-11T06:11:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f08477fe640 1 -- 172.21.15.84:0/732358531 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08400309a0 con 0x7f084806bc40 2023-10-11T06:11:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f08477fe640 1 -- 172.21.15.84:0/732358531 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840030d30 con 0x7f084806bc40 2023-10-11T06:11:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 -- 172.21.15.84:0/732358531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084806bc40 msgr2=0x7f08480cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 --2- 172.21.15.84:0/732358531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084806bc40 0x7f08480cd950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0840009870 tx=0x7f084002b980 comp rx=0 tx=0).stop 2023-10-11T06:11:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 -- 172.21.15.84:0/732358531 shutdown_connections 2023-10-11T06:11:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 --2- 172.21.15.84:0/732358531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084806bc40 0x7f08480cd950 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 --2- 172.21.15.84:0/732358531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08480d0b90 0x7f08480d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 -- 172.21.15.84:0/732358531 >> 172.21.15.84:0/732358531 conn(0x7f0848089ff0 msgr2=0x7f084809bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 -- 172.21.15.84:0/732358531 shutdown_connections 2023-10-11T06:11:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.792+0000 7f084d88a640 1 -- 172.21.15.84:0/732358531 wait complete. 2023-10-11T06:11:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.794+0000 7f084d88a640 1 Processor -- start 2023-10-11T06:11:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.795+0000 7f084d88a640 1 -- start start 2023-10-11T06:11:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d88a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f084806bc40 0x7f08480b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d88a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08480b1d50 0x7f0848159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d88a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08480d0b90 0x7f084815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d88a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480afec0 con 0x7f08480d0b90 2023-10-11T06:11:08.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d88a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084815e830 con 0x7f08480b1d50 2023-10-11T06:11:08.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d88a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480b0670 con 0x7f084806bc40 2023-10-11T06:11:08.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d089640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08480d0b90 0x7f084815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d089640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08480d0b90 0x7f084815c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59774/0 (socket says 172.21.15.84:59774) 2023-10-11T06:11:08.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.796+0000 7f084d089640 1 -- 172.21.15.84:0/2673411625 learned_addr learned my addr 172.21.15.84:0/2673411625 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:08.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084c888640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f084806bc40 0x7f08480b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f0847fff640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08480b1d50 0x7f0848159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d089640 1 -- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f084806bc40 msgr2=0x7f08480b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d089640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f084806bc40 0x7f08480b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d089640 1 -- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08480b1d50 msgr2=0x7f0848159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d089640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08480b1d50 0x7f0848159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d089640 1 -- 172.21.15.84:0/2673411625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400094d0 con 0x7f08480d0b90 2023-10-11T06:11:08.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f0847fff640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08480b1d50 0x7f0848159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:08.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d089640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08480d0b90 0x7f084815c720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f084002b9c0 tx=0x7f0840004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:08.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f08457fa640 1 -- 172.21.15.84:0/2673411625 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840039c90 con 0x7f08480d0b90 2023-10-11T06:11:08.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f08457fa640 1 -- 172.21.15.84:0/2673411625 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0840039df0 con 0x7f08480d0b90 2023-10-11T06:11:08.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f08457fa640 1 -- 172.21.15.84:0/2673411625 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840043640 con 0x7f08480d0b90 2023-10-11T06:11:08.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08480b00b0 con 0x7f08480d0b90 2023-10-11T06:11:08.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.797+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084815e4c0 con 0x7f08480d0b90 2023-10-11T06:11:08.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.799+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084815e320 con 0x7f08480d0b90 2023-10-11T06:11:08.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.799+0000 7f08457fa640 1 -- 172.21.15.84:0/2673411625 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f08400437a0 con 0x7f08480d0b90 2023-10-11T06:11:08.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.800+0000 7f08457fa640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f082806a830 0x7f082806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:08.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.800+0000 7f08457fa640 1 -- 172.21.15.84:0/2673411625 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f084003e070 con 0x7f08480d0b90 2023-10-11T06:11:08.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.801+0000 7f084c888640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f082806a830 0x7f082806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:08.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.801+0000 7f084c888640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f082806a830 0x7f082806cce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0838004470 tx=0x7f0838009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:08.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.804+0000 7f08457fa640 1 -- 172.21.15.84:0/2673411625 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f084815e320 con 0x7f08480d0b90 2023-10-11T06:11:08.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.991+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f084815e320 con 0x7f08480d0b90 2023-10-11T06:11:08.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.992+0000 7f08457fa640 1 -- 172.21.15.84:0/2673411625 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f084815e320 con 0x7f08480d0b90 2023-10-11T06:11:08.995 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f082806a830 msgr2=0x7f082806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f082806a830 0x7f082806cce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0838004470 tx=0x7f0838009040 comp rx=0 tx=0).stop 2023-10-11T06:11:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08480d0b90 msgr2=0x7f084815c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:08.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08480d0b90 0x7f084815c720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f084002b9c0 tx=0x7f0840004730 comp rx=0 tx=0).stop 2023-10-11T06:11:08.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 shutdown_connections 2023-10-11T06:11:08.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f082806a830 0x7f082806cce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08480d0b90 0x7f084815c720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08480b1d50 0x7f0848159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:08.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 --2- 172.21.15.84:0/2673411625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f084806bc40 0x7f08480b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.995+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 >> 172.21.15.84:0/2673411625 conn(0x7f0848089ff0 msgr2=0x7f0848072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:09.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.996+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 shutdown_connections 2023-10-11T06:11:09.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:08.996+0000 7f084d88a640 1 -- 172.21.15.84:0/2673411625 wait complete. 2023-10-11T06:11:09.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.210+0000 7f9f8759e640 1 Processor -- start 2023-10-11T06:11:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.211+0000 7f9f8759e640 1 -- start start 2023-10-11T06:11:09.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.212+0000 7f9f8759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f8815c350 0x7f9f8806a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.212+0000 7f9f8759e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f88160b80 0x7f9f880affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.212+0000 7f9f8759e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f880cc860 con 0x7f9f880b0510 2023-10-11T06:11:09.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.212+0000 7f9f8759e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f880b0990 con 0x7f9f8815c350 2023-10-11T06:11:09.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.212+0000 7f9f8759e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880c8cd0 con 0x7f9f88160b80 2023-10-11T06:11:09.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.212+0000 7f9f8659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f8815c350 0x7f9f8806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.212+0000 7f9f8659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f8815c350 0x7f9f8806a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37084/0 (socket says 172.21.15.84:37084) 2023-10-11T06:11:09.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.213+0000 7f9f8659c640 1 -- 172.21.15.84:0/3881636240 learned_addr learned my addr 172.21.15.84:0/3881636240 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:09.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.213+0000 7f9f85d9b640 1 --2- 172.21.15.84:0/3881636240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f88160b80 0x7f9f880affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.213+0000 7f9f8659c640 1 -- 172.21.15.84:0/3881636240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f88160b80 msgr2=0x7f9f880affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:09.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.213+0000 7f9f8659c640 1 --2- 172.21.15.84:0/3881636240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f88160b80 0x7f9f880affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.213+0000 7f9f8659c640 1 -- 172.21.15.84:0/3881636240 >> v1:172.21.15.99:6789/0 conn(0x7f9f880b0510 legacy=0x7f9f880caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:09.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.213+0000 7f9f8659c640 1 -- 172.21.15.84:0/3881636240 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880c8fb0 con 0x7f9f8815c350 2023-10-11T06:11:09.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.214+0000 7f9f8659c640 1 --2- 172.21.15.84:0/3881636240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f8815c350 0x7f9f8806a550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009870 tx=0x7f9f7802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcbdc622eb7f271 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:09.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.214+0000 7f9f8559a640 1 -- 172.21.15.84:0/3881636240 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78031640 con 0x7f9f8815c350 2023-10-11T06:11:09.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.214+0000 7f9f8559a640 1 -- 172.21.15.84:0/3881636240 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f780317a0 con 0x7f9f8815c350 2023-10-11T06:11:09.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.214+0000 7f9f8559a640 1 -- 172.21.15.84:0/3881636240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78031b30 con 0x7f9f8815c350 2023-10-11T06:11:09.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 -- 172.21.15.84:0/3881636240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f8815c350 msgr2=0x7f9f8806a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3881636240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f8815c350 0x7f9f8806a550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009870 tx=0x7f9f7802b8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 -- 172.21.15.84:0/3881636240 shutdown_connections 2023-10-11T06:11:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3881636240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f88160b80 0x7f9f880affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3881636240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f8815c350 0x7f9f8806a550 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 -- 172.21.15.84:0/3881636240 >> 172.21.15.84:0/3881636240 conn(0x7f9f8806bfb0 msgr2=0x7f9f8806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 -- 172.21.15.84:0/3881636240 shutdown_connections 2023-10-11T06:11:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.215+0000 7f9f8759e640 1 -- 172.21.15.84:0/3881636240 wait complete. 2023-10-11T06:11:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.217+0000 7f9f8759e640 1 Processor -- start 2023-10-11T06:11:09.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.217+0000 7f9f8759e640 1 -- start start 2023-10-11T06:11:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f8759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f880b0510 0x7f9f8815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f8759e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f8815c350 0x7f9f881f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f8759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f88160b80 0x7f9f881f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f8759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f881f5df0 con 0x7f9f88160b80 2023-10-11T06:11:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f8759e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f881f2860 con 0x7f9f880b0510 2023-10-11T06:11:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f8759e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f881f1790 con 0x7f9f8815c350 2023-10-11T06:11:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f86d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f88160b80 0x7f9f881f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f86d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f88160b80 0x7f9f881f1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37094/0 (socket says 172.21.15.84:37094) 2023-10-11T06:11:09.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f86d9d640 1 -- 172.21.15.84:0/3226167607 learned_addr learned my addr 172.21.15.84:0/3226167607 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:09.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.218+0000 7f9f8659c640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f880b0510 0x7f9f8815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f85d9b640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f8815c350 0x7f9f881f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f86d9d640 1 -- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f8815c350 msgr2=0x7f9f881f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f86d9d640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f8815c350 0x7f9f881f0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f86d9d640 1 -- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f880b0510 msgr2=0x7f9f8815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f86d9d640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f880b0510 0x7f9f8815a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f86d9d640 1 -- 172.21.15.84:0/3226167607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f780094d0 con 0x7f9f88160b80 2023-10-11T06:11:09.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f85d9b640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f8815c350 0x7f9f881f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:09.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f86d9d640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f88160b80 0x7f9f881f1250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000b410 tx=0x7f9f8000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:09.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f7effd640 1 -- 172.21.15.84:0/3226167607 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80004020 con 0x7f9f88160b80 2023-10-11T06:11:09.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f7effd640 1 -- 172.21.15.84:0/3226167607 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f80004180 con 0x7f9f88160b80 2023-10-11T06:11:09.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f881f5030 con 0x7f9f88160b80 2023-10-11T06:11:09.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f7effd640 1 -- 172.21.15.84:0/3226167607 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80011630 con 0x7f9f88160b80 2023-10-11T06:11:09.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.219+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f881f55f0 con 0x7f9f88160b80 2023-10-11T06:11:09.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.220+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8804e890 con 0x7f9f88160b80 2023-10-11T06:11:09.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.221+0000 7f9f7effd640 1 -- 172.21.15.84:0/3226167607 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9f80011790 con 0x7f9f88160b80 2023-10-11T06:11:09.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.222+0000 7f9f7effd640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f6806a830 0x7f9f6806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.222+0000 7f9f7effd640 1 -- 172.21.15.84:0/3226167607 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9f80088860 con 0x7f9f88160b80 2023-10-11T06:11:09.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.222+0000 7f9f8659c640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f6806a830 0x7f9f6806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.223+0000 7f9f8659c640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f6806a830 0x7f9f6806cce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f78036000 tx=0x7f9f7803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:09.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.227+0000 7f9f7effd640 1 -- 172.21.15.84:0/3226167607 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9f80053ec0 con 0x7f9f88160b80 2023-10-11T06:11:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.420+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9f8806a330 con 0x7f9f88160b80 2023-10-11T06:11:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.421+0000 7f9f7effd640 1 -- 172.21.15.84:0/3226167607 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9f800150a0 con 0x7f9f88160b80 2023-10-11T06:11:09.423 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f6806a830 msgr2=0x7f9f6806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f6806a830 0x7f9f6806cce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f78036000 tx=0x7f9f7803c040 comp rx=0 tx=0).stop 2023-10-11T06:11:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f88160b80 msgr2=0x7f9f881f1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f88160b80 0x7f9f881f1250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000b410 tx=0x7f9f8000b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 shutdown_connections 2023-10-11T06:11:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f6806a830 0x7f9f6806cce0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f88160b80 0x7f9f881f1250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f8815c350 0x7f9f881f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 --2- 172.21.15.84:0/3226167607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f880b0510 0x7f9f8815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 >> 172.21.15.84:0/3226167607 conn(0x7f9f8806bfb0 msgr2=0x7f9f880718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 shutdown_connections 2023-10-11T06:11:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.424+0000 7f9f8759e640 1 -- 172.21.15.84:0/3226167607 wait complete. 2023-10-11T06:11:09.444 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:11:09.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.651+0000 7f0717567640 1 Processor -- start 2023-10-11T06:11:09.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.652+0000 7f0717567640 1 -- start start 2023-10-11T06:11:09.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.653+0000 7f0717567640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 0x7f0710164f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.653+0000 7f0717567640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0710071a90 con 0x7f07100ca300 2023-10-11T06:11:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.653+0000 7f0717567640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100cb9c0 con 0x7f071015d4f0 2023-10-11T06:11:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.653+0000 7f0717567640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0710072310 con 0x7f07100cb540 2023-10-11T06:11:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0714b02640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f07100ca300 0x7f07100c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40546/0 (socket says 172.21.15.84:40546) 2023-10-11T06:11:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0714b02640 1 -- 172.21.15.84:0/804128784 learned_addr learned my addr 172.21.15.84:0/804128784 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:09.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0715b04640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 0x7f0710164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804426569 0 0) 0x7f0710071a90 con 0x7f07100ca300 2023-10-11T06:11:09.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06fc003520 con 0x7f07100ca300 2023-10-11T06:11:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875844247 0 0) 0x7f06fc003520 con 0x7f07100ca300 2023-10-11T06:11:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 msgr2=0x7f0710164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 0x7f0710164f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 >> v1:172.21.15.99:6789/0 conn(0x7f071015d4f0 legacy=0x7f07100c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710070280 con 0x7f07100ca300 2023-10-11T06:11:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f0704001690 con 0x7f07100ca300 2023-10-11T06:11:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 >> v1:172.21.15.84:6789/0 conn(0x7f07100ca300 legacy=0x7f07100c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.654+0000 7f0703fff640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 0x7f06fc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0703fff640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06fc003f70 0x7f06fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0703fff640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f06fc005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0710072310 con 0x7f06fc004d00 2023-10-11T06:11:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc00bb40 con 0x7f06fc003f70 2023-10-11T06:11:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc00bd00 con 0x7f07100cb540 2023-10-11T06:11:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715b04640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f06fc005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715303640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 0x7f06fc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715b04640 1 -- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 msgr2=0x7f06fc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0714b02640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06fc003f70 0x7f06fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715b04640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 0x7f06fc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715b04640 1 -- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06fc003f70 msgr2=0x7f06fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715b04640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06fc003f70 0x7f06fc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715b04640 1 -- 172.21.15.84:0/804128784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc003520 con 0x7f06fc004d00 2023-10-11T06:11:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.655+0000 7f0715303640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07100cb540 0x7f06fc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.656+0000 7f0715b04640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f06fc005170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0708009fc0 tx=0x7f06fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77052fc10ef1531e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.656+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070802cbf0 con 0x7f06fc004d00 2023-10-11T06:11:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.656+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070800f040 con 0x7f06fc004d00 2023-10-11T06:11:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.656+0000 7f0703fff640 1 -- 172.21.15.84:0/804128784 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708034440 con 0x7f06fc004d00 2023-10-11T06:11:09.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.657+0000 7f0717567640 1 -- 172.21.15.84:0/804128784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 msgr2=0x7f06fc005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.657+0000 7f0717567640 1 --2- 172.21.15.84:0/804128784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f06fc005170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0708009fc0 tx=0x7f06fc0057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.657+0000 7f0715b04640 1 -- 172.21.15.84:0/804128784 reap_dead start 2023-10-11T06:11:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.657+0000 7f0717567640 1 -- 172.21.15.84:0/804128784 shutdown_connections 2023-10-11T06:11:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.657+0000 7f0717567640 1 -- 172.21.15.84:0/804128784 >> 172.21.15.84:0/804128784 conn(0x7f0710069c70 msgr2=0x7f071006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.657+0000 7f0717567640 1 -- 172.21.15.84:0/804128784 shutdown_connections 2023-10-11T06:11:09.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.657+0000 7f0717567640 1 -- 172.21.15.84:0/804128784 wait complete. 2023-10-11T06:11:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.659+0000 7f0717567640 1 Processor -- start 2023-10-11T06:11:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.659+0000 7f0717567640 1 -- start start 2023-10-11T06:11:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0717567640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f07100c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0717567640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07100c92e0 0x7f0710158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0717567640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0710159450 0x7f07100b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0717567640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07100c5c10 con 0x7f06fc004d00 2023-10-11T06:11:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0717567640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0710159a80 con 0x7f07100c92e0 2023-10-11T06:11:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0717567640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0710159d60 con 0x7f0710159450 2023-10-11T06:11:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0715303640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f07100c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0715303640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f07100c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37114/0 (socket says 172.21.15.84:37114) 2023-10-11T06:11:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.660+0000 7f0715303640 1 -- 172.21.15.84:0/2105353906 learned_addr learned my addr 172.21.15.84:0/2105353906 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0715303640 1 -- 172.21.15.84:0/2105353906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0710159450 msgr2=0x7f07100b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0715b04640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0710159450 0x7f07100b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0715303640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0710159450 0x7f07100b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0715303640 1 -- 172.21.15.84:0/2105353906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07100c92e0 msgr2=0x7f0710158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0715303640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07100c92e0 0x7f0710158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0715303640 1 -- 172.21.15.84:0/2105353906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071015ac50 con 0x7f06fc004d00 2023-10-11T06:11:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0715303640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f07100c56d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f070c00d7a0 tx=0x7f070c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0701ffb640 1 -- 172.21.15.84:0/2105353906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c005270 con 0x7f06fc004d00 2023-10-11T06:11:09.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0701ffb640 1 -- 172.21.15.84:0/2105353906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c0053f0 con 0x7f06fc004d00 2023-10-11T06:11:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0701ffb640 1 -- 172.21.15.84:0/2105353906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c006a60 con 0x7f06fc004d00 2023-10-11T06:11:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071015af30 con 0x7f06fc004d00 2023-10-11T06:11:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.661+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07100c9750 con 0x7f06fc004d00 2023-10-11T06:11:09.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.662+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071015aab0 con 0x7f06fc004d00 2023-10-11T06:11:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.663+0000 7f0701ffb640 1 -- 172.21.15.84:0/2105353906 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f070c006bc0 con 0x7f06fc004d00 2023-10-11T06:11:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.664+0000 7f0701ffb640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f406a860 0x7f06f406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.664+0000 7f0701ffb640 1 -- 172.21.15.84:0/2105353906 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f070c088160 con 0x7f06fc004d00 2023-10-11T06:11:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.664+0000 7f0714b02640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f406a860 0x7f06f406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:09.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.665+0000 7f0714b02640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f406a860 0x7f06f406cd10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0704001eb0 tx=0x7f0704009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:09.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.668+0000 7f0701ffb640 1 -- 172.21.15.84:0/2105353906 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f071015aab0 con 0x7f06fc004d00 2023-10-11T06:11:09.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.896+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07100cb9c0 con 0x7f06fc004d00 2023-10-11T06:11:09.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.897+0000 7f0701ffb640 1 -- 172.21.15.84:0/2105353906 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3515 (secure 0 0 0) 0x7f070c019dd0 con 0x7f06fc004d00 2023-10-11T06:11:09.899 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:09.900 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-11T06:11:09.900 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 3 2023-10-11T06:11:09.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f406a860 msgr2=0x7f06f406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f406a860 0x7f06f406cd10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0704001eb0 tx=0x7f0704009040 comp rx=0 tx=0).stop 2023-10-11T06:11:09.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 msgr2=0x7f07100c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:09.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f07100c56d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f070c00d7a0 tx=0x7f070c00dc60 comp rx=0 tx=0).stop 2023-10-11T06:11:09.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 shutdown_connections 2023-10-11T06:11:09.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f406a860 0x7f06f406cd10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0710159450 0x7f07100b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07100c92e0 0x7f0710158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 --2- 172.21.15.84:0/2105353906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc004d00 0x7f07100c56d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:09.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 >> 172.21.15.84:0/2105353906 conn(0x7f0710069c70 msgr2=0x7f07100ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:09.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 shutdown_connections 2023-10-11T06:11:09.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:09.899+0000 7f0717567640 1 -- 172.21.15.84:0/2105353906 wait complete. 2023-10-11T06:11:09.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:10.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.124+0000 7f0f317c5640 1 Processor -- start 2023-10-11T06:11:10.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.124+0000 7f0f317c5640 1 -- start start 2023-10-11T06:11:10.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.125+0000 7f0f317c5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0f2c15d4f0 0x7f0f2c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.125+0000 7f0f317c5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f2c0c9360 0x7f0f2c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.125+0000 7f0f317c5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c071a90 con 0x7f0f2c0ca300 2023-10-11T06:11:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.125+0000 7f0f317c5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c072310 con 0x7f0f2c0c9360 2023-10-11T06:11:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.125+0000 7f0f317c5640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c070280 con 0x7f0f2c15d4f0 2023-10-11T06:11:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f2c0c9360 0x7f0f2c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f2c0c9360 0x7f0f2c0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37122/0 (socket says 172.21.15.84:37122) 2023-10-11T06:11:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2b7fe640 1 -- 172.21.15.84:0/3995263768 learned_addr learned my addr 172.21.15.84:0/3995263768 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:10.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f23fff640 1 --1- 172.21.15.84:0/3995263768 >> v1:172.21.15.84:6789/0 conn(0x7f0f2c0ca300 0x7f0f2c0c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40548/0 (socket says 172.21.15.84:40548) 2023-10-11T06:11:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2affd640 1 --2- 172.21.15.84:0/3995263768 >> v2:172.21.15.99:3300/0 conn(0x7f0f2c15d4f0 0x7f0f2c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2b7fe640 1 -- 172.21.15.84:0/3995263768 >> v2:172.21.15.99:3300/0 conn(0x7f0f2c15d4f0 msgr2=0x7f0f2c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2b7fe640 1 --2- 172.21.15.84:0/3995263768 >> v2:172.21.15.99:3300/0 conn(0x7f0f2c15d4f0 0x7f0f2c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2b7fe640 1 -- 172.21.15.84:0/3995263768 >> v1:172.21.15.84:6789/0 conn(0x7f0f2c0ca300 legacy=0x7f0f2c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.126+0000 7f0f2b7fe640 1 -- 172.21.15.84:0/3995263768 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c071cc0 con 0x7f0f2c0c9360 2023-10-11T06:11:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.127+0000 7f0f2b7fe640 1 --2- 172.21.15.84:0/3995263768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c0ce550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0cf6a0 tx=0x7f0f2402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b088280633d96600 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.127+0000 7f0f2a7fc640 1 -- 172.21.15.84:0/3995263768 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2402cb10 con 0x7f0f2c0c9360 2023-10-11T06:11:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f2a7fc640 1 -- 172.21.15.84:0/3995263768 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2402cc70 con 0x7f0f2c0c9360 2023-10-11T06:11:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f2a7fc640 1 -- 172.21.15.84:0/3995263768 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240343f0 con 0x7f0f2c0c9360 2023-10-11T06:11:10.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 -- 172.21.15.84:0/3995263768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 msgr2=0x7f0f2c0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3995263768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c0ce550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0cf6a0 tx=0x7f0f2402bf70 comp rx=0 tx=0).stop 2023-10-11T06:11:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 -- 172.21.15.84:0/3995263768 shutdown_connections 2023-10-11T06:11:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3995263768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c0ce550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3995263768 >> v2:172.21.15.99:3300/0 conn(0x7f0f2c15d4f0 0x7f0f2c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 -- 172.21.15.84:0/3995263768 >> 172.21.15.84:0/3995263768 conn(0x7f0f2c069c70 msgr2=0x7f0f2c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:10.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 -- 172.21.15.84:0/3995263768 shutdown_connections 2023-10-11T06:11:10.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.128+0000 7f0f317c5640 1 -- 172.21.15.84:0/3995263768 wait complete. 2023-10-11T06:11:10.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.130+0000 7f0f317c5640 1 Processor -- start 2023-10-11T06:11:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.130+0000 7f0f317c5640 1 -- start start 2023-10-11T06:11:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f317c5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f317c5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c0ca300 0x7f0f2c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f317c5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c15d4f0 0x7f0f2c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f317c5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c155260 con 0x7f0f2c0c9360 2023-10-11T06:11:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f317c5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c15c1d0 con 0x7f0f2c0ca300 2023-10-11T06:11:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f317c5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c156eb0 con 0x7f0f2c15d4f0 2023-10-11T06:11:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f2affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f2affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37138/0 (socket says 172.21.15.84:37138) 2023-10-11T06:11:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.131+0000 7f0f2affd640 1 -- 172.21.15.84:0/3328286462 learned_addr learned my addr 172.21.15.84:0/3328286462 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f2affd640 1 -- 172.21.15.84:0/3328286462 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c15d4f0 msgr2=0x7f0f2c0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f2affd640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c15d4f0 0x7f0f2c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f2affd640 1 -- 172.21.15.84:0/3328286462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c0ca300 msgr2=0x7f0f2c0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f23fff640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c0ca300 0x7f0f2c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f2affd640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c0ca300 0x7f0f2c0b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f2affd640 1 -- 172.21.15.84:0/3328286462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24009b30 con 0x7f0f2c0c9360 2023-10-11T06:11:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f2affd640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c154d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00d800 tx=0x7f0f1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.132+0000 7f0f237fe640 1 -- 172.21.15.84:0/3328286462 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00b6c0 con 0x7f0f2c0c9360 2023-10-11T06:11:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.133+0000 7f0f237fe640 1 -- 172.21.15.84:0/3328286462 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00b820 con 0x7f0f2c0c9360 2023-10-11T06:11:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.133+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c157190 con 0x7f0f2c0c9360 2023-10-11T06:11:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.133+0000 7f0f237fe640 1 -- 172.21.15.84:0/3328286462 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00bbd0 con 0x7f0f2c0c9360 2023-10-11T06:11:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.133+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c157750 con 0x7f0f2c0c9360 2023-10-11T06:11:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.134+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c1578e0 con 0x7f0f2c0c9360 2023-10-11T06:11:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.135+0000 7f0f237fe640 1 -- 172.21.15.84:0/3328286462 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0f1c00bd30 con 0x7f0f2c0c9360 2023-10-11T06:11:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.135+0000 7f0f237fe640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f0c06a830 0x7f0f0c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.136+0000 7f0f237fe640 1 -- 172.21.15.84:0/3328286462 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0f1c089250 con 0x7f0f2c0c9360 2023-10-11T06:11:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.136+0000 7f0f23fff640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f0c06a830 0x7f0f0c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.136+0000 7f0f23fff640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f0c06a830 0x7f0f0c06cce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f14007fd0 tx=0x7f0f14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.143+0000 7f0f237fe640 1 -- 172.21.15.84:0/3328286462 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0f1c054780 con 0x7f0f2c0c9360 2023-10-11T06:11:10.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.318+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f2c073ed0 con 0x7f0f2c0c9360 2023-10-11T06:11:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.319+0000 7f0f237fe640 1 -- 172.21.15.84:0/3328286462 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14120 (secure 0 0 0) 0x7f0f1c057da0 con 0x7f0f2c0c9360 2023-10-11T06:11:10.322 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:10.322 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":20,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:09.122990+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f0c06a830 msgr2=0x7f0f0c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f0c06a830 0x7f0f0c06cce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f14007fd0 tx=0x7f0f14009040 comp rx=0 tx=0).stop 2023-10-11T06:11:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 msgr2=0x7f0f2c154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c154d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00d800 tx=0x7f0f1c00dcc0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 shutdown_connections 2023-10-11T06:11:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f0c06a830 0x7f0f0c06cce0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c15d4f0 0x7f0f2c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c0ca300 0x7f0f2c0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 --2- 172.21.15.84:0/3328286462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f2c0c9360 0x7f0f2c154d20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 >> 172.21.15.84:0/3328286462 conn(0x7f0f2c069c70 msgr2=0x7f0f2c0af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 shutdown_connections 2023-10-11T06:11:10.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.322+0000 7f0f317c5640 1 -- 172.21.15.84:0/3328286462 wait complete. 2023-10-11T06:11:10.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:10.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:10.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.537+0000 7fa92ded5640 1 Processor -- start 2023-10-11T06:11:10.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.537+0000 7fa92ded5640 1 -- start start 2023-10-11T06:11:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.539+0000 7fa92ded5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 0x7fa9280d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.539+0000 7fa92ded5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280d3450 con 0x7fa9280d0940 2023-10-11T06:11:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.539+0000 7fa92ded5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280cf220 con 0x7fa92815f7e0 2023-10-11T06:11:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.539+0000 7fa92ded5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280cf500 con 0x7fa9280d2ac0 2023-10-11T06:11:10.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.539+0000 7fa927fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa9280d0940 0x7fa9280d0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40552/0 (socket says 172.21.15.84:40552) 2023-10-11T06:11:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.539+0000 7fa927fff640 1 -- 172.21.15.84:0/986455867 learned_addr learned my addr 172.21.15.84:0/986455867 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa926ffd640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 0x7fa9280d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112440986 0 0) 0x7fa920001110 con 0x7fa9280d0940 2023-10-11T06:11:10.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa914003520 con 0x7fa9280d0940 2023-10-11T06:11:10.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3189681116 0 0) 0x7fa914003520 con 0x7fa9280d0940 2023-10-11T06:11:10.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 msgr2=0x7fa9280d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 0x7fa9280d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 >> v1:172.21.15.99:6789/0 conn(0x7fa92815f7e0 legacy=0x7fa9280cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280cf7e0 con 0x7fa9280d0940 2023-10-11T06:11:10.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fa920001800 con 0x7fa9280d0940 2023-10-11T06:11:10.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 >> v1:172.21.15.84:6789/0 conn(0x7fa9280d0940 legacy=0x7fa9280d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9140039a0 0x7fa914003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 0x7fa9140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa914005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280cf500 con 0x7fa914004d00 2023-10-11T06:11:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91400bb10 con 0x7fa9140039a0 2023-10-11T06:11:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.540+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91400bca0 con 0x7fa9280d2ac0 2023-10-11T06:11:10.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa927fff640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa914005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa9277fe640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9140039a0 0x7fa914003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa926ffd640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 0x7fa9140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa927fff640 1 -- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 msgr2=0x7fa9140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa927fff640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9280d2ac0 0x7fa9140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa927fff640 1 -- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9140039a0 msgr2=0x7fa914003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa927fff640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9140039a0 0x7fa914003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.541+0000 7fa927fff640 1 -- 172.21.15.84:0/986455867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914003520 con 0x7fa914004d00 2023-10-11T06:11:10.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.542+0000 7fa927fff640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa914005150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa920007fc0 tx=0x7fa9140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68899dac23784b31 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.542+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920005ad0 con 0x7fa914004d00 2023-10-11T06:11:10.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.542+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa920005c50 con 0x7fa914004d00 2023-10-11T06:11:10.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.542+0000 7fa9267fc640 1 -- 172.21.15.84:0/986455867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92002c7e0 con 0x7fa914004d00 2023-10-11T06:11:10.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.543+0000 7fa92ded5640 1 -- 172.21.15.84:0/986455867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 msgr2=0x7fa914005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.543+0000 7fa92ded5640 1 --2- 172.21.15.84:0/986455867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa914005150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa920007fc0 tx=0x7fa9140057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.543+0000 7fa927fff640 1 -- 172.21.15.84:0/986455867 reap_dead start 2023-10-11T06:11:10.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.543+0000 7fa92ded5640 1 -- 172.21.15.84:0/986455867 shutdown_connections 2023-10-11T06:11:10.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.543+0000 7fa92ded5640 1 -- 172.21.15.84:0/986455867 >> 172.21.15.84:0/986455867 conn(0x7fa9280713a0 msgr2=0x7fa9280717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:10.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.543+0000 7fa92ded5640 1 -- 172.21.15.84:0/986455867 shutdown_connections 2023-10-11T06:11:10.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.543+0000 7fa92ded5640 1 -- 172.21.15.84:0/986455867 wait complete. 2023-10-11T06:11:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.545+0000 7fa92ded5640 1 Processor -- start 2023-10-11T06:11:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.545+0000 7fa92ded5640 1 -- start start 2023-10-11T06:11:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa92ded5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa9280c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa92ded5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9280c8920 0x7fa9280b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa92ded5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9281502c0 0x7fa9280b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa92ded5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281f1790 con 0x7fa914004d00 2023-10-11T06:11:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa92ded5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281f2760 con 0x7fa9280c8920 2023-10-11T06:11:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa92ded5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280ca2a0 con 0x7fa9281502c0 2023-10-11T06:11:10.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa9277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa9280c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa9277fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa9280c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37160/0 (socket says 172.21.15.84:37160) 2023-10-11T06:11:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.546+0000 7fa9277fe640 1 -- 172.21.15.84:0/3660397048 learned_addr learned my addr 172.21.15.84:0/3660397048 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa9277fe640 1 -- 172.21.15.84:0/3660397048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9281502c0 msgr2=0x7fa9280b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa9277fe640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9281502c0 0x7fa9280b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa9277fe640 1 -- 172.21.15.84:0/3660397048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9280c8920 msgr2=0x7fa9280b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa9277fe640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9280c8920 0x7fa9280b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa9277fe640 1 -- 172.21.15.84:0/3660397048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280c9520 con 0x7fa914004d00 2023-10-11T06:11:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa9277fe640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa9280c9c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e920 tx=0x7fa91800ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa91ffff640 1 -- 172.21.15.84:0/3660397048 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918003b80 con 0x7fa914004d00 2023-10-11T06:11:10.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa91ffff640 1 -- 172.21.15.84:0/3660397048 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918003ce0 con 0x7fa914004d00 2023-10-11T06:11:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa91ffff640 1 -- 172.21.15.84:0/3660397048 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9180052f0 con 0x7fa914004d00 2023-10-11T06:11:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9281f19a0 con 0x7fa914004d00 2023-10-11T06:11:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.547+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9281f1f60 con 0x7fa914004d00 2023-10-11T06:11:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.548+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928073ed0 con 0x7fa914004d00 2023-10-11T06:11:10.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.549+0000 7fa91ffff640 1 -- 172.21.15.84:0/3660397048 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa918005450 con 0x7fa914004d00 2023-10-11T06:11:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.550+0000 7fa91ffff640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa90806a860 0x7fa90806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.550+0000 7fa91ffff640 1 -- 172.21.15.84:0/3660397048 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa918088da0 con 0x7fa914004d00 2023-10-11T06:11:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.550+0000 7fa926ffd640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa90806a860 0x7fa90806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.551+0000 7fa926ffd640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa90806a860 0x7fa90806cd10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa92815af90 tx=0x7fa91000b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.554+0000 7fa91ffff640 1 -- 172.21.15.84:0/3660397048 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa918054370 con 0x7fa914004d00 2023-10-11T06:11:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.737+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9280c9000 con 0x7fa914004d00 2023-10-11T06:11:10.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.737+0000 7fa91ffff640 1 -- 172.21.15.84:0/3660397048 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa918057990 con 0x7fa914004d00 2023-10-11T06:11:10.740 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:10.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa90806a860 msgr2=0x7fa90806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa90806a860 0x7fa90806cd10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa92815af90 tx=0x7fa91000b2e0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 msgr2=0x7fa9280c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa9280c9c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e920 tx=0x7fa91800ede0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 shutdown_connections 2023-10-11T06:11:10.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa90806a860 0x7fa90806cd10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9281502c0 0x7fa9280b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9280c8920 0x7fa9280b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 --2- 172.21.15.84:0/3660397048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa914004d00 0x7fa9280c9c00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 >> 172.21.15.84:0/3660397048 conn(0x7fa9280713a0 msgr2=0x7fa9280cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:10.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 shutdown_connections 2023-10-11T06:11:10.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.740+0000 7fa92ded5640 1 -- 172.21.15.84:0/3660397048 wait complete. 2023-10-11T06:11:10.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:10.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.955+0000 7f1ea1171640 1 Processor -- start 2023-10-11T06:11:10.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.956+0000 7f1ea1171640 1 -- start start 2023-10-11T06:11:10.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.957+0000 7f1ea1171640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e9c0d2f40 0x7f1e9c0d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.957+0000 7f1ea1171640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c0cf220 con 0x7f1e9c15f7e0 2023-10-11T06:11:10.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.957+0000 7f1ea1171640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c0cf500 con 0x7f1e9c0d2ac0 2023-10-11T06:11:10.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.957+0000 7f1ea1171640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c0cf7e0 con 0x7f1e9c0d2f40 2023-10-11T06:11:10.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.957+0000 7f1e9ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e9c15f7e0 0x7f1e9c0cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40556/0 (socket says 172.21.15.84:40556) 2023-10-11T06:11:10.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.957+0000 7f1e9ad9d640 1 -- 172.21.15.84:0/3576640077 learned_addr learned my addr 172.21.15.84:0/3576640077 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:10.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731272868 0 0) 0x7f1e9c0cf220 con 0x7f1e9c15f7e0 2023-10-11T06:11:10.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e88003520 con 0x7f1e9c15f7e0 2023-10-11T06:11:10.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9b59e640 1 --2- 172.21.15.84:0/3576640077 >> v2:172.21.15.99:3300/0 conn(0x7f1e9c0d2f40 0x7f1e9c0d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1660056014 0 0) 0x7f1e88003520 con 0x7f1e9c15f7e0 2023-10-11T06:11:10.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 >> v2:172.21.15.99:3300/0 conn(0x7f1e9c0d2f40 msgr2=0x7f1e9c0d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 --2- 172.21.15.84:0/3576640077 >> v2:172.21.15.99:3300/0 conn(0x7f1e9c0d2f40 0x7f1e9c0d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 >> v1:172.21.15.99:6789/0 conn(0x7f1e9c0d2ac0 legacy=0x7f1e9c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0cfac0 con 0x7f1e9c15f7e0 2023-10-11T06:11:10.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f1e8c001670 con 0x7f1e9c15f7e0 2023-10-11T06:11:10.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 >> v1:172.21.15.84:6789/0 conn(0x7f1e9c15f7e0 legacy=0x7f1e9c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e88003970 0x7f1e88003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e880047c0 0x7f1e88004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e88005170 0x7f1e8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0cf7e0 con 0x7f1e88005170 2023-10-11T06:11:10.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8800c080 con 0x7f1e88003970 2023-10-11T06:11:10.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8800c240 con 0x7f1e880047c0 2023-10-11T06:11:10.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.958+0000 7f1e9b59e640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e88005170 0x7f1e8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.959+0000 7f1e9ad9d640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e88003970 0x7f1e88003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.959+0000 7f1e9b59e640 1 -- 172.21.15.84:0/3576640077 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e880047c0 msgr2=0x7f1e88004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.959+0000 7f1e9b59e640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e880047c0 0x7f1e88004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.959+0000 7f1e9b59e640 1 -- 172.21.15.84:0/3576640077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e88003970 msgr2=0x7f1e88003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:10.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.959+0000 7f1e9b59e640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e88003970 0x7f1e88003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.959+0000 7f1e9b59e640 1 -- 172.21.15.84:0/3576640077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e88003520 con 0x7f1e88005170 2023-10-11T06:11:10.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.960+0000 7f1e9b59e640 1 -- 172.21.15.84:0/3576640077 reap_dead start 2023-10-11T06:11:10.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.960+0000 7f1e9b59e640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e88005170 0x7f1e8800b980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007360 tx=0x7f1e880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b7a5e0eafb725d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.960+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9402ccf0 con 0x7f1e88005170 2023-10-11T06:11:10.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.960+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9402ce90 con 0x7f1e88005170 2023-10-11T06:11:10.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.960+0000 7f1e9a59c640 1 -- 172.21.15.84:0/3576640077 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94003bd0 con 0x7f1e88005170 2023-10-11T06:11:10.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.960+0000 7f1ea1171640 1 -- 172.21.15.84:0/3576640077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e88005170 msgr2=0x7f1e8800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.960+0000 7f1ea1171640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e88005170 0x7f1e8800b980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007360 tx=0x7f1e880057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.961+0000 7f1ea1171640 1 -- 172.21.15.84:0/3576640077 shutdown_connections 2023-10-11T06:11:10.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.961+0000 7f1ea1171640 1 --2- 172.21.15.84:0/3576640077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e88005170 0x7f1e8800b980 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.961+0000 7f1ea1171640 1 -- 172.21.15.84:0/3576640077 >> 172.21.15.84:0/3576640077 conn(0x7f1e9c0713a0 msgr2=0x7f1e9c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:10.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.961+0000 7f1ea1171640 1 -- 172.21.15.84:0/3576640077 shutdown_connections 2023-10-11T06:11:10.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.961+0000 7f1ea1171640 1 -- 172.21.15.84:0/3576640077 wait complete. 2023-10-11T06:11:10.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.962+0000 7f1ea1171640 1 Processor -- start 2023-10-11T06:11:10.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.962+0000 7f1ea1171640 1 -- start start 2023-10-11T06:11:10.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.963+0000 7f1ea1171640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e88005170 0x7f1e9c0c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.963+0000 7f1ea1171640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9c14aab0 0x7f1e9c149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.963+0000 7f1ea1171640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9c14d660 0x7f1e9c159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.963+0000 7f1ea1171640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0ca140 con 0x7f1e9c14aab0 2023-10-11T06:11:10.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.963+0000 7f1ea1171640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0ca370 con 0x7f1e9c14d660 2023-10-11T06:11:10.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.963+0000 7f1ea1171640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0c78c0 con 0x7f1e88005170 2023-10-11T06:11:10.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.963+0000 7f1e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9c14aab0 0x7f1e9c149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9c14aab0 0x7f1e9c149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37172/0 (socket says 172.21.15.84:37172) 2023-10-11T06:11:10.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 -- 172.21.15.84:0/383924863 learned_addr learned my addr 172.21.15.84:0/383924863 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:10.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e9ad9d640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e88005170 0x7f1e9c0c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 -- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e88005170 msgr2=0x7f1e9c0c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e88005170 0x7f1e9c0c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 -- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9c14d660 msgr2=0x7f1e9c159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:10.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e9b59e640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9c14d660 0x7f1e9c159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9c14d660 0x7f1e9c159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:10.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 -- 172.21.15.84:0/383924863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0c7ba0 con 0x7f1e9c14aab0 2023-10-11T06:11:10.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e9ad9d640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e88005170 0x7f1e9c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:10.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e93fff640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9c14aab0 0x7f1e9c149f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400c480 tx=0x7f1e8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e937fe640 1 -- 172.21.15.84:0/383924863 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84012830 con 0x7f1e9c14aab0 2023-10-11T06:11:10.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.964+0000 7f1e937fe640 1 -- 172.21.15.84:0/383924863 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84012990 con 0x7f1e9c14aab0 2023-10-11T06:11:10.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.965+0000 7f1e937fe640 1 -- 172.21.15.84:0/383924863 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84012d40 con 0x7f1e9c14aab0 2023-10-11T06:11:10.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.965+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c0c7e80 con 0x7f1e9c14aab0 2023-10-11T06:11:10.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.965+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c14a480 con 0x7f1e9c14aab0 2023-10-11T06:11:10.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.966+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c0c83a0 con 0x7f1e9c14aab0 2023-10-11T06:11:10.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.967+0000 7f1e937fe640 1 -- 172.21.15.84:0/383924863 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1e8401b400 con 0x7f1e9c14aab0 2023-10-11T06:11:10.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.967+0000 7f1e937fe640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e7c06a860 0x7f1e7c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:10.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.968+0000 7f1e937fe640 1 -- 172.21.15.84:0/383924863 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1e84088ea0 con 0x7f1e9c14aab0 2023-10-11T06:11:10.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.968+0000 7f1e9ad9d640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e7c06a860 0x7f1e7c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:10.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.968+0000 7f1e9ad9d640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e7c06a860 0x7f1e7c06cd10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c001050 tx=0x7f1e8c000f60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:10.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:10.972+0000 7f1e937fe640 1 -- 172.21.15.84:0/383924863 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1e9c0c83a0 con 0x7f1e9c14aab0 2023-10-11T06:11:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.159+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e9c0c83a0 con 0x7f1e9c14aab0 2023-10-11T06:11:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.159+0000 7f1e937fe640 1 -- 172.21.15.84:0/383924863 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e9c0c83a0 con 0x7f1e9c14aab0 2023-10-11T06:11:11.162 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e7c06a860 msgr2=0x7f1e7c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e7c06a860 0x7f1e7c06cd10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c001050 tx=0x7f1e8c000f60 comp rx=0 tx=0).stop 2023-10-11T06:11:11.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9c14aab0 msgr2=0x7f1e9c149f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9c14aab0 0x7f1e9c149f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400c480 tx=0x7f1e8400c940 comp rx=0 tx=0).stop 2023-10-11T06:11:11.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 shutdown_connections 2023-10-11T06:11:11.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e7c06a860 0x7f1e7c06cd10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e9c14d660 0x7f1e9c159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e9c14aab0 0x7f1e9c149f40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 --2- 172.21.15.84:0/383924863 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e88005170 0x7f1e9c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 >> 172.21.15.84:0/383924863 conn(0x7f1e9c0713a0 msgr2=0x7f1e9c15ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:11.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 shutdown_connections 2023-10-11T06:11:11.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.162+0000 7f1ea1171640 1 -- 172.21.15.84:0/383924863 wait complete. 2023-10-11T06:11:11.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T06:11:11.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.395+0000 7f44b55ab640 1 Processor -- start 2023-10-11T06:11:11.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.396+0000 7f44b55ab640 1 -- start start 2023-10-11T06:11:11.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.397+0000 7f44b55ab640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b00cafb0 0x7f44b00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.397+0000 7f44b55ab640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44b00cc410 0x7f44b00cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.397+0000 7f44b55ab640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b00cb970 con 0x7f44b015c290 2023-10-11T06:11:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.397+0000 7f44b55ab640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b00cad70 con 0x7f44b00cc410 2023-10-11T06:11:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.397+0000 7f44b55ab640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b015fe40 con 0x7f44b00cafb0 2023-10-11T06:11:11.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44aeffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f44b015c290 0x7f44b00c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40570/0 (socket says 172.21.15.84:40570) 2023-10-11T06:11:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44aeffd640 1 -- 172.21.15.84:0/1496473620 learned_addr learned my addr 172.21.15.84:0/1496473620 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44af7fe640 1 --2- 172.21.15.84:0/1496473620 >> v2:172.21.15.84:3300/0 conn(0x7f44b00cc410 0x7f44b00cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44a7fff640 1 --2- 172.21.15.84:0/1496473620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b00cafb0 0x7f44b00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44ae7fc640 1 -- 172.21.15.84:0/1496473620 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813061745 0 0) 0x7f44b00cb970 con 0x7f44b015c290 2023-10-11T06:11:11.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44ae7fc640 1 -- 172.21.15.84:0/1496473620 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f449c003520 con 0x7f44b015c290 2023-10-11T06:11:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44af7fe640 1 -- 172.21.15.84:0/1496473620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b00cafb0 msgr2=0x7f44b00cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44af7fe640 1 --2- 172.21.15.84:0/1496473620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b00cafb0 0x7f44b00cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44af7fe640 1 -- 172.21.15.84:0/1496473620 >> v1:172.21.15.84:6789/0 conn(0x7f44b015c290 legacy=0x7f44b00c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.398+0000 7f44af7fe640 1 -- 172.21.15.84:0/1496473620 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b006ff30 con 0x7f44b00cc410 2023-10-11T06:11:11.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.399+0000 7f44af7fe640 1 --2- 172.21.15.84:0/1496473620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cc410 0x7f44b00cc870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009d90 tx=0x7f44a802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8891416bcfe7f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:11.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44ae7fc640 1 -- 172.21.15.84:0/1496473620 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a802cc60 con 0x7f44b00cc410 2023-10-11T06:11:11.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44ae7fc640 1 -- 172.21.15.84:0/1496473620 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a802cdc0 con 0x7f44b00cc410 2023-10-11T06:11:11.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44ae7fc640 1 -- 172.21.15.84:0/1496473620 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8034640 con 0x7f44b00cc410 2023-10-11T06:11:11.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44b55ab640 1 -- 172.21.15.84:0/1496473620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cc410 msgr2=0x7f44b00cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44b55ab640 1 --2- 172.21.15.84:0/1496473620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cc410 0x7f44b00cc870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009d90 tx=0x7f44a802beb0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44b55ab640 1 -- 172.21.15.84:0/1496473620 shutdown_connections 2023-10-11T06:11:11.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44b55ab640 1 --2- 172.21.15.84:0/1496473620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cc410 0x7f44b00cc870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44b55ab640 1 --2- 172.21.15.84:0/1496473620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b00cafb0 0x7f44b00cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44b55ab640 1 -- 172.21.15.84:0/1496473620 >> 172.21.15.84:0/1496473620 conn(0x7f44b0069c70 msgr2=0x7f44b006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:11.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.400+0000 7f44b55ab640 1 -- 172.21.15.84:0/1496473620 shutdown_connections 2023-10-11T06:11:11.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.401+0000 7f44b55ab640 1 -- 172.21.15.84:0/1496473620 wait complete. 2023-10-11T06:11:11.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.402+0000 7f44b55ab640 1 Processor -- start 2023-10-11T06:11:11.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.402+0000 7f44b55ab640 1 -- start start 2023-10-11T06:11:11.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44b55ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cafb0 0x7f44b00c2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44b55ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b00cc410 0x7f44b00c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44b55ab640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b015c290 0x7f44b00c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44b55ab640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00c54e0 con 0x7f44b00cafb0 2023-10-11T06:11:11.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44b55ab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00c4f20 con 0x7f44b00cc410 2023-10-11T06:11:11.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44b55ab640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00c5200 con 0x7f44b015c290 2023-10-11T06:11:11.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cafb0 0x7f44b00c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cafb0 0x7f44b00c2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37184/0 (socket says 172.21.15.84:37184) 2023-10-11T06:11:11.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.403+0000 7f44aeffd640 1 -- 172.21.15.84:0/410136911 learned_addr learned my addr 172.21.15.84:0/410136911 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:11.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44aeffd640 1 -- 172.21.15.84:0/410136911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b015c290 msgr2=0x7f44b00c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44aeffd640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b015c290 0x7f44b00c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44aeffd640 1 -- 172.21.15.84:0/410136911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b00cc410 msgr2=0x7f44b00c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44aeffd640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b00cc410 0x7f44b00c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44aeffd640 1 -- 172.21.15.84:0/410136911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a8009a70 con 0x7f44b00cafb0 2023-10-11T06:11:11.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44aeffd640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cafb0 0x7f44b00c2400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c700 tx=0x7f44a000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:11.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44a77fe640 1 -- 172.21.15.84:0/410136911 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0007270 con 0x7f44b00cafb0 2023-10-11T06:11:11.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44a77fe640 1 -- 172.21.15.84:0/410136911 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a00073d0 con 0x7f44b00cafb0 2023-10-11T06:11:11.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44a77fe640 1 -- 172.21.15.84:0/410136911 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0001d40 con 0x7f44b00cafb0 2023-10-11T06:11:11.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b00b15d0 con 0x7f44b00cafb0 2023-10-11T06:11:11.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.404+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b00c4c00 con 0x7f44b00cafb0 2023-10-11T06:11:11.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.406+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b00b1430 con 0x7f44b00cafb0 2023-10-11T06:11:11.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.406+0000 7f44a77fe640 1 -- 172.21.15.84:0/410136911 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f44a00057a0 con 0x7f44b00cafb0 2023-10-11T06:11:11.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.407+0000 7f44a77fe640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f449006a830 0x7f449006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.407+0000 7f44a77fe640 1 -- 172.21.15.84:0/410136911 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f44a00881b0 con 0x7f44b00cafb0 2023-10-11T06:11:11.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.408+0000 7f44a7fff640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f449006a830 0x7f449006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.408+0000 7f44a7fff640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f449006a830 0x7f449006cce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4498006fd0 tx=0x7f4498008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:11.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.412+0000 7f44a77fe640 1 -- 172.21.15.84:0/410136911 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f44b00b1430 con 0x7f44b00cafb0 2023-10-11T06:11:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.604+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f44b00b1af0 con 0x7f44b00cafb0 2023-10-11T06:11:11.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.604+0000 7f44a77fe640 1 -- 172.21.15.84:0/410136911 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f44b00b1af0 con 0x7f44b00cafb0 2023-10-11T06:11:11.607 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:11.607 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-11T06:11:11.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:11:11.607 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-11T06:11:11.608 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-11T06:11:11.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:11.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:11:11.608 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 2, 2023-10-11T06:11:11.608 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "rbd" 2023-10-11T06:11:11.608 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:11:11.609 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-11T06:11:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f449006a830 msgr2=0x7f449006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f449006a830 0x7f449006cce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4498006fd0 tx=0x7f4498008040 comp rx=0 tx=0).stop 2023-10-11T06:11:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cafb0 msgr2=0x7f44b00c2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cafb0 0x7f44b00c2400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c700 tx=0x7f44a000cbc0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 shutdown_connections 2023-10-11T06:11:11.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f449006a830 0x7f449006cce0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44b015c290 0x7f44b00c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b00cc410 0x7f44b00c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 --2- 172.21.15.84:0/410136911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44b00cafb0 0x7f44b00c2400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 >> 172.21.15.84:0/410136911 conn(0x7f44b0069c70 msgr2=0x7f44b00ac4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:11.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 shutdown_connections 2023-10-11T06:11:11.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.607+0000 7f44b55ab640 1 -- 172.21.15.84:0/410136911 wait complete. 2023-10-11T06:11:11.626 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T06:11:11.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T06:11:11.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.834+0000 7fcc8edd8640 1 Processor -- start 2023-10-11T06:11:11.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.834+0000 7fcc8edd8640 1 -- start start 2023-10-11T06:11:11.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.836+0000 7fcc8edd8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcc880cbad0 0x7fcc8806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.836+0000 7fcc8edd8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8806c730 con 0x7fcc8806c2f0 2023-10-11T06:11:11.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.836+0000 7fcc8edd8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc880cd560 con 0x7fcc88158cc0 2023-10-11T06:11:11.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.836+0000 7fcc8edd8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc88070640 con 0x7fcc880cbad0 2023-10-11T06:11:11.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.836+0000 7fcc8d5d5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcc880cbad0 0x7fcc8806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.837+0000 7fcc8e5d7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcc8806c2f0 0x7fcc880c8f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40576/0 (socket says 172.21.15.84:40576) 2023-10-11T06:11:11.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.837+0000 7fcc8d5d5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcc880cbad0 0x7fcc8806bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37186/0 (socket says 172.21.15.84:37186) 2023-10-11T06:11:11.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.837+0000 7fcc8e5d7640 1 -- 172.21.15.84:0/3040066666 learned_addr learned my addr 172.21.15.84:0/3040066666 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:11.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.837+0000 7fcc8d5d5640 1 -- 172.21.15.84:0/3040066666 >> v1:172.21.15.99:6789/0 conn(0x7fcc88158cc0 legacy=0x7fcc880ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.837+0000 7fcc8d5d5640 1 -- 172.21.15.84:0/3040066666 >> v1:172.21.15.84:6789/0 conn(0x7fcc8806c2f0 legacy=0x7fcc880c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.837+0000 7fcc8d5d5640 1 -- 172.21.15.84:0/3040066666 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc880cd840 con 0x7fcc880cbad0 2023-10-11T06:11:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.838+0000 7fcc8d5d5640 1 --2- 172.21.15.84:0/3040066666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc880cbad0 0x7fcc8806bdb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097a0 tx=0x7fcc7802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee330564f6d8da server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.838+0000 7fcc8cdd4640 1 -- 172.21.15.84:0/3040066666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78030610 con 0x7fcc880cbad0 2023-10-11T06:11:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.838+0000 7fcc8cdd4640 1 -- 172.21.15.84:0/3040066666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78030770 con 0x7fcc880cbad0 2023-10-11T06:11:11.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.838+0000 7fcc8cdd4640 1 -- 172.21.15.84:0/3040066666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78030b00 con 0x7fcc880cbad0 2023-10-11T06:11:11.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.839+0000 7fcc8edd8640 1 -- 172.21.15.84:0/3040066666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc880cbad0 msgr2=0x7fcc8806bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.839+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/3040066666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc880cbad0 0x7fcc8806bdb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097a0 tx=0x7fcc7802b960 comp rx=0 tx=0).stop 2023-10-11T06:11:11.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.839+0000 7fcc8edd8640 1 -- 172.21.15.84:0/3040066666 shutdown_connections 2023-10-11T06:11:11.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.839+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/3040066666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc880cbad0 0x7fcc8806bdb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.839+0000 7fcc8edd8640 1 -- 172.21.15.84:0/3040066666 >> 172.21.15.84:0/3040066666 conn(0x7fcc88069c70 msgr2=0x7fcc8806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:11.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.839+0000 7fcc8edd8640 1 -- 172.21.15.84:0/3040066666 shutdown_connections 2023-10-11T06:11:11.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.839+0000 7fcc8edd8640 1 -- 172.21.15.84:0/3040066666 wait complete. 2023-10-11T06:11:11.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.841+0000 7fcc8edd8640 1 Processor -- start 2023-10-11T06:11:11.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.841+0000 7fcc8edd8640 1 -- start start 2023-10-11T06:11:11.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8edd8640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc8806c2f0 0x7fcc880c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8edd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc880cbad0 0x7fcc881f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8edd8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc88158cc0 0x7fcc880c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8edd8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c5d50 con 0x7fcc88158cc0 2023-10-11T06:11:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8edd8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c4790 con 0x7fcc880cbad0 2023-10-11T06:11:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8edd8640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880c56b0 con 0x7fcc8806c2f0 2023-10-11T06:11:11.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8e5d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc88158cc0 0x7fcc880c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8e5d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc88158cc0 0x7fcc880c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37192/0 (socket says 172.21.15.84:37192) 2023-10-11T06:11:11.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.842+0000 7fcc8e5d7640 1 -- 172.21.15.84:0/742281852 learned_addr learned my addr 172.21.15.84:0/742281852 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:11.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8ddd6640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc8806c2f0 0x7fcc880c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8d5d5640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc880cbad0 0x7fcc881f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8e5d7640 1 -- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc8806c2f0 msgr2=0x7fcc880c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8e5d7640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc8806c2f0 0x7fcc880c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8e5d7640 1 -- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc880cbad0 msgr2=0x7fcc881f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:11.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8e5d7640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc880cbad0 0x7fcc881f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:11.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8e5d7640 1 -- 172.21.15.84:0/742281852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78009400 con 0x7fcc88158cc0 2023-10-11T06:11:11.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8d5d5640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc880cbad0 0x7fcc881f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:11.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8e5d7640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc88158cc0 0x7fcc880c6930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000ca80 tx=0x7fcc8000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:11.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc7e7fc640 1 -- 172.21.15.84:0/742281852 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000ac30 con 0x7fcc88158cc0 2023-10-11T06:11:11.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc7e7fc640 1 -- 172.21.15.84:0/742281852 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8000ad90 con 0x7fcc88158cc0 2023-10-11T06:11:11.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc880c7340 con 0x7fcc88158cc0 2023-10-11T06:11:11.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc7e7fc640 1 -- 172.21.15.84:0/742281852 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80012600 con 0x7fcc88158cc0 2023-10-11T06:11:11.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.843+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc881f1500 con 0x7fcc88158cc0 2023-10-11T06:11:11.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.845+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc880c5e90 con 0x7fcc88158cc0 2023-10-11T06:11:11.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.846+0000 7fcc7e7fc640 1 -- 172.21.15.84:0/742281852 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fcc80007070 con 0x7fcc88158cc0 2023-10-11T06:11:11.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.846+0000 7fcc7e7fc640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcc6c06a830 0x7fcc6c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:11.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.847+0000 7fcc7e7fc640 1 -- 172.21.15.84:0/742281852 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcc80088a20 con 0x7fcc88158cc0 2023-10-11T06:11:11.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.847+0000 7fcc8ddd6640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcc6c06a830 0x7fcc6c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:11.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.847+0000 7fcc8ddd6640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcc6c06a830 0x7fcc6c06cce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc84000f60 tx=0x7fcc84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:11.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:11.851+0000 7fcc7e7fc640 1 -- 172.21.15.84:0/742281852 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fcc880c5e90 con 0x7fcc88158cc0 2023-10-11T06:11:12.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.025+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcc880c5e90 con 0x7fcc88158cc0 2023-10-11T06:11:12.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.168+0000 7fcc7e7fc640 1 -- 172.21.15.84:0/742281852 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc880c5e90 con 0x7fcc88158cc0 2023-10-11T06:11:12.171 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-11T06:11:12.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.172+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcc6c06a830 msgr2=0x7fcc6c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:12.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.172+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcc6c06a830 0x7fcc6c06cce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc84000f60 tx=0x7fcc84009040 comp rx=0 tx=0).stop 2023-10-11T06:11:12.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc88158cc0 msgr2=0x7fcc880c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:12.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc88158cc0 0x7fcc880c6930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000ca80 tx=0x7fcc8000cf40 comp rx=0 tx=0).stop 2023-10-11T06:11:12.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 shutdown_connections 2023-10-11T06:11:12.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcc6c06a830 0x7fcc6c06cce0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc88158cc0 0x7fcc880c6930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc880cbad0 0x7fcc881f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 --2- 172.21.15.84:0/742281852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc8806c2f0 0x7fcc880c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 >> 172.21.15.84:0/742281852 conn(0x7fcc88069c70 msgr2=0x7fcc880718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:12.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 shutdown_connections 2023-10-11T06:11:12.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.173+0000 7fcc8edd8640 1 -- 172.21.15.84:0/742281852 wait complete. 2023-10-11T06:11:12.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T06:11:12.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.401+0000 7f54dcf0a640 1 Processor -- start 2023-10-11T06:11:12.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.402+0000 7f54dcf0a640 1 -- start start 2023-10-11T06:11:12.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54dcf0a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54d815f7e0 0x7f54d80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:12.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54dcf0a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54d80d31d0 0x7f54d80d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:12.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54dcf0a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d80cf220 con 0x7f54d80cd6c0 2023-10-11T06:11:12.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54dcf0a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d80cf500 con 0x7f54d80d31d0 2023-10-11T06:11:12.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54dcf0a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d80cf7e0 con 0x7f54d815f7e0 2023-10-11T06:11:12.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54d6d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54d80d31d0 0x7f54d80d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:12.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54d6d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54d80d31d0 0x7f54d80d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37206/0 (socket says 172.21.15.84:37206) 2023-10-11T06:11:12.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54d6d9d640 1 -- 172.21.15.84:0/715792897 learned_addr learned my addr 172.21.15.84:0/715792897 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:12.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.403+0000 7f54d5d9b640 1 --1- 172.21.15.84:0/715792897 >> v1:172.21.15.84:6789/0 conn(0x7f54d80cd6c0 0x7f54d80d2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40588/0 (socket says 172.21.15.84:40588) 2023-10-11T06:11:12.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.404+0000 7f54d659c640 1 --2- 172.21.15.84:0/715792897 >> v2:172.21.15.99:3300/0 conn(0x7f54d815f7e0 0x7f54d80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:12.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.404+0000 7f54d6d9d640 1 -- 172.21.15.84:0/715792897 >> v2:172.21.15.99:3300/0 conn(0x7f54d815f7e0 msgr2=0x7f54d80cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:12.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.404+0000 7f54d6d9d640 1 --2- 172.21.15.84:0/715792897 >> v2:172.21.15.99:3300/0 conn(0x7f54d815f7e0 0x7f54d80cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.404+0000 7f54d6d9d640 1 -- 172.21.15.84:0/715792897 >> v1:172.21.15.84:6789/0 conn(0x7f54d80cd6c0 legacy=0x7f54d80d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:12.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.404+0000 7f54d6d9d640 1 -- 172.21.15.84:0/715792897 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d80cfac0 con 0x7f54d80d31d0 2023-10-11T06:11:12.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.404+0000 7f54d559a640 1 -- 172.21.15.84:0/715792897 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725766504 0 0) 0x7f54d80cf220 con 0x7f54d80cd6c0 2023-10-11T06:11:12.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.405+0000 7f54d6d9d640 1 --2- 172.21.15.84:0/715792897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80d31d0 0x7f54d80d0940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54d8072100 tx=0x7f54d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881376064b0786db server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:12.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.405+0000 7f54d559a640 1 -- 172.21.15.84:0/715792897 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0030860 con 0x7f54d80d31d0 2023-10-11T06:11:12.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.405+0000 7f54d559a640 1 -- 172.21.15.84:0/715792897 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d00309c0 con 0x7f54d80d31d0 2023-10-11T06:11:12.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.405+0000 7f54d559a640 1 -- 172.21.15.84:0/715792897 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0030d50 con 0x7f54d80d31d0 2023-10-11T06:11:12.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 -- 172.21.15.84:0/715792897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80d31d0 msgr2=0x7f54d80d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:12.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/715792897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80d31d0 0x7f54d80d0940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54d8072100 tx=0x7f54d002b9a0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 -- 172.21.15.84:0/715792897 shutdown_connections 2023-10-11T06:11:12.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/715792897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80d31d0 0x7f54d80d0940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/715792897 >> v2:172.21.15.99:3300/0 conn(0x7f54d815f7e0 0x7f54d80cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 -- 172.21.15.84:0/715792897 >> 172.21.15.84:0/715792897 conn(0x7f54d80713a0 msgr2=0x7f54d80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:12.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 -- 172.21.15.84:0/715792897 shutdown_connections 2023-10-11T06:11:12.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.406+0000 7f54dcf0a640 1 -- 172.21.15.84:0/715792897 wait complete. 2023-10-11T06:11:12.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.407+0000 7f54dcf0a640 1 Processor -- start 2023-10-11T06:11:12.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.408+0000 7f54dcf0a640 1 -- start start 2023-10-11T06:11:12.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.408+0000 7f54dcf0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80cd6c0 0x7f54d80b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:12.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.408+0000 7f54dcf0a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d80d31d0 0x7f54d8154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:12.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54dcf0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d815f7e0 0x7f54d815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:12.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54dcf0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80b7a80 con 0x7f54d80cd6c0 2023-10-11T06:11:12.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54dcf0a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80b64a0 con 0x7f54d815f7e0 2023-10-11T06:11:12.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54dcf0a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80b6780 con 0x7f54d80d31d0 2023-10-11T06:11:12.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80cd6c0 0x7f54d80b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:12.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80cd6c0 0x7f54d80b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37212/0 (socket says 172.21.15.84:37212) 2023-10-11T06:11:12.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 -- 172.21.15.84:0/2257844105 learned_addr learned my addr 172.21.15.84:0/2257844105 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:12.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 -- 172.21.15.84:0/2257844105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d80d31d0 msgr2=0x7f54d8154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:12.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d80d31d0 0x7f54d8154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 -- 172.21.15.84:0/2257844105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d815f7e0 msgr2=0x7f54d815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:12.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d815f7e0 0x7f54d815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:12.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 -- 172.21.15.84:0/2257844105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d00094d0 con 0x7f54d80cd6c0 2023-10-11T06:11:12.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.409+0000 7f54d659c640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80cd6c0 0x7f54d80b7540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54c000d800 tx=0x7f54c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:12.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.410+0000 7f54ceffd640 1 -- 172.21.15.84:0/2257844105 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c000b6c0 con 0x7f54d80cd6c0 2023-10-11T06:11:12.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.410+0000 7f54ceffd640 1 -- 172.21.15.84:0/2257844105 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c000b820 con 0x7f54d80cd6c0 2023-10-11T06:11:12.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.410+0000 7f54ceffd640 1 -- 172.21.15.84:0/2257844105 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c000bbb0 con 0x7f54d80cd6c0 2023-10-11T06:11:12.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.410+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d80b6a60 con 0x7f54d80cd6c0 2023-10-11T06:11:12.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.410+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d80b7020 con 0x7f54d80cd6c0 2023-10-11T06:11:12.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.411+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d80b71e0 con 0x7f54d80cd6c0 2023-10-11T06:11:12.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.412+0000 7f54ceffd640 1 -- 172.21.15.84:0/2257844105 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f54c000bd10 con 0x7f54d80cd6c0 2023-10-11T06:11:12.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.413+0000 7f54ceffd640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f54b806a830 0x7f54b806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:12.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.414+0000 7f54ceffd640 1 -- 172.21.15.84:0/2257844105 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f54c00893a0 con 0x7f54d80cd6c0 2023-10-11T06:11:12.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.414+0000 7f54d5d9b640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f54b806a830 0x7f54b806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:12.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.414+0000 7f54d5d9b640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f54b806a830 0x7f54b806cce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54c8007fd0 tx=0x7f54c8007ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:12.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.419+0000 7f54ceffd640 1 -- 172.21.15.84:0/2257844105 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f54c0054760 con 0x7f54d80cd6c0 2023-10-11T06:11:12.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:12.607+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f54d8073ed0 con 0x7f54d80cd6c0 2023-10-11T06:11:13.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.183+0000 7f54ceffd640 1 -- 172.21.15.84:0/2257844105 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v22) v1 ==== 170+0+0 (secure 0 0 0) 0x7f54c0057d80 con 0x7f54d80cd6c0 2023-10-11T06:11:13.186 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-11T06:11:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.193+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f54b806a830 msgr2=0x7f54b806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.193+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f54b806a830 0x7f54b806cce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54c8007fd0 tx=0x7f54c8007ce0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.193+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80cd6c0 msgr2=0x7f54d80b7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.193+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80cd6c0 0x7f54d80b7540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54c000d800 tx=0x7f54c000dcc0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.198+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 shutdown_connections 2023-10-11T06:11:13.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.198+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f54b806a830 0x7f54b806cce0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.198+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d815f7e0 0x7f54d815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.198+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d80d31d0 0x7f54d8154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.198+0000 7f54dcf0a640 1 --2- 172.21.15.84:0/2257844105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54d80cd6c0 0x7f54d80b7540 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.198+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 >> 172.21.15.84:0/2257844105 conn(0x7f54d80713a0 msgr2=0x7f54d815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.202+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 shutdown_connections 2023-10-11T06:11:13.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.202+0000 7f54dcf0a640 1 -- 172.21.15.84:0/2257844105 wait complete. 2023-10-11T06:11:13.230 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T06:11:13.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.417+0000 7fb38d25f640 1 Processor -- start 2023-10-11T06:11:13.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.417+0000 7fb38d25f640 1 -- start start 2023-10-11T06:11:13.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.418+0000 7fb38d25f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb38814ff30 0x7fb3880c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.418+0000 7fb38d25f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3880c9b90 con 0x7fb388150300 2023-10-11T06:11:13.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.418+0000 7fb38d25f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38815da10 con 0x7fb3880ce510 2023-10-11T06:11:13.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.418+0000 7fb38d25f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880cc990 con 0x7fb38814ff30 2023-10-11T06:11:13.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.419+0000 7fb38659c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb388150300 0x7fb3880cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40596/0 (socket says 172.21.15.84:40596) 2023-10-11T06:11:13.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.419+0000 7fb38659c640 1 -- 172.21.15.84:0/3600597888 learned_addr learned my addr 172.21.15.84:0/3600597888 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:13.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.419+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106012292 0 0) 0x7fb3880c9b90 con 0x7fb388150300 2023-10-11T06:11:13.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.419+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb370003520 con 0x7fb388150300 2023-10-11T06:11:13.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.419+0000 7fb386d9d640 1 --2- 172.21.15.84:0/3600597888 >> v2:172.21.15.99:3300/0 conn(0x7fb38814ff30 0x7fb3880c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.419+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1441196657 0 0) 0x7fb370003520 con 0x7fb388150300 2023-10-11T06:11:13.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.419+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 >> v2:172.21.15.99:3300/0 conn(0x7fb38814ff30 msgr2=0x7fb3880c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:13.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 --2- 172.21.15.84:0/3600597888 >> v2:172.21.15.99:3300/0 conn(0x7fb38814ff30 0x7fb3880c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 >> v1:172.21.15.99:6789/0 conn(0x7fb3880ce510 legacy=0x7fb3880ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38815ce00 con 0x7fb388150300 2023-10-11T06:11:13.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fb37c001670 con 0x7fb388150300 2023-10-11T06:11:13.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 >> v1:172.21.15.84:6789/0 conn(0x7fb388150300 legacy=0x7fb3880cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb370003970 0x7fb370003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3700047c0 0x7fb370004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb370005290 0x7fb37000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880cc990 con 0x7fb370005290 2023-10-11T06:11:13.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37000c150 con 0x7fb3700047c0 2023-10-11T06:11:13.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37000c310 con 0x7fb370003970 2023-10-11T06:11:13.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb386d9d640 1 --2- 172.21.15.84:0/3600597888 >> v2:172.21.15.99:3300/0 conn(0x7fb38814ff30 0x7fb3880c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:11:13.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb38759e640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb370005290 0x7fb37000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb38659c640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3700047c0 0x7fb370004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb386d9d640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb370003970 0x7fb370003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb38759e640 1 -- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb370003970 msgr2=0x7fb370003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:13.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb38759e640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb370003970 0x7fb370003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb38759e640 1 -- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3700047c0 msgr2=0x7fb370004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:13.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb38759e640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3700047c0 0x7fb370004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.420+0000 7fb38759e640 1 -- 172.21.15.84:0/3600597888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb370003520 con 0x7fb370005290 2023-10-11T06:11:13.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.421+0000 7fb38759e640 1 -- 172.21.15.84:0/3600597888 reap_dead start 2023-10-11T06:11:13.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.421+0000 7fb38759e640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb370005290 0x7fb37000b980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb380001e50 tx=0x7fb3700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3a924f3fcec43b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380003b70 con 0x7fb370005290 2023-10-11T06:11:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380003d10 con 0x7fb370005290 2023-10-11T06:11:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb385d9b640 1 -- 172.21.15.84:0/3600597888 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380005310 con 0x7fb370005290 2023-10-11T06:11:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb38d25f640 1 -- 172.21.15.84:0/3600597888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb370005290 msgr2=0x7fb37000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb38d25f640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb370005290 0x7fb37000b980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb380001e50 tx=0x7fb3700057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb38d25f640 1 -- 172.21.15.84:0/3600597888 shutdown_connections 2023-10-11T06:11:13.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb38d25f640 1 --2- 172.21.15.84:0/3600597888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb370005290 0x7fb37000b980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.422+0000 7fb38d25f640 1 -- 172.21.15.84:0/3600597888 >> 172.21.15.84:0/3600597888 conn(0x7fb388069c70 msgr2=0x7fb38806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.423+0000 7fb38d25f640 1 -- 172.21.15.84:0/3600597888 shutdown_connections 2023-10-11T06:11:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.423+0000 7fb38d25f640 1 -- 172.21.15.84:0/3600597888 wait complete. 2023-10-11T06:11:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.424+0000 7fb38d25f640 1 Processor -- start 2023-10-11T06:11:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.424+0000 7fb38d25f640 1 -- start start 2023-10-11T06:11:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.425+0000 7fb38d25f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb370005290 0x7fb388158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.425+0000 7fb38d25f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3881f6110 0x7fb38813c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.425+0000 7fb38d25f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb388140180 0x7fb3881405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.425+0000 7fb38d25f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb388157200 con 0x7fb3881f6110 2023-10-11T06:11:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.425+0000 7fb38d25f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881577c0 con 0x7fb370005290 2023-10-11T06:11:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.425+0000 7fb38d25f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3881450c0 con 0x7fb388140180 2023-10-11T06:11:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3881f6110 0x7fb38813c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3881f6110 0x7fb38813c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37238/0 (socket says 172.21.15.84:37238) 2023-10-11T06:11:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 -- 172.21.15.84:0/2078246441 learned_addr learned my addr 172.21.15.84:0/2078246441 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb386d9d640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb370005290 0x7fb388158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 -- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb388140180 msgr2=0x7fb3881405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38759e640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb388140180 0x7fb3881405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb388140180 0x7fb3881405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 -- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb370005290 msgr2=0x7fb388158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb370005290 0x7fb388158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 -- 172.21.15.84:0/2078246441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3881453a0 con 0x7fb3881f6110 2023-10-11T06:11:13.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb386d9d640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb370005290 0x7fb388158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb38659c640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3881f6110 0x7fb38813c240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00e730 tx=0x7fb37c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb36f7fe640 1 -- 172.21.15.84:0/2078246441 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00c6c0 con 0x7fb3881f6110 2023-10-11T06:11:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.426+0000 7fb36f7fe640 1 -- 172.21.15.84:0/2078246441 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00c820 con 0x7fb3881f6110 2023-10-11T06:11:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.427+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb388157450 con 0x7fb3881f6110 2023-10-11T06:11:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.427+0000 7fb36f7fe640 1 -- 172.21.15.84:0/2078246441 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00cbd0 con 0x7fb3881f6110 2023-10-11T06:11:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.427+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3881f6560 con 0x7fb3881f6110 2023-10-11T06:11:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.428+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38806ab90 con 0x7fb3881f6110 2023-10-11T06:11:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.429+0000 7fb36f7fe640 1 -- 172.21.15.84:0/2078246441 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb37c00cd30 con 0x7fb3881f6110 2023-10-11T06:11:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.430+0000 7fb36f7fe640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb37406a830 0x7fb37406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.430+0000 7fb36f7fe640 1 -- 172.21.15.84:0/2078246441 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fb37c088da0 con 0x7fb3881f6110 2023-10-11T06:11:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.431+0000 7fb386d9d640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb37406a830 0x7fb37406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.431+0000 7fb386d9d640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb37406a830 0x7fb37406cce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb378006fd0 tx=0x7fb378008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.435+0000 7fb36f7fe640 1 -- 172.21.15.84:0/2078246441 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb37c054080 con 0x7fb3881f6110 2023-10-11T06:11:13.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.663+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb388073ed0 con 0x7fb3881f6110 2023-10-11T06:11:13.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.672+0000 7fb36f7fe640 1 -- 172.21.15.84:0/2078246441 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb37c0576a0 con 0x7fb3881f6110 2023-10-11T06:11:13.675 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 3 and data pool 4 2023-10-11T06:11:13.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.677+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb37406a830 msgr2=0x7fb37406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.677+0000 7fb38d25f640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb37406a830 0x7fb37406cce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb378006fd0 tx=0x7fb378008040 comp rx=0 tx=0).stop 2023-10-11T06:11:13.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.677+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3881f6110 msgr2=0x7fb38813c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.677+0000 7fb38d25f640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3881f6110 0x7fb38813c240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00e730 tx=0x7fb37c00ebf0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.683+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 shutdown_connections 2023-10-11T06:11:13.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.683+0000 7fb38d25f640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb37406a830 0x7fb37406cce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.683+0000 7fb38d25f640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb388140180 0x7fb3881405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.683+0000 7fb38d25f640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3881f6110 0x7fb38813c240 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.683+0000 7fb38d25f640 1 --2- 172.21.15.84:0/2078246441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb370005290 0x7fb388158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.683+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 >> 172.21.15.84:0/2078246441 conn(0x7fb388069c70 msgr2=0x7fb3880cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:13.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.685+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 shutdown_connections 2023-10-11T06:11:13.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.686+0000 7fb38d25f640 1 -- 172.21.15.84:0/2078246441 wait complete. 2023-10-11T06:11:13.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:13.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.888+0000 7f94b2e8b640 1 Processor -- start 2023-10-11T06:11:13.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.888+0000 7f94b2e8b640 1 -- start start 2023-10-11T06:11:13.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b2e8b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94ac15c290 0x7f94ac0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b2e8b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0cb6c0 0x7f94ac0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b2e8b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac15fe40 con 0x7f94ac0c9190 2023-10-11T06:11:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b2e8b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac0cad70 con 0x7f94ac15c290 2023-10-11T06:11:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b2e8b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac06ff30 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b0c27640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94ac15c290 0x7f94ac0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b0c27640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94ac15c290 0x7f94ac0c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37252/0 (socket says 172.21.15.84:37252) 2023-10-11T06:11:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b0c27640 1 -- 172.21.15.84:0/3644327091 learned_addr learned my addr 172.21.15.84:0/3644327091 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.890+0000 7f94b0c27640 1 -- 172.21.15.84:0/3644327091 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0cb6c0 msgr2=0x7f94ac0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.891+0000 7f94b0c27640 1 --2- 172.21.15.84:0/3644327091 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0cb6c0 0x7f94ac0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.891+0000 7f94b0c27640 1 -- 172.21.15.84:0/3644327091 >> v1:172.21.15.99:6789/0 conn(0x7f94ac0c9190 legacy=0x7f94ac0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:13.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.891+0000 7f94b0c27640 1 -- 172.21.15.84:0/3644327091 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac1600e0 con 0x7f94ac15c290 2023-10-11T06:11:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.892+0000 7f94b0c27640 1 --2- 172.21.15.84:0/3644327091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac15c290 0x7f94ac0c8c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009860 tx=0x7f94a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ef3780ae902309 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.892+0000 7f94ab7fe640 1 -- 172.21.15.84:0/3644327091 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0031640 con 0x7f94ac15c290 2023-10-11T06:11:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.892+0000 7f94ab7fe640 1 -- 172.21.15.84:0/3644327091 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a00317a0 con 0x7f94ac15c290 2023-10-11T06:11:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.892+0000 7f94ab7fe640 1 -- 172.21.15.84:0/3644327091 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0031b30 con 0x7f94ac15c290 2023-10-11T06:11:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.892+0000 7f94b2e8b640 1 -- 172.21.15.84:0/3644327091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac15c290 msgr2=0x7f94ac0c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.892+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/3644327091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac15c290 0x7f94ac0c8c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009860 tx=0x7f94a002b8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.893+0000 7f94b2e8b640 1 -- 172.21.15.84:0/3644327091 shutdown_connections 2023-10-11T06:11:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.893+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/3644327091 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0cb6c0 0x7f94ac0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.893+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/3644327091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac15c290 0x7f94ac0c8c50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.893+0000 7f94b2e8b640 1 -- 172.21.15.84:0/3644327091 >> 172.21.15.84:0/3644327091 conn(0x7f94ac069c70 msgr2=0x7f94ac06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.893+0000 7f94b2e8b640 1 -- 172.21.15.84:0/3644327091 shutdown_connections 2023-10-11T06:11:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.893+0000 7f94b2e8b640 1 -- 172.21.15.84:0/3644327091 wait complete. 2023-10-11T06:11:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.894+0000 7f94b2e8b640 1 Processor -- start 2023-10-11T06:11:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.894+0000 7f94b2e8b640 1 -- start start 2023-10-11T06:11:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.895+0000 7f94b2e8b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0c9190 0x7f94ac15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.895+0000 7f94b2e8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac0cb6c0 0x7f94ac0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.895+0000 7f94b2e8b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac15c290 0x7f94ac0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.895+0000 7f94b2e8b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0b4350 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.895+0000 7f94b2e8b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac159700 con 0x7f94ac15c290 2023-10-11T06:11:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.895+0000 7f94b2e8b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac1f1ac0 con 0x7f94ac0c9190 2023-10-11T06:11:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.895+0000 7f94abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac0cb6c0 0x7f94ac0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac0cb6c0 0x7f94ac0b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37254/0 (socket says 172.21.15.84:37254) 2023-10-11T06:11:13.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 -- 172.21.15.84:0/430932685 learned_addr learned my addr 172.21.15.84:0/430932685 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 -- 172.21.15.84:0/430932685 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0c9190 msgr2=0x7f94ac15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0c9190 0x7f94ac15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 -- 172.21.15.84:0/430932685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac15c290 msgr2=0x7f94ac0c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac15c290 0x7f94ac0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 -- 172.21.15.84:0/430932685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0009470 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94abfff640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac0cb6c0 0x7f94ac0b3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f949800c400 tx=0x7f949800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94a97fa640 1 -- 172.21.15.84:0/430932685 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94980035f0 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94a97fa640 1 -- 172.21.15.84:0/430932685 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498003750 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94a97fa640 1 -- 172.21.15.84:0/430932685 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498003b40 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.896+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac0c33f0 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.897+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac0c3970 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.898+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac15f790 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.899+0000 7f94a97fa640 1 -- 172.21.15.84:0/430932685 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9498003ca0 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.899+0000 7f94a97fa640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949006a830 0x7f949006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.900+0000 7f94a97fa640 1 -- 172.21.15.84:0/430932685 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94980881c0 con 0x7f94ac0cb6c0 2023-10-11T06:11:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.900+0000 7f94b0c27640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949006a830 0x7f949006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:13.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.900+0000 7f94b0c27640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949006a830 0x7f949006cce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94a0036000 tx=0x7f94a0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:13.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:13.904+0000 7f94a97fa640 1 -- 172.21.15.84:0/430932685 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f94980534e0 con 0x7f94ac0cb6c0 2023-10-11T06:11:14.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.078+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94ac073ed0 con 0x7f94ac0cb6c0 2023-10-11T06:11:14.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.080+0000 7f94a97fa640 1 -- 172.21.15.84:0/430932685 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17776 (secure 0 0 0) 0x7f9498056b00 con 0x7f94ac0cb6c0 2023-10-11T06:11:14.083 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:14.083 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:13.666386+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:14.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.083+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949006a830 msgr2=0x7f949006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949006a830 0x7f949006cce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94a0036000 tx=0x7f94a0030000 comp rx=0 tx=0).stop 2023-10-11T06:11:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac0cb6c0 msgr2=0x7f94ac0b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac0cb6c0 0x7f94ac0b3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f949800c400 tx=0x7f949800c8c0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 shutdown_connections 2023-10-11T06:11:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949006a830 0x7f949006cce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac15c290 0x7f94ac0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ac0cb6c0 0x7f94ac0b3770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 --2- 172.21.15.84:0/430932685 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac0c9190 0x7f94ac15c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 >> 172.21.15.84:0/430932685 conn(0x7f94ac069c70 msgr2=0x7f94ac0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 shutdown_connections 2023-10-11T06:11:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.084+0000 7f94b2e8b640 1 -- 172.21.15.84:0/430932685 wait complete. 2023-10-11T06:11:14.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:14.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.300+0000 7fcc10d6a640 1 Processor -- start 2023-10-11T06:11:14.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.300+0000 7fcc10d6a640 1 -- start start 2023-10-11T06:11:14.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc10d6a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcc0c0cd330 0x7fcc0c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc10d6a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcc0c071600 0x7fcc0c0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc10d6a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c158b20 con 0x7fcc0c158750 2023-10-11T06:11:14.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc10d6a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0ce1f0 con 0x7fcc0c0cd330 2023-10-11T06:11:14.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc10d6a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cc570 con 0x7fcc0c071600 2023-10-11T06:11:14.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcc0c0cd330 0x7fcc0c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcc0c0cd330 0x7fcc0c0ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37270/0 (socket says 172.21.15.84:37270) 2023-10-11T06:11:14.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0affd640 1 -- 172.21.15.84:0/584204296 learned_addr learned my addr 172.21.15.84:0/584204296 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:14.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0bfff640 1 --2- 172.21.15.84:0/584204296 >> v2:172.21.15.99:3300/0 conn(0x7fcc0c071600 0x7fcc0c0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0affd640 1 -- 172.21.15.84:0/584204296 >> v2:172.21.15.99:3300/0 conn(0x7fcc0c071600 msgr2=0x7fcc0c0ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:14.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0affd640 1 --2- 172.21.15.84:0/584204296 >> v2:172.21.15.99:3300/0 conn(0x7fcc0c071600 0x7fcc0c0ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0affd640 1 -- 172.21.15.84:0/584204296 >> v1:172.21.15.99:6789/0 conn(0x7fcc0c158750 legacy=0x7fcc0c0cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:14.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.302+0000 7fcc0affd640 1 -- 172.21.15.84:0/584204296 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0cc850 con 0x7fcc0c0cd330 2023-10-11T06:11:14.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.303+0000 7fcc0affd640 1 --2- 172.21.15.84:0/584204296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c0ce3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0b4590 tx=0x7fcc0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58169891869bef83 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:14.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc0a7fc640 1 -- 172.21.15.84:0/584204296 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00031870 con 0x7fcc0c0cd330 2023-10-11T06:11:14.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc0a7fc640 1 -- 172.21.15.84:0/584204296 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc000319d0 con 0x7fcc0c0cd330 2023-10-11T06:11:14.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc0a7fc640 1 -- 172.21.15.84:0/584204296 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00031d60 con 0x7fcc0c0cd330 2023-10-11T06:11:14.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc10d6a640 1 -- 172.21.15.84:0/584204296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c0cd330 msgr2=0x7fcc0c0ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/584204296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c0ce3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0b4590 tx=0x7fcc0002b910 comp rx=0 tx=0).stop 2023-10-11T06:11:14.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc10d6a640 1 -- 172.21.15.84:0/584204296 shutdown_connections 2023-10-11T06:11:14.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/584204296 >> v2:172.21.15.99:3300/0 conn(0x7fcc0c071600 0x7fcc0c0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/584204296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c0ce3d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.304+0000 7fcc10d6a640 1 -- 172.21.15.84:0/584204296 >> 172.21.15.84:0/584204296 conn(0x7fcc0c069c70 msgr2=0x7fcc0c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:14.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.305+0000 7fcc10d6a640 1 -- 172.21.15.84:0/584204296 shutdown_connections 2023-10-11T06:11:14.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.305+0000 7fcc10d6a640 1 -- 172.21.15.84:0/584204296 wait complete. 2023-10-11T06:11:14.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.306+0000 7fcc10d6a640 1 Processor -- start 2023-10-11T06:11:14.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.306+0000 7fcc10d6a640 1 -- start start 2023-10-11T06:11:14.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.307+0000 7fcc10d6a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc0c071600 0x7fcc0c0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.307+0000 7fcc10d6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.307+0000 7fcc10d6a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c158750 0x7fcc0c155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.307+0000 7fcc10d6a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0ac980 con 0x7fcc0c158750 2023-10-11T06:11:14.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.307+0000 7fcc10d6a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0aed40 con 0x7fcc0c0cd330 2023-10-11T06:11:14.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.307+0000 7fcc10d6a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0c4f60 con 0x7fcc0c071600 2023-10-11T06:11:14.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c158750 0x7fcc0c155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c158750 0x7fcc0c155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37276/0 (socket says 172.21.15.84:37276) 2023-10-11T06:11:14.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 -- 172.21.15.84:0/1135537716 learned_addr learned my addr 172.21.15.84:0/1135537716 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:14.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0b7fe640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc0c071600 0x7fcc0c0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0affd640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 -- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc0c071600 msgr2=0x7fcc0c0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc0c071600 0x7fcc0c0c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 -- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc0c0cd330 msgr2=0x7fcc0c1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0bfff640 1 -- 172.21.15.84:0/1135537716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc000094d0 con 0x7fcc0c158750 2023-10-11T06:11:14.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.308+0000 7fcc0affd640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:14.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.309+0000 7fcc0bfff640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c158750 0x7fcc0c155490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc04007e70 tx=0x7fcc0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:14.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.309+0000 7fcbf3fff640 1 -- 172.21.15.84:0/1135537716 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04014620 con 0x7fcc0c158750 2023-10-11T06:11:14.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.309+0000 7fcbf3fff640 1 -- 172.21.15.84:0/1135537716 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc040147a0 con 0x7fcc0c158750 2023-10-11T06:11:14.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.309+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c0c4140 con 0x7fcc0c158750 2023-10-11T06:11:14.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.309+0000 7fcbf3fff640 1 -- 172.21.15.84:0/1135537716 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04014b50 con 0x7fcc0c158750 2023-10-11T06:11:14.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.309+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c0c43e0 con 0x7fcc0c158750 2023-10-11T06:11:14.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.310+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c0c1fe0 con 0x7fcc0c158750 2023-10-11T06:11:14.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.311+0000 7fcbf3fff640 1 -- 172.21.15.84:0/1135537716 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fcc04014cb0 con 0x7fcc0c158750 2023-10-11T06:11:14.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.312+0000 7fcbf3fff640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcbf806a830 0x7fcbf806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.312+0000 7fcbf3fff640 1 -- 172.21.15.84:0/1135537716 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc04089990 con 0x7fcc0c158750 2023-10-11T06:11:14.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.312+0000 7fcc0b7fe640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcbf806a830 0x7fcbf806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.313+0000 7fcc0b7fe640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcbf806a830 0x7fcbf806cce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc006fd0 tx=0x7fcbfc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:14.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.317+0000 7fcbf3fff640 1 -- 172.21.15.84:0/1135537716 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fcc0c0c1fe0 con 0x7fcc0c158750 2023-10-11T06:11:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.507+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc0c0c1fe0 con 0x7fcc0c158750 2023-10-11T06:11:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.509+0000 7fcbf3fff640 1 -- 172.21.15.84:0/1135537716 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17776 (secure 0 0 0) 0x7fcc0c0c1fe0 con 0x7fcc0c158750 2023-10-11T06:11:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:14.512 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:13.666386+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.511+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcbf806a830 msgr2=0x7fcbf806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.511+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcbf806a830 0x7fcbf806cce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc006fd0 tx=0x7fcbfc008040 comp rx=0 tx=0).stop 2023-10-11T06:11:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.511+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c158750 msgr2=0x7fcc0c155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.511+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c158750 0x7fcc0c155490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc04007e70 tx=0x7fcc0400e4b0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 shutdown_connections 2023-10-11T06:11:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fcbf806a830 0x7fcbf806cce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc0c158750 0x7fcc0c155490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc0c0cd330 0x7fcc0c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 --2- 172.21.15.84:0/1135537716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc0c071600 0x7fcc0c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 >> 172.21.15.84:0/1135537716 conn(0x7fcc0c069c70 msgr2=0x7fcc0c0b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:14.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 shutdown_connections 2023-10-11T06:11:14.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.512+0000 7fcc10d6a640 1 -- 172.21.15.84:0/1135537716 wait complete. 2023-10-11T06:11:14.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T06:11:14.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.746+0000 7ff9a3ef7640 1 Processor -- start 2023-10-11T06:11:14.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.746+0000 7ff9a3ef7640 1 -- start start 2023-10-11T06:11:14.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.748+0000 7ff9a3ef7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff99c06c880 0x7ff99c06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.748+0000 7ff9a3ef7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 0x7ff99c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.748+0000 7ff9a3ef7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99c0cde80 con 0x7ff99c156810 2023-10-11T06:11:14.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.748+0000 7ff9a3ef7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99c0ce140 con 0x7ff99c06c880 2023-10-11T06:11:14.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.748+0000 7ff9a3ef7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c15da30 con 0x7ff99c156be0 2023-10-11T06:11:14.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.748+0000 7ff9a1492640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff99c06c880 0x7ff99c06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.748+0000 7ff9a1492640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff99c06c880 0x7ff99c06d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37286/0 (socket says 172.21.15.84:37286) 2023-10-11T06:11:14.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.749+0000 7ff9a1492640 1 -- 172.21.15.84:0/1277993210 learned_addr learned my addr 172.21.15.84:0/1277993210 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:14.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.749+0000 7ff9a1492640 1 -- 172.21.15.84:0/1277993210 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 msgr2=0x7ff99c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:14.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.749+0000 7ff9a1492640 1 --2- 172.21.15.84:0/1277993210 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 0x7ff99c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.749+0000 7ff9a1492640 1 -- 172.21.15.84:0/1277993210 >> v1:172.21.15.99:6789/0 conn(0x7ff99c156810 legacy=0x7ff99c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:14.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.749+0000 7ff9a1492640 1 -- 172.21.15.84:0/1277993210 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c06df80 con 0x7ff99c06c880 2023-10-11T06:11:14.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.750+0000 7ff9a1492640 1 --2- 172.21.15.84:0/1277993210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c06d8e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff990009870 tx=0x7ff99002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbee904766ce1fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:14.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.750+0000 7ff9a0c91640 1 -- 172.21.15.84:0/1277993210 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900317c0 con 0x7ff99c06c880 2023-10-11T06:11:14.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.750+0000 7ff9a0c91640 1 -- 172.21.15.84:0/1277993210 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff990031920 con 0x7ff99c06c880 2023-10-11T06:11:14.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.750+0000 7ff9a0c91640 1 -- 172.21.15.84:0/1277993210 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990031cb0 con 0x7ff99c06c880 2023-10-11T06:11:14.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/1277993210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 msgr2=0x7ff99c06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/1277993210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c06d8e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff990009870 tx=0x7ff99002b860 comp rx=0 tx=0).stop 2023-10-11T06:11:14.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/1277993210 shutdown_connections 2023-10-11T06:11:14.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/1277993210 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 0x7ff99c15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/1277993210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c06d8e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/1277993210 >> 172.21.15.84:0/1277993210 conn(0x7ff99c069c70 msgr2=0x7ff99c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:14.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/1277993210 shutdown_connections 2023-10-11T06:11:14.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.751+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/1277993210 wait complete. 2023-10-11T06:11:14.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.753+0000 7ff9a3ef7640 1 Processor -- start 2023-10-11T06:11:14.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.753+0000 7ff9a3ef7640 1 -- start start 2023-10-11T06:11:14.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a3ef7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c1f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a3ef7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff99c156810 0x7ff99c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a3ef7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 0x7ff99c1f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a3ef7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c1f33f0 con 0x7ff99c06c880 2023-10-11T06:11:14.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a3ef7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c1f3650 con 0x7ff99c156810 2023-10-11T06:11:14.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a3ef7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c1f3930 con 0x7ff99c156be0 2023-10-11T06:11:14.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c1f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c1f2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37302/0 (socket says 172.21.15.84:37302) 2023-10-11T06:11:14.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 -- 172.21.15.84:0/2136917906 learned_addr learned my addr 172.21.15.84:0/2136917906 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:14.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 -- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 msgr2=0x7ff99c1f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 0x7ff99c1f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1492640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff99c156810 0x7ff99c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 -- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff99c156810 msgr2=0x7ff99c1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:14.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff99c156810 0x7ff99c1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:14.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.754+0000 7ff9a1c93640 1 -- 172.21.15.84:0/2136917906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900094d0 con 0x7ff99c06c880 2023-10-11T06:11:14.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.755+0000 7ff9a1492640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff99c156810 0x7ff99c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:14.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.755+0000 7ff9a1c93640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c1f2890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c430 tx=0x7ff99800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:14.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.755+0000 7ff98e7fc640 1 -- 172.21.15.84:0/2136917906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998011790 con 0x7ff99c06c880 2023-10-11T06:11:14.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.755+0000 7ff98e7fc640 1 -- 172.21.15.84:0/2136917906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9980118f0 con 0x7ff99c06c880 2023-10-11T06:11:14.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.755+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c1f3c10 con 0x7ff99c06c880 2023-10-11T06:11:14.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.755+0000 7ff98e7fc640 1 -- 172.21.15.84:0/2136917906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998011ce0 con 0x7ff99c06c880 2023-10-11T06:11:14.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.755+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c1f41d0 con 0x7ff99c06c880 2023-10-11T06:11:14.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.756+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c0cbd30 con 0x7ff99c06c880 2023-10-11T06:11:14.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.757+0000 7ff98e7fc640 1 -- 172.21.15.84:0/2136917906 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff99801c460 con 0x7ff99c06c880 2023-10-11T06:11:14.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.759+0000 7ff98e7fc640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff98006a830 0x7ff98006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:14.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.759+0000 7ff98e7fc640 1 -- 172.21.15.84:0/2136917906 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff998088fb0 con 0x7ff99c06c880 2023-10-11T06:11:14.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.759+0000 7ff9a1492640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff98006a830 0x7ff98006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:14.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.760+0000 7ff9a1492640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff98006a830 0x7ff98006cce0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff990036000 tx=0x7ff99003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:14.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.763+0000 7ff98e7fc640 1 -- 172.21.15.84:0/2136917906 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff998054220 con 0x7ff99c06c880 2023-10-11T06:11:14.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:14.993+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff99c073ed0 con 0x7ff99c06c880 2023-10-11T06:11:15.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.694+0000 7ff98e7fc640 1 -- 172.21.15.84:0/2136917906 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff998057840 con 0x7ff99c06c880 2023-10-11T06:11:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff98006a830 msgr2=0x7ff98006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff98006a830 0x7ff98006cce0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff990036000 tx=0x7ff99003d040 comp rx=0 tx=0).stop 2023-10-11T06:11:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 msgr2=0x7ff99c1f2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c1f2890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c430 tx=0x7ff99800c8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 shutdown_connections 2023-10-11T06:11:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff98006a830 0x7ff98006cce0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff99c156be0 0x7ff99c1f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff99c156810 0x7ff99c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 --2- 172.21.15.84:0/2136917906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff99c06c880 0x7ff99c1f2890 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.697+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 >> 172.21.15.84:0/2136917906 conn(0x7ff99c069c70 msgr2=0x7ff99c0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:15.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.698+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 shutdown_connections 2023-10-11T06:11:15.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.698+0000 7ff9a3ef7640 1 -- 172.21.15.84:0/2136917906 wait complete. 2023-10-11T06:11:15.718 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T06:11:15.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T06:11:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.928+0000 7f0ac47f6640 1 Processor -- start 2023-10-11T06:11:15.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.928+0000 7f0ac47f6640 1 -- start start 2023-10-11T06:11:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.929+0000 7f0ac47f6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 0x7f0abc0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.929+0000 7f0ac47f6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc071380 con 0x7f0abc0ced30 2023-10-11T06:11:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.929+0000 7f0ac47f6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc06de10 con 0x7f0abc0c8d00 2023-10-11T06:11:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.929+0000 7f0ac47f6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc06e0f0 con 0x7f0abc15a0b0 2023-10-11T06:11:15.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.930+0000 7f0ac2d93640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0abc0ced30 0x7f0abc06c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40606/0 (socket says 172.21.15.84:40606) 2023-10-11T06:11:15.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.930+0000 7f0ac2d93640 1 -- 172.21.15.84:0/3784924937 learned_addr learned my addr 172.21.15.84:0/3784924937 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.930+0000 7f0ac2592640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 0x7f0abc0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.930+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045849982 0 0) 0x7f0abc071380 con 0x7f0abc0ced30 2023-10-11T06:11:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.930+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa8003520 con 0x7f0abc0ced30 2023-10-11T06:11:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1896638594 0 0) 0x7f0aa8003520 con 0x7f0abc0ced30 2023-10-11T06:11:15.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 msgr2=0x7f0abc0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 0x7f0abc0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 >> v1:172.21.15.99:6789/0 conn(0x7f0abc0c8d00 legacy=0x7f0abc0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc070dc0 con 0x7f0abc0ced30 2023-10-11T06:11:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f0ab40019f0 con 0x7f0abc0ced30 2023-10-11T06:11:15.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 >> v1:172.21.15.84:6789/0 conn(0x7f0abc0ced30 legacy=0x7f0abc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 0x7f0aa80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa8003ee0 0x7f0aa80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0aa8004d00 0x7f0aa8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc06e0f0 con 0x7f0aa8004d00 2023-10-11T06:11:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa800bb10 con 0x7f0aa8003ee0 2023-10-11T06:11:15.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa800bcd0 con 0x7f0abc15a0b0 2023-10-11T06:11:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac2d93640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0aa8004d00 0x7f0aa8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac2592640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 0x7f0aa80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.931+0000 7f0ac1d91640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa8003ee0 0x7f0aa80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.932+0000 7f0ac2d93640 1 -- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 msgr2=0x7f0aa80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:15.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.932+0000 7f0ac2d93640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0abc15a0b0 0x7f0aa80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.932+0000 7f0ac2d93640 1 -- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa8003ee0 msgr2=0x7f0aa80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.932+0000 7f0ac2d93640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0aa8003ee0 0x7f0aa80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.932+0000 7f0ac2d93640 1 -- 172.21.15.84:0/3784924937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8003520 con 0x7f0aa8004d00 2023-10-11T06:11:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.933+0000 7f0ac2d93640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0aa8004d00 0x7f0aa8005170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400a840 tx=0x7f0aa80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d84d581e1b62b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:15.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.933+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab402cea0 con 0x7f0aa8004d00 2023-10-11T06:11:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.933+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab402f070 con 0x7f0aa8004d00 2023-10-11T06:11:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.933+0000 7f0ac1590640 1 -- 172.21.15.84:0/3784924937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40348b0 con 0x7f0aa8004d00 2023-10-11T06:11:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.933+0000 7f0ac47f6640 1 -- 172.21.15.84:0/3784924937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0aa8004d00 msgr2=0x7f0aa8005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.933+0000 7f0ac47f6640 1 --2- 172.21.15.84:0/3784924937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0aa8004d00 0x7f0aa8005170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400a840 tx=0x7f0aa80057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.933+0000 7f0ac2d93640 1 -- 172.21.15.84:0/3784924937 reap_dead start 2023-10-11T06:11:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.934+0000 7f0ac47f6640 1 -- 172.21.15.84:0/3784924937 shutdown_connections 2023-10-11T06:11:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.934+0000 7f0ac47f6640 1 -- 172.21.15.84:0/3784924937 >> 172.21.15.84:0/3784924937 conn(0x7f0abc069c70 msgr2=0x7f0abc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.934+0000 7f0ac47f6640 1 -- 172.21.15.84:0/3784924937 shutdown_connections 2023-10-11T06:11:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.934+0000 7f0ac47f6640 1 -- 172.21.15.84:0/3784924937 wait complete. 2023-10-11T06:11:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.935+0000 7f0ac47f6640 1 Processor -- start 2023-10-11T06:11:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.936+0000 7f0ac47f6640 1 -- start start 2023-10-11T06:11:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac47f6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0aa8004d00 0x7f0abc14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac47f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0abc1538a0 0x7f0abc1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac47f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0abc153d10 0x7f0abc152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac47f6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc14c280 con 0x7f0abc1538a0 2023-10-11T06:11:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac47f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc154280 con 0x7f0abc153d10 2023-10-11T06:11:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac47f6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc154560 con 0x7f0aa8004d00 2023-10-11T06:11:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0abc1538a0 0x7f0abc1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0abc1538a0 0x7f0abc1525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37330/0 (socket says 172.21.15.84:37330) 2023-10-11T06:11:15.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 -- 172.21.15.84:0/2516466459 learned_addr learned my addr 172.21.15.84:0/2516466459 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:15.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac2592640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0aa8004d00 0x7f0abc14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac2d93640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0abc153d10 0x7f0abc152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 -- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0aa8004d00 msgr2=0x7f0abc14bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:15.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0aa8004d00 0x7f0abc14bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 -- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0abc153d10 msgr2=0x7f0abc152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:15.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0abc153d10 0x7f0abc152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:15.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.937+0000 7f0ac1d91640 1 -- 172.21.15.84:0/2516466459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc154840 con 0x7f0abc1538a0 2023-10-11T06:11:15.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.938+0000 7f0ac1d91640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0abc1538a0 0x7f0abc1525f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00e5a0 tx=0x7f0aac00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:15.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.938+0000 7f0ab2ffd640 1 -- 172.21.15.84:0/2516466459 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0053a0 con 0x7f0abc1538a0 2023-10-11T06:11:15.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.938+0000 7f0ab2ffd640 1 -- 172.21.15.84:0/2516466459 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac0035f0 con 0x7f0abc1538a0 2023-10-11T06:11:15.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.938+0000 7f0ab2ffd640 1 -- 172.21.15.84:0/2516466459 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0039a0 con 0x7f0abc1538a0 2023-10-11T06:11:15.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.938+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc154b20 con 0x7f0abc1538a0 2023-10-11T06:11:15.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.938+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc14b050 con 0x7f0abc1538a0 2023-10-11T06:11:15.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.939+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc14b210 con 0x7f0abc1538a0 2023-10-11T06:11:15.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.940+0000 7f0ab2ffd640 1 -- 172.21.15.84:0/2516466459 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0aac003b00 con 0x7f0abc1538a0 2023-10-11T06:11:15.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.942+0000 7f0ab2ffd640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0aa006a830 0x7f0aa006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:15.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.942+0000 7f0ab2ffd640 1 -- 172.21.15.84:0/2516466459 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0aac088e10 con 0x7f0abc1538a0 2023-10-11T06:11:15.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.942+0000 7f0ac2592640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0aa006a830 0x7f0aa006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:15.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.942+0000 7f0ac2592640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0aa006a830 0x7f0aa006cce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009a10 tx=0x7f0ab8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:15.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:15.946+0000 7f0ab2ffd640 1 -- 172.21.15.84:0/2516466459 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0aac054090 con 0x7f0abc1538a0 2023-10-11T06:11:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.174+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0abc073ed0 con 0x7f0abc1538a0 2023-10-11T06:11:16.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.829+0000 7f0ab2ffd640 1 -- 172.21.15.84:0/2516466459 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0aac003db0 con 0x7f0abc1538a0 2023-10-11T06:11:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0aa006a830 msgr2=0x7f0aa006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0aa006a830 0x7f0aa006cce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009a10 tx=0x7f0ab8008040 comp rx=0 tx=0).stop 2023-10-11T06:11:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0abc1538a0 msgr2=0x7f0abc1525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0abc1538a0 0x7f0abc1525f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00e5a0 tx=0x7f0aac00ea60 comp rx=0 tx=0).stop 2023-10-11T06:11:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 shutdown_connections 2023-10-11T06:11:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0aa006a830 0x7f0aa006cce0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0abc153d10 0x7f0abc152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0abc1538a0 0x7f0abc1525f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 --2- 172.21.15.84:0/2516466459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0aa8004d00 0x7f0abc14bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 >> 172.21.15.84:0/2516466459 conn(0x7f0abc069c70 msgr2=0x7f0abc14f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:16.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 shutdown_connections 2023-10-11T06:11:16.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:16.834+0000 7f0ac47f6640 1 -- 172.21.15.84:0/2516466459 wait complete. 2023-10-11T06:11:16.858 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T06:11:16.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:11:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d2261a640 1 Processor -- start 2023-10-11T06:11:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d2261a640 1 -- start start 2023-10-11T06:11:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d2261a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d1c15d4f0 0x7f4d1c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d2261a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0c9360 0x7f4d1c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d2261a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c071a90 con 0x7f4d1c0ca300 2023-10-11T06:11:17.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d2261a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c072310 con 0x7f4d1c15d4f0 2023-10-11T06:11:17.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d2261a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c070280 con 0x7f4d1c0c9360 2023-10-11T06:11:17.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.076+0000 7f4d1bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d1c15d4f0 0x7f4d1c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.077+0000 7f4d1bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d1c15d4f0 0x7f4d1c0cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37332/0 (socket says 172.21.15.84:37332) 2023-10-11T06:11:17.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.077+0000 7f4d1bfff640 1 -- 172.21.15.84:0/4091381608 learned_addr learned my addr 172.21.15.84:0/4091381608 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:17.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.077+0000 7f4d1bfff640 1 -- 172.21.15.84:0/4091381608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0c9360 msgr2=0x7f4d1c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:17.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.077+0000 7f4d1bfff640 1 --2- 172.21.15.84:0/4091381608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0c9360 0x7f4d1c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.077+0000 7f4d1bfff640 1 -- 172.21.15.84:0/4091381608 >> v1:172.21.15.84:6789/0 conn(0x7f4d1c0ca300 legacy=0x7f4d1c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.077+0000 7f4d1bfff640 1 -- 172.21.15.84:0/4091381608 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c071cc0 con 0x7f4d1c15d4f0 2023-10-11T06:11:17.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.078+0000 7f4d1bfff640 1 --2- 172.21.15.84:0/4091381608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c15d4f0 0x7f4d1c0cb540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009ea0 tx=0x7f4d0c02bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df44d135c0f6911e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:17.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.078+0000 7f4d1affd640 1 -- 172.21.15.84:0/4091381608 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c02cb40 con 0x7f4d1c15d4f0 2023-10-11T06:11:17.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.078+0000 7f4d1affd640 1 -- 172.21.15.84:0/4091381608 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c02cca0 con 0x7f4d1c15d4f0 2023-10-11T06:11:17.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.078+0000 7f4d1affd640 1 -- 172.21.15.84:0/4091381608 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c034610 con 0x7f4d1c15d4f0 2023-10-11T06:11:17.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 -- 172.21.15.84:0/4091381608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c15d4f0 msgr2=0x7f4d1c0cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 --2- 172.21.15.84:0/4091381608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c15d4f0 0x7f4d1c0cb540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009ea0 tx=0x7f4d0c02bfa0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 -- 172.21.15.84:0/4091381608 shutdown_connections 2023-10-11T06:11:17.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 --2- 172.21.15.84:0/4091381608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0c9360 0x7f4d1c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 --2- 172.21.15.84:0/4091381608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c15d4f0 0x7f4d1c0cb540 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 -- 172.21.15.84:0/4091381608 >> 172.21.15.84:0/4091381608 conn(0x7f4d1c069c70 msgr2=0x7f4d1c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:17.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 -- 172.21.15.84:0/4091381608 shutdown_connections 2023-10-11T06:11:17.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.079+0000 7f4d2261a640 1 -- 172.21.15.84:0/4091381608 wait complete. 2023-10-11T06:11:17.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.081+0000 7f4d2261a640 1 Processor -- start 2023-10-11T06:11:17.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.081+0000 7f4d2261a640 1 -- start start 2023-10-11T06:11:17.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d2261a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0c9360 0x7f4d1c0c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d2261a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0ca300 0x7f4d1c14cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d2261a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c15d4f0 0x7f4d1c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d2261a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c89f0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d2261a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c7de0 con 0x7f4d1c15d4f0 2023-10-11T06:11:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d2261a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c80c0 con 0x7f4d1c0ca300 2023-10-11T06:11:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0c9360 0x7f4d1c0c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0c9360 0x7f4d1c0c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37346/0 (socket says 172.21.15.84:37346) 2023-10-11T06:11:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.082+0000 7f4d1bfff640 1 -- 172.21.15.84:0/146907795 learned_addr learned my addr 172.21.15.84:0/146907795 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d1bfff640 1 -- 172.21.15.84:0/146907795 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0ca300 msgr2=0x7f4d1c14cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d1bfff640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0ca300 0x7f4d1c14cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d1bfff640 1 -- 172.21.15.84:0/146907795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c15d4f0 msgr2=0x7f4d1c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d1bfff640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c15d4f0 0x7f4d1c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d1bfff640 1 -- 172.21.15.84:0/146907795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c009b30 con 0x7f4d1c0c9360 2023-10-11T06:11:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d1bfff640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0c9360 0x7f4d1c0c7700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009ea0 tx=0x7f4d0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d18ff9640 1 -- 172.21.15.84:0/146907795 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c03e450 con 0x7f4d1c0c9360 2023-10-11T06:11:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d18ff9640 1 -- 172.21.15.84:0/146907795 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c03e5f0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d18ff9640 1 -- 172.21.15.84:0/146907795 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c03e9a0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c0c83a0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.083+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c14d050 con 0x7f4d1c0c9360 2023-10-11T06:11:17.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.084+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c073ed0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.085+0000 7f4d18ff9640 1 -- 172.21.15.84:0/146907795 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4d0c03eb00 con 0x7f4d1c0c9360 2023-10-11T06:11:17.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.086+0000 7f4d18ff9640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cfc06a860 0x7f4cfc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.086+0000 7f4d18ff9640 1 -- 172.21.15.84:0/146907795 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d0c0abd00 con 0x7f4d1c0c9360 2023-10-11T06:11:17.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.087+0000 7f4d1b7fe640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cfc06a860 0x7f4cfc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.087+0000 7f4d1b7fe640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cfc06a860 0x7f4cfc06cd10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0ce320 tx=0x7f4d04007950 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:17.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.091+0000 7f4d18ff9640 1 -- 172.21.15.84:0/146907795 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4d0c076ee0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.318+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d1c0b09b0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.319+0000 7f4d18ff9640 1 -- 172.21.15.84:0/146907795 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7f4d0c03edb0 con 0x7f4d1c0c9360 2023-10-11T06:11:17.322 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:17.322 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:11:17.322 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:11:17.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cfc06a860 msgr2=0x7f4cfc06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cfc06a860 0x7f4cfc06cd10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0ce320 tx=0x7f4d04007950 comp rx=0 tx=0).stop 2023-10-11T06:11:17.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0c9360 msgr2=0x7f4d1c0c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0c9360 0x7f4d1c0c7700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009ea0 tx=0x7f4d0c004a40 comp rx=0 tx=0).stop 2023-10-11T06:11:17.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 shutdown_connections 2023-10-11T06:11:17.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cfc06a860 0x7f4cfc06cd10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c15d4f0 0x7f4d1c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d1c0ca300 0x7f4d1c14cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 --2- 172.21.15.84:0/146907795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0c9360 0x7f4d1c0c7700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 >> 172.21.15.84:0/146907795 conn(0x7f4d1c069c70 msgr2=0x7f4d1c0cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:17.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 shutdown_connections 2023-10-11T06:11:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.322+0000 7f4d2261a640 1 -- 172.21.15.84:0/146907795 wait complete. 2023-10-11T06:11:17.344 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:17.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.556+0000 7f94be5e2640 1 Processor -- start 2023-10-11T06:11:17.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.556+0000 7f94be5e2640 1 -- start start 2023-10-11T06:11:17.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94be5e2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80b1e50 0x7f94b806ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94be5e2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94b80cf210 0x7f94b80d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94be5e2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b806b7f0 con 0x7f94b80d08f0 2023-10-11T06:11:17.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94be5e2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b80d0d70 con 0x7f94b80cf210 2023-10-11T06:11:17.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94be5e2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b815c650 con 0x7f94b80b1e50 2023-10-11T06:11:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bcddf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94b80cf210 0x7f94b80d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bcddf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94b80cf210 0x7f94b80d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37354/0 (socket says 172.21.15.84:37354) 2023-10-11T06:11:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bcddf640 1 -- 172.21.15.84:0/97108268 learned_addr learned my addr 172.21.15.84:0/97108268 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bd5e0640 1 --2- 172.21.15.84:0/97108268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80b1e50 0x7f94b806ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bcddf640 1 -- 172.21.15.84:0/97108268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80b1e50 msgr2=0x7f94b806ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bcddf640 1 --2- 172.21.15.84:0/97108268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80b1e50 0x7f94b806ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bcddf640 1 -- 172.21.15.84:0/97108268 >> v1:172.21.15.84:6789/0 conn(0x7f94b80d08f0 legacy=0x7f94b80d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.558+0000 7f94bcddf640 1 -- 172.21.15.84:0/97108268 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b806bad0 con 0x7f94b80cf210 2023-10-11T06:11:17.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.559+0000 7f94bcddf640 1 --2- 172.21.15.84:0/97108268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80cf210 0x7f94b80d03b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009870 tx=0x7f94a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b34d5f546cf15c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:17.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.559+0000 7f94affff640 1 -- 172.21.15.84:0/97108268 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8030840 con 0x7f94b80cf210 2023-10-11T06:11:17.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.559+0000 7f94affff640 1 -- 172.21.15.84:0/97108268 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a80309a0 con 0x7f94b80cf210 2023-10-11T06:11:17.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.559+0000 7f94affff640 1 -- 172.21.15.84:0/97108268 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8030d30 con 0x7f94b80cf210 2023-10-11T06:11:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 -- 172.21.15.84:0/97108268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80cf210 msgr2=0x7f94b80d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 --2- 172.21.15.84:0/97108268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80cf210 0x7f94b80d03b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009870 tx=0x7f94a802b980 comp rx=0 tx=0).stop 2023-10-11T06:11:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 -- 172.21.15.84:0/97108268 shutdown_connections 2023-10-11T06:11:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 --2- 172.21.15.84:0/97108268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80cf210 0x7f94b80d03b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 --2- 172.21.15.84:0/97108268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80b1e50 0x7f94b806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 -- 172.21.15.84:0/97108268 >> 172.21.15.84:0/97108268 conn(0x7f94b808c230 msgr2=0x7f94b809bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:17.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 -- 172.21.15.84:0/97108268 shutdown_connections 2023-10-11T06:11:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.560+0000 7f94be5e2640 1 -- 172.21.15.84:0/97108268 wait complete. 2023-10-11T06:11:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.562+0000 7f94be5e2640 1 Processor -- start 2023-10-11T06:11:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.562+0000 7f94be5e2640 1 -- start start 2023-10-11T06:11:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94be5e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b80b1e50 0x7f94b815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94be5e2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80cf210 0x7f94b80cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94be5e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80d08f0 0x7f94b80cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94be5e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80b5610 con 0x7f94b80d08f0 2023-10-11T06:11:17.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94be5e2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80b2f30 con 0x7f94b80b1e50 2023-10-11T06:11:17.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94be5e2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80b3210 con 0x7f94b80cf210 2023-10-11T06:11:17.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94bdde1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80d08f0 0x7f94b80cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94bdde1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80d08f0 0x7f94b80cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37358/0 (socket says 172.21.15.84:37358) 2023-10-11T06:11:17.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94bdde1640 1 -- 172.21.15.84:0/3331750282 learned_addr learned my addr 172.21.15.84:0/3331750282 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:17.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94bd5e0640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b80b1e50 0x7f94b815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94bdde1640 1 -- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80cf210 msgr2=0x7f94b80cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.563+0000 7f94bdde1640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80cf210 0x7f94b80cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94bdde1640 1 -- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b80b1e50 msgr2=0x7f94b815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94bdde1640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b80b1e50 0x7f94b815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94bdde1640 1 -- 172.21.15.84:0/3331750282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80094d0 con 0x7f94b80d08f0 2023-10-11T06:11:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94bd5e0640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b80b1e50 0x7f94b815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94bdde1640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80d08f0 0x7f94b80cbfc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c7a0 tx=0x7f94b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94adffb640 1 -- 172.21.15.84:0/3331750282 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0003b70 con 0x7f94b80d08f0 2023-10-11T06:11:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94adffb640 1 -- 172.21.15.84:0/3331750282 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0003cf0 con 0x7f94b80d08f0 2023-10-11T06:11:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94adffb640 1 -- 172.21.15.84:0/3331750282 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0003ea0 con 0x7f94b80d08f0 2023-10-11T06:11:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b80b5860 con 0x7f94b80d08f0 2023-10-11T06:11:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.564+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b815ab10 con 0x7f94b80d08f0 2023-10-11T06:11:17.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.565+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b80b5d80 con 0x7f94b80d08f0 2023-10-11T06:11:17.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.566+0000 7f94adffb640 1 -- 172.21.15.84:0/3331750282 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f94b000aa90 con 0x7f94b80d08f0 2023-10-11T06:11:17.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.567+0000 7f94adffb640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949c06a830 0x7f949c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.567+0000 7f94adffb640 1 -- 172.21.15.84:0/3331750282 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94b0088d90 con 0x7f94b80d08f0 2023-10-11T06:11:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.568+0000 7f94bd5e0640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949c06a830 0x7f949c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.568+0000 7f94bd5e0640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949c06a830 0x7f949c06cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.571+0000 7f94adffb640 1 -- 172.21.15.84:0/3331750282 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f94b80b5d80 con 0x7f94b80d08f0 2023-10-11T06:11:17.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.761+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94b80b5d80 con 0x7f94b80d08f0 2023-10-11T06:11:17.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.763+0000 7f94adffb640 1 -- 172.21.15.84:0/3331750282 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17763 (secure 0 0 0) 0x7f94b80b5d80 con 0x7f94b80d08f0 2023-10-11T06:11:17.765 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:17.766 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:14.671090+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.765+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949c06a830 msgr2=0x7f949c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.765+0000 7f94be5e2640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949c06a830 0x7f949c06cce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4008040 comp rx=0 tx=0).stop 2023-10-11T06:11:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.765+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80d08f0 msgr2=0x7f94b80cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:17.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.765+0000 7f94be5e2640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80d08f0 0x7f94b80cbfc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c7a0 tx=0x7f94b000cc60 comp rx=0 tx=0).stop 2023-10-11T06:11:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 shutdown_connections 2023-10-11T06:11:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f949c06a830 0x7f949c06cce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94b80d08f0 0x7f94b80cbfc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94b80cf210 0x7f94b80cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 --2- 172.21.15.84:0/3331750282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94b80b1e50 0x7f94b815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:17.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 >> 172.21.15.84:0/3331750282 conn(0x7f94b808c230 msgr2=0x7f94b815bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:17.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 shutdown_connections 2023-10-11T06:11:17.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:17.766+0000 7f94be5e2640 1 -- 172.21.15.84:0/3331750282 wait complete. 2023-10-11T06:11:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T06:11:17.787 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:11:18.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.002+0000 7f227a9a4640 1 Processor -- start 2023-10-11T06:11:18.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.002+0000 7f227a9a4640 1 -- start start 2023-10-11T06:11:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f227a9a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22740b4ef0 0x7f22740aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f227a9a4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2274071d80 0x7f2274069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f227a9a4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740b2190 con 0x7f227406a1b0 2023-10-11T06:11:18.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f227a9a4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227415af60 con 0x7f2274071d80 2023-10-11T06:11:18.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f227a9a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227406a650 con 0x7f22740b4ef0 2023-10-11T06:11:18.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f2278f41640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f227406a1b0 0x7f227415dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40640/0 (socket says 172.21.15.84:40640) 2023-10-11T06:11:18.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f2278f41640 1 -- 172.21.15.84:0/3322976543 learned_addr learned my addr 172.21.15.84:0/3322976543 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:18.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f22737fe640 1 --2- 172.21.15.84:0/3322976543 >> v2:172.21.15.84:3300/0 conn(0x7f2274071d80 0x7f2274069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.003+0000 7f2273fff640 1 --2- 172.21.15.84:0/3322976543 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22740b4ef0 0x7f22740aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.004+0000 7f22737fe640 1 -- 172.21.15.84:0/3322976543 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22740b4ef0 msgr2=0x7f22740aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:18.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.004+0000 7f22737fe640 1 --2- 172.21.15.84:0/3322976543 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22740b4ef0 0x7f22740aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.004+0000 7f22737fe640 1 -- 172.21.15.84:0/3322976543 >> v1:172.21.15.84:6789/0 conn(0x7f227406a1b0 legacy=0x7f227415dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.004+0000 7f22737fe640 1 -- 172.21.15.84:0/3322976543 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740b52c0 con 0x7f2274071d80 2023-10-11T06:11:18.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.004+0000 7f2272ffd640 1 -- 172.21.15.84:0/3322976543 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901990628 0 0) 0x7f22740b2190 con 0x7f227406a1b0 2023-10-11T06:11:18.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.005+0000 7f22737fe640 1 --2- 172.21.15.84:0/3322976543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f2274069c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f225c009d90 tx=0x7f225c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6779458d77a735c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:18.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.005+0000 7f2272ffd640 1 -- 172.21.15.84:0/3322976543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c02cc50 con 0x7f2274071d80 2023-10-11T06:11:18.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.005+0000 7f2272ffd640 1 -- 172.21.15.84:0/3322976543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c02cdb0 con 0x7f2274071d80 2023-10-11T06:11:18.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.005+0000 7f2272ffd640 1 -- 172.21.15.84:0/3322976543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c034660 con 0x7f2274071d80 2023-10-11T06:11:18.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 -- 172.21.15.84:0/3322976543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 msgr2=0x7f2274069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 --2- 172.21.15.84:0/3322976543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f2274069c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f225c009d90 tx=0x7f225c02bea0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 -- 172.21.15.84:0/3322976543 shutdown_connections 2023-10-11T06:11:18.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 --2- 172.21.15.84:0/3322976543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f2274069c70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 --2- 172.21.15.84:0/3322976543 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22740b4ef0 0x7f22740aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 -- 172.21.15.84:0/3322976543 >> 172.21.15.84:0/3322976543 conn(0x7f227406b9b0 msgr2=0x7f227406d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:18.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 -- 172.21.15.84:0/3322976543 shutdown_connections 2023-10-11T06:11:18.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.006+0000 7f227a9a4640 1 -- 172.21.15.84:0/3322976543 wait complete. 2023-10-11T06:11:18.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.008+0000 7f227a9a4640 1 Processor -- start 2023-10-11T06:11:18.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.008+0000 7f227a9a4640 1 -- start start 2023-10-11T06:11:18.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f227a9a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f227406a1b0 0x7f227414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f227a9a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f22741f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f227a9a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22740b4ef0 0x7f22740710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f227a9a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741f2830 con 0x7f2274071d80 2023-10-11T06:11:18.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f227a9a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741f2270 con 0x7f22740b4ef0 2023-10-11T06:11:18.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f227a9a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22741f2550 con 0x7f227406a1b0 2023-10-11T06:11:18.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f22737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f22741f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f22737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f22741f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37370/0 (socket says 172.21.15.84:37370) 2023-10-11T06:11:18.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.009+0000 7f22737fe640 1 -- 172.21.15.84:0/2905091868 learned_addr learned my addr 172.21.15.84:0/2905091868 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:18.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f2273fff640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f227406a1b0 0x7f227414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f2278f41640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22740b4ef0 0x7f22740710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f22737fe640 1 -- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f227406a1b0 msgr2=0x7f227414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f22737fe640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f227406a1b0 0x7f227414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f22737fe640 1 -- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22740b4ef0 msgr2=0x7f22740710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f22737fe640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22740b4ef0 0x7f22740710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f22737fe640 1 -- 172.21.15.84:0/2905091868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c009a70 con 0x7f2274071d80 2023-10-11T06:11:18.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f22737fe640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f22741f1850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f225c037040 tx=0x7f225c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:18.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.010+0000 7f2270ff9640 1 -- 172.21.15.84:0/2905091868 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c03e610 con 0x7f2274071d80 2023-10-11T06:11:18.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.011+0000 7f2270ff9640 1 -- 172.21.15.84:0/2905091868 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c03e770 con 0x7f2274071d80 2023-10-11T06:11:18.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.011+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22740c42c0 con 0x7f2274071d80 2023-10-11T06:11:18.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.011+0000 7f2270ff9640 1 -- 172.21.15.84:0/2905091868 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c03eb60 con 0x7f2274071d80 2023-10-11T06:11:18.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.011+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227414f040 con 0x7f2274071d80 2023-10-11T06:11:18.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.012+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22740c4120 con 0x7f2274071d80 2023-10-11T06:11:18.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.013+0000 7f2270ff9640 1 -- 172.21.15.84:0/2905091868 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f225c03b050 con 0x7f2274071d80 2023-10-11T06:11:18.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.014+0000 7f2270ff9640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f225406a830 0x7f225406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.014+0000 7f2270ff9640 1 -- 172.21.15.84:0/2905091868 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f225c0abaf0 con 0x7f2274071d80 2023-10-11T06:11:18.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.014+0000 7f2273fff640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f225406a830 0x7f225406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.015+0000 7f2273fff640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f225406a830 0x7f225406cce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2264006fd0 tx=0x7f2264008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:18.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.018+0000 7f2270ff9640 1 -- 172.21.15.84:0/2905091868 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f22740c4120 con 0x7f2274071d80 2023-10-11T06:11:18.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.200+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f227415a8a0 con 0x7f225406a830 2023-10-11T06:11:18.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.210+0000 7f2270ff9640 1 -- 172.21.15.84:0/2905091868 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286589 (secure 0 0 0) 0x7f226400f860 con 0x7f225406a830 2023-10-11T06:11:18.215 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:18.217 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:11:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f225406a830 msgr2=0x7f225406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f225406a830 0x7f225406cce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2264006fd0 tx=0x7f2264008040 comp rx=0 tx=0).stop 2023-10-11T06:11:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 msgr2=0x7f22741f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f22741f1850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f225c037040 tx=0x7f225c004a40 comp rx=0 tx=0).stop 2023-10-11T06:11:18.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 shutdown_connections 2023-10-11T06:11:18.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f225406a830 0x7f225406cce0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22740b4ef0 0x7f22740710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2274071d80 0x7f22741f1850 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 --2- 172.21.15.84:0/2905091868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f227406a1b0 0x7f227414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 >> 172.21.15.84:0/2905091868 conn(0x7f227406b9b0 msgr2=0x7f227415c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:18.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 shutdown_connections 2023-10-11T06:11:18.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.217+0000 7f227a9a4640 1 -- 172.21.15.84:0/2905091868 wait complete. 2023-10-11T06:11:18.237 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-11T06:11:16.771800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218728,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755992,"statfs":{"total":773094113280,"available":772870135808,"internally_reserved":0,"allocated":3186688,"data_stored":1886396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.603593"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688857+0000","last_change":"2023-10-11T06:11:13.702417+0000","last_active":"2023-10-11T06:11:14.688857+0000","last_peered":"2023-10-11T06:11:14.688857+0000","last_clean":"2023-10-11T06:11:14.688857+0000","last_became_active":"2023-10-11T06:11:13.702128+0000","last_became_peered":"2023-10-11T06:11:13.702128+0000","last_unstale":"2023-10-11T06:11:14.688857+0000","last_undegraded":"2023-10-11T06:11:14.688857+0000","last_fullsized":"2023-10-11T06:11:14.688857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:26.874292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935985+0000","last_change":"2023-10-11T06:11:13.701166+0000","last_active":"2023-10-11T06:11:14.935985+0000","last_peered":"2023-10-11T06:11:14.935985+0000","last_clean":"2023-10-11T06:11:14.935985+0000","last_became_active":"2023-10-11T06:11:13.700706+0000","last_became_peered":"2023-10-11T06:11:13.700706+0000","last_unstale":"2023-10-11T06:11:14.935985+0000","last_undegraded":"2023-10-11T06:11:14.935985+0000","last_fullsized":"2023-10-11T06:11:14.935985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937001+0000","last_change":"2023-10-11T06:11:13.701254+0000","last_active":"2023-10-11T06:11:14.937001+0000","last_peered":"2023-10-11T06:11:14.937001+0000","last_clean":"2023-10-11T06:11:14.937001+0000","last_became_active":"2023-10-11T06:11:13.700986+0000","last_became_peered":"2023-10-11T06:11:13.700986+0000","last_unstale":"2023-10-11T06:11:14.937001+0000","last_undegraded":"2023-10-11T06:11:14.937001+0000","last_fullsized":"2023-10-11T06:11:14.937001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:08:07.089082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689491+0000","last_change":"2023-10-11T06:11:13.692093+0000","last_active":"2023-10-11T06:11:14.689491+0000","last_peered":"2023-10-11T06:11:14.689491+0000","last_clean":"2023-10-11T06:11:14.689491+0000","last_became_active":"2023-10-11T06:11:13.691749+0000","last_became_peered":"2023-10-11T06:11:13.691749+0000","last_unstale":"2023-10-11T06:11:14.689491+0000","last_undegraded":"2023-10-11T06:11:14.689491+0000","last_fullsized":"2023-10-11T06:11:14.689491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933130+0000","last_change":"2023-10-11T06:11:13.706178+0000","last_active":"2023-10-11T06:11:14.933130+0000","last_peered":"2023-10-11T06:11:14.933130+0000","last_clean":"2023-10-11T06:11:14.933130+0000","last_became_active":"2023-10-11T06:11:13.705917+0000","last_became_peered":"2023-10-11T06:11:13.705917+0000","last_unstale":"2023-10-11T06:11:14.933130+0000","last_undegraded":"2023-10-11T06:11:14.933130+0000","last_fullsized":"2023-10-11T06:11:14.933130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935149+0000","last_change":"2023-10-11T06:11:13.702491+0000","last_active":"2023-10-11T06:11:14.935149+0000","last_peered":"2023-10-11T06:11:14.935149+0000","last_clean":"2023-10-11T06:11:14.935149+0000","last_became_active":"2023-10-11T06:11:13.701716+0000","last_became_peered":"2023-10-11T06:11:13.701716+0000","last_unstale":"2023-10-11T06:11:14.935149+0000","last_undegraded":"2023-10-11T06:11:14.935149+0000","last_fullsized":"2023-10-11T06:11:14.935149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:16:18.652379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934429+0000","last_change":"2023-10-11T06:11:13.713375+0000","last_active":"2023-10-11T06:11:14.934429+0000","last_peered":"2023-10-11T06:11:14.934429+0000","last_clean":"2023-10-11T06:11:14.934429+0000","last_became_active":"2023-10-11T06:11:13.712800+0000","last_became_peered":"2023-10-11T06:11:13.712800+0000","last_unstale":"2023-10-11T06:11:14.934429+0000","last_undegraded":"2023-10-11T06:11:14.934429+0000","last_fullsized":"2023-10-11T06:11:14.934429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934085+0000","last_change":"2023-10-11T06:11:13.700021+0000","last_active":"2023-10-11T06:11:14.934085+0000","last_peered":"2023-10-11T06:11:14.934085+0000","last_clean":"2023-10-11T06:11:14.934085+0000","last_became_active":"2023-10-11T06:11:13.699710+0000","last_became_peered":"2023-10-11T06:11:13.699710+0000","last_unstale":"2023-10-11T06:11:14.934085+0000","last_undegraded":"2023-10-11T06:11:14.934085+0000","last_fullsized":"2023-10-11T06:11:14.934085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933410+0000","last_change":"2023-10-11T06:11:13.705628+0000","last_active":"2023-10-11T06:11:14.933410+0000","last_peered":"2023-10-11T06:11:14.933410+0000","last_clean":"2023-10-11T06:11:14.933410+0000","last_became_active":"2023-10-11T06:11:13.705205+0000","last_became_peered":"2023-10-11T06:11:13.705205+0000","last_unstale":"2023-10-11T06:11:14.933410+0000","last_undegraded":"2023-10-11T06:11:14.933410+0000","last_fullsized":"2023-10-11T06:11:14.933410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936060+0000","last_change":"2023-10-11T06:11:13.700979+0000","last_active":"2023-10-11T06:11:14.936060+0000","last_peered":"2023-10-11T06:11:14.936060+0000","last_clean":"2023-10-11T06:11:14.936060+0000","last_became_active":"2023-10-11T06:11:13.700344+0000","last_became_peered":"2023-10-11T06:11:13.700344+0000","last_unstale":"2023-10-11T06:11:14.936060+0000","last_undegraded":"2023-10-11T06:11:14.936060+0000","last_fullsized":"2023-10-11T06:11:14.936060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937234+0000","last_change":"2023-10-11T06:11:13.707695+0000","last_active":"2023-10-11T06:11:14.937234+0000","last_peered":"2023-10-11T06:11:14.937234+0000","last_clean":"2023-10-11T06:11:14.937234+0000","last_became_active":"2023-10-11T06:11:13.707358+0000","last_became_peered":"2023-10-11T06:11:13.707358+0000","last_unstale":"2023-10-11T06:11:14.937234+0000","last_undegraded":"2023-10-11T06:11:14.937234+0000","last_fullsized":"2023-10-11T06:11:14.937234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:38:53.832203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688144+0000","last_change":"2023-10-11T06:11:13.705483+0000","last_active":"2023-10-11T06:11:14.688144+0000","last_peered":"2023-10-11T06:11:14.688144+0000","last_clean":"2023-10-11T06:11:14.688144+0000","last_became_active":"2023-10-11T06:11:13.705228+0000","last_became_peered":"2023-10-11T06:11:13.705228+0000","last_unstale":"2023-10-11T06:11:14.688144+0000","last_undegraded":"2023-10-11T06:11:14.688144+0000","last_fullsized":"2023-10-11T06:11:14.688144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690422+0000","last_change":"2023-10-11T06:11:13.701448+0000","last_active":"2023-10-11T06:11:14.690422+0000","last_peered":"2023-10-11T06:11:14.690422+0000","last_clean":"2023-10-11T06:11:14.690422+0000","last_became_active":"2023-10-11T06:11:13.701186+0000","last_became_peered":"2023-10-11T06:11:13.701186+0000","last_unstale":"2023-10-11T06:11:14.690422+0000","last_undegraded":"2023-10-11T06:11:14.690422+0000","last_fullsized":"2023-10-11T06:11:14.690422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:01.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935482+0000","last_change":"2023-10-11T06:11:13.703675+0000","last_active":"2023-10-11T06:11:14.935482+0000","last_peered":"2023-10-11T06:11:14.935482+0000","last_clean":"2023-10-11T06:11:14.935482+0000","last_became_active":"2023-10-11T06:11:13.703365+0000","last_became_peered":"2023-10-11T06:11:13.703365+0000","last_unstale":"2023-10-11T06:11:14.935482+0000","last_undegraded":"2023-10-11T06:11:14.935482+0000","last_fullsized":"2023-10-11T06:11:14.935482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934721+0000","last_change":"2023-10-11T06:11:13.713483+0000","last_active":"2023-10-11T06:11:14.934721+0000","last_peered":"2023-10-11T06:11:14.934721+0000","last_clean":"2023-10-11T06:11:14.934721+0000","last_became_active":"2023-10-11T06:11:13.712594+0000","last_became_peered":"2023-10-11T06:11:13.712594+0000","last_unstale":"2023-10-11T06:11:14.934721+0000","last_undegraded":"2023-10-11T06:11:14.934721+0000","last_fullsized":"2023-10-11T06:11:14.934721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935163+0000","last_change":"2023-10-11T06:11:13.704217+0000","last_active":"2023-10-11T06:11:14.935163+0000","last_peered":"2023-10-11T06:11:14.935163+0000","last_clean":"2023-10-11T06:11:14.935163+0000","last_became_active":"2023-10-11T06:11:13.702922+0000","last_became_peered":"2023-10-11T06:11:13.702922+0000","last_unstale":"2023-10-11T06:11:14.935163+0000","last_undegraded":"2023-10-11T06:11:14.935163+0000","last_fullsized":"2023-10-11T06:11:14.935163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:10.963337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933536+0000","last_change":"2023-10-11T06:11:13.702292+0000","last_active":"2023-10-11T06:11:14.933536+0000","last_peered":"2023-10-11T06:11:14.933536+0000","last_clean":"2023-10-11T06:11:14.933536+0000","last_became_active":"2023-10-11T06:11:13.702060+0000","last_became_peered":"2023-10-11T06:11:13.702060+0000","last_unstale":"2023-10-11T06:11:14.933536+0000","last_undegraded":"2023-10-11T06:11:14.933536+0000","last_fullsized":"2023-10-11T06:11:14.933536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936185+0000","last_change":"2023-10-11T06:11:13.694348+0000","last_active":"2023-10-11T06:11:14.936185+0000","last_peered":"2023-10-11T06:11:14.936185+0000","last_clean":"2023-10-11T06:11:14.936185+0000","last_became_active":"2023-10-11T06:11:13.694074+0000","last_became_peered":"2023-10-11T06:11:13.694074+0000","last_unstale":"2023-10-11T06:11:14.936185+0000","last_undegraded":"2023-10-11T06:11:14.936185+0000","last_fullsized":"2023-10-11T06:11:14.936185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689126+0000","last_change":"2023-10-11T06:11:13.693722+0000","last_active":"2023-10-11T06:11:14.689126+0000","last_peered":"2023-10-11T06:11:14.689126+0000","last_clean":"2023-10-11T06:11:14.689126+0000","last_became_active":"2023-10-11T06:11:13.693338+0000","last_became_peered":"2023-10-11T06:11:13.693338+0000","last_unstale":"2023-10-11T06:11:14.689126+0000","last_undegraded":"2023-10-11T06:11:14.689126+0000","last_fullsized":"2023-10-11T06:11:14.689126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936628+0000","last_change":"2023-10-11T06:11:13.692755+0000","last_active":"2023-10-11T06:11:14.936628+0000","last_peered":"2023-10-11T06:11:14.936628+0000","last_clean":"2023-10-11T06:11:14.936628+0000","last_became_active":"2023-10-11T06:11:13.692467+0000","last_became_peered":"2023-10-11T06:11:13.692467+0000","last_unstale":"2023-10-11T06:11:14.936628+0000","last_undegraded":"2023-10-11T06:11:14.936628+0000","last_fullsized":"2023-10-11T06:11:14.936628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:12.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934603+0000","last_change":"2023-10-11T06:11:13.712835+0000","last_active":"2023-10-11T06:11:14.934603+0000","last_peered":"2023-10-11T06:11:14.934603+0000","last_clean":"2023-10-11T06:11:14.934603+0000","last_became_active":"2023-10-11T06:11:13.712498+0000","last_became_peered":"2023-10-11T06:11:13.712498+0000","last_unstale":"2023-10-11T06:11:14.934603+0000","last_undegraded":"2023-10-11T06:11:14.934603+0000","last_fullsized":"2023-10-11T06:11:14.934603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932549+0000","last_change":"2023-10-11T06:11:13.700139+0000","last_active":"2023-10-11T06:11:14.932549+0000","last_peered":"2023-10-11T06:11:14.932549+0000","last_clean":"2023-10-11T06:11:14.932549+0000","last_became_active":"2023-10-11T06:11:13.699888+0000","last_became_peered":"2023-10-11T06:11:13.699888+0000","last_unstale":"2023-10-11T06:11:14.932549+0000","last_undegraded":"2023-10-11T06:11:14.932549+0000","last_fullsized":"2023-10-11T06:11:14.932549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687599+0000","last_change":"2023-10-11T06:11:13.694139+0000","last_active":"2023-10-11T06:11:14.687599+0000","last_peered":"2023-10-11T06:11:14.687599+0000","last_clean":"2023-10-11T06:11:14.687599+0000","last_became_active":"2023-10-11T06:11:13.693870+0000","last_became_peered":"2023-10-11T06:11:13.693870+0000","last_unstale":"2023-10-11T06:11:14.687599+0000","last_undegraded":"2023-10-11T06:11:14.687599+0000","last_fullsized":"2023-10-11T06:11:14.687599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934975+0000","last_change":"2023-10-11T06:11:13.713679+0000","last_active":"2023-10-11T06:11:14.934975+0000","last_peered":"2023-10-11T06:11:14.934975+0000","last_clean":"2023-10-11T06:11:14.934975+0000","last_became_active":"2023-10-11T06:11:13.712476+0000","last_became_peered":"2023-10-11T06:11:13.712476+0000","last_unstale":"2023-10-11T06:11:14.934975+0000","last_undegraded":"2023-10-11T06:11:14.934975+0000","last_fullsized":"2023-10-11T06:11:14.934975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934737+0000","last_change":"2023-10-11T06:11:13.693952+0000","last_active":"2023-10-11T06:11:14.934737+0000","last_peered":"2023-10-11T06:11:14.934737+0000","last_clean":"2023-10-11T06:11:14.934737+0000","last_became_active":"2023-10-11T06:11:13.693582+0000","last_became_peered":"2023-10-11T06:11:13.693582+0000","last_unstale":"2023-10-11T06:11:14.934737+0000","last_undegraded":"2023-10-11T06:11:14.934737+0000","last_fullsized":"2023-10-11T06:11:14.934737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934890+0000","last_change":"2023-10-11T06:11:13.694225+0000","last_active":"2023-10-11T06:11:14.934890+0000","last_peered":"2023-10-11T06:11:14.934890+0000","last_clean":"2023-10-11T06:11:14.934890+0000","last_became_active":"2023-10-11T06:11:13.693965+0000","last_became_peered":"2023-10-11T06:11:13.693965+0000","last_unstale":"2023-10-11T06:11:14.934890+0000","last_undegraded":"2023-10-11T06:11:14.934890+0000","last_fullsized":"2023-10-11T06:11:14.934890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:07.237906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935270+0000","last_change":"2023-10-11T06:11:13.710202+0000","last_active":"2023-10-11T06:11:14.935270+0000","last_peered":"2023-10-11T06:11:14.935270+0000","last_clean":"2023-10-11T06:11:14.935270+0000","last_became_active":"2023-10-11T06:11:13.709963+0000","last_became_peered":"2023-10-11T06:11:13.709963+0000","last_unstale":"2023-10-11T06:11:14.935270+0000","last_undegraded":"2023-10-11T06:11:14.935270+0000","last_fullsized":"2023-10-11T06:11:14.935270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935088+0000","last_change":"2023-10-11T06:11:13.709433+0000","last_active":"2023-10-11T06:11:14.935088+0000","last_peered":"2023-10-11T06:11:14.935088+0000","last_clean":"2023-10-11T06:11:14.935088+0000","last_became_active":"2023-10-11T06:11:13.709162+0000","last_became_peered":"2023-10-11T06:11:13.709162+0000","last_unstale":"2023-10-11T06:11:14.935088+0000","last_undegraded":"2023-10-11T06:11:14.935088+0000","last_fullsized":"2023-10-11T06:11:14.935088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688453+0000","last_change":"2023-10-11T06:11:13.709986+0000","last_active":"2023-10-11T06:11:14.688453+0000","last_peered":"2023-10-11T06:11:14.688453+0000","last_clean":"2023-10-11T06:11:14.688453+0000","last_became_active":"2023-10-11T06:11:13.709545+0000","last_became_peered":"2023-10-11T06:11:13.709545+0000","last_unstale":"2023-10-11T06:11:14.688453+0000","last_undegraded":"2023-10-11T06:11:14.688453+0000","last_fullsized":"2023-10-11T06:11:14.688453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935870+0000","last_change":"2023-10-11T06:11:13.690029+0000","last_active":"2023-10-11T06:11:14.935870+0000","last_peered":"2023-10-11T06:11:14.935870+0000","last_clean":"2023-10-11T06:11:14.935870+0000","last_became_active":"2023-10-11T06:11:13.689452+0000","last_became_peered":"2023-10-11T06:11:13.689452+0000","last_unstale":"2023-10-11T06:11:14.935870+0000","last_undegraded":"2023-10-11T06:11:14.935870+0000","last_fullsized":"2023-10-11T06:11:14.935870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:44:24.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687703+0000","last_change":"2023-10-11T06:11:13.697848+0000","last_active":"2023-10-11T06:11:14.687703+0000","last_peered":"2023-10-11T06:11:14.687703+0000","last_clean":"2023-10-11T06:11:14.687703+0000","last_became_active":"2023-10-11T06:11:13.697611+0000","last_became_peered":"2023-10-11T06:11:13.697611+0000","last_unstale":"2023-10-11T06:11:14.687703+0000","last_undegraded":"2023-10-11T06:11:14.687703+0000","last_fullsized":"2023-10-11T06:11:14.687703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689328+0000","last_change":"2023-10-11T06:11:13.696436+0000","last_active":"2023-10-11T06:11:14.689328+0000","last_peered":"2023-10-11T06:11:14.689328+0000","last_clean":"2023-10-11T06:11:14.689328+0000","last_became_active":"2023-10-11T06:11:13.696110+0000","last_became_peered":"2023-10-11T06:11:13.696110+0000","last_unstale":"2023-10-11T06:11:14.689328+0000","last_undegraded":"2023-10-11T06:11:14.689328+0000","last_fullsized":"2023-10-11T06:11:14.689328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689195+0000","last_change":"2023-10-11T06:11:13.697298+0000","last_active":"2023-10-11T06:11:14.689195+0000","last_peered":"2023-10-11T06:11:14.689195+0000","last_clean":"2023-10-11T06:11:14.689195+0000","last_became_active":"2023-10-11T06:11:13.697002+0000","last_became_peered":"2023-10-11T06:11:13.697002+0000","last_unstale":"2023-10-11T06:11:14.689195+0000","last_undegraded":"2023-10-11T06:11:14.689195+0000","last_fullsized":"2023-10-11T06:11:14.689195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937247+0000","last_change":"2023-10-11T06:11:13.221800+0000","last_active":"2023-10-11T06:11:14.937247+0000","last_peered":"2023-10-11T06:11:14.937247+0000","last_clean":"2023-10-11T06:11:14.937247+0000","last_became_active":"2023-10-11T06:11:13.221375+0000","last_became_peered":"2023-10-11T06:11:13.221375+0000","last_unstale":"2023-10-11T06:11:14.937247+0000","last_undegraded":"2023-10-11T06:11:14.937247+0000","last_fullsized":"2023-10-11T06:11:14.937247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:57.513899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935532+0000","last_change":"2023-10-11T06:11:13.706371+0000","last_active":"2023-10-11T06:11:14.935532+0000","last_peered":"2023-10-11T06:11:14.935532+0000","last_clean":"2023-10-11T06:11:14.935532+0000","last_became_active":"2023-10-11T06:11:13.706038+0000","last_became_peered":"2023-10-11T06:11:13.706038+0000","last_unstale":"2023-10-11T06:11:14.935532+0000","last_undegraded":"2023-10-11T06:11:14.935532+0000","last_fullsized":"2023-10-11T06:11:14.935532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932620+0000","last_change":"2023-10-11T06:11:13.217589+0000","last_active":"2023-10-11T06:11:14.932620+0000","last_peered":"2023-10-11T06:11:14.932620+0000","last_clean":"2023-10-11T06:11:14.932620+0000","last_became_active":"2023-10-11T06:11:13.217168+0000","last_became_peered":"2023-10-11T06:11:13.217168+0000","last_unstale":"2023-10-11T06:11:14.932620+0000","last_undegraded":"2023-10-11T06:11:14.932620+0000","last_fullsized":"2023-10-11T06:11:14.932620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933735+0000","last_change":"2023-10-11T06:11:13.703040+0000","last_active":"2023-10-11T06:11:14.933735+0000","last_peered":"2023-10-11T06:11:14.933735+0000","last_clean":"2023-10-11T06:11:14.933735+0000","last_became_active":"2023-10-11T06:11:13.702687+0000","last_became_peered":"2023-10-11T06:11:13.702687+0000","last_unstale":"2023-10-11T06:11:14.933735+0000","last_undegraded":"2023-10-11T06:11:14.933735+0000","last_fullsized":"2023-10-11T06:11:14.933735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934835+0000","last_change":"2023-10-11T06:11:13.221450+0000","last_active":"2023-10-11T06:11:14.934835+0000","last_peered":"2023-10-11T06:11:14.934835+0000","last_clean":"2023-10-11T06:11:14.934835+0000","last_became_active":"2023-10-11T06:11:13.221110+0000","last_became_peered":"2023-10-11T06:11:13.221110+0000","last_unstale":"2023-10-11T06:11:14.934835+0000","last_undegraded":"2023-10-11T06:11:14.934835+0000","last_fullsized":"2023-10-11T06:11:14.934835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:48.142934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690013+0000","last_change":"2023-10-11T06:11:13.710650+0000","last_active":"2023-10-11T06:11:14.690013+0000","last_peered":"2023-10-11T06:11:14.690013+0000","last_clean":"2023-10-11T06:11:14.690013+0000","last_became_active":"2023-10-11T06:11:13.710414+0000","last_became_peered":"2023-10-11T06:11:13.710414+0000","last_unstale":"2023-10-11T06:11:14.690013+0000","last_undegraded":"2023-10-11T06:11:14.690013+0000","last_fullsized":"2023-10-11T06:11:14.690013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689695+0000","last_change":"2023-10-11T06:11:13.209784+0000","last_active":"2023-10-11T06:11:14.689695+0000","last_peered":"2023-10-11T06:11:14.689695+0000","last_clean":"2023-10-11T06:11:14.689695+0000","last_became_active":"2023-10-11T06:11:13.209455+0000","last_became_peered":"2023-10-11T06:11:13.209455+0000","last_unstale":"2023-10-11T06:11:14.689695+0000","last_undegraded":"2023-10-11T06:11:14.689695+0000","last_fullsized":"2023-10-11T06:11:14.689695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937957+0000","last_change":"2023-10-11T06:11:13.221554+0000","last_active":"2023-10-11T06:11:14.937957+0000","last_peered":"2023-10-11T06:11:14.937957+0000","last_clean":"2023-10-11T06:11:14.937957+0000","last_became_active":"2023-10-11T06:11:13.221257+0000","last_became_peered":"2023-10-11T06:11:13.221257+0000","last_unstale":"2023-10-11T06:11:14.937957+0000","last_undegraded":"2023-10-11T06:11:14.937957+0000","last_fullsized":"2023-10-11T06:11:14.937957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:43.560921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934456+0000","last_change":"2023-10-11T06:11:13.706727+0000","last_active":"2023-10-11T06:11:14.934456+0000","last_peered":"2023-10-11T06:11:14.934456+0000","last_clean":"2023-10-11T06:11:14.934456+0000","last_became_active":"2023-10-11T06:11:13.706016+0000","last_became_peered":"2023-10-11T06:11:13.706016+0000","last_unstale":"2023-10-11T06:11:14.934456+0000","last_undegraded":"2023-10-11T06:11:14.934456+0000","last_fullsized":"2023-10-11T06:11:14.934456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937828+0000","last_change":"2023-10-11T06:11:13.222531+0000","last_active":"2023-10-11T06:11:14.937828+0000","last_peered":"2023-10-11T06:11:14.937828+0000","last_clean":"2023-10-11T06:11:14.937828+0000","last_became_active":"2023-10-11T06:11:13.222126+0000","last_became_peered":"2023-10-11T06:11:13.222126+0000","last_unstale":"2023-10-11T06:11:14.937828+0000","last_undegraded":"2023-10-11T06:11:14.937828+0000","last_fullsized":"2023-10-11T06:11:14.937828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:03.840067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935577+0000","last_change":"2023-10-11T06:11:13.700622+0000","last_active":"2023-10-11T06:11:14.935577+0000","last_peered":"2023-10-11T06:11:14.935577+0000","last_clean":"2023-10-11T06:11:14.935577+0000","last_became_active":"2023-10-11T06:11:13.699582+0000","last_became_peered":"2023-10-11T06:11:13.699582+0000","last_unstale":"2023-10-11T06:11:14.935577+0000","last_undegraded":"2023-10-11T06:11:14.935577+0000","last_fullsized":"2023-10-11T06:11:14.935577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937834+0000","last_change":"2023-10-11T06:11:13.220125+0000","last_active":"2023-10-11T06:11:14.937834+0000","last_peered":"2023-10-11T06:11:14.937834+0000","last_clean":"2023-10-11T06:11:14.937834+0000","last_became_active":"2023-10-11T06:11:13.219575+0000","last_became_peered":"2023-10-11T06:11:13.219575+0000","last_unstale":"2023-10-11T06:11:14.937834+0000","last_undegraded":"2023-10-11T06:11:14.937834+0000","last_fullsized":"2023-10-11T06:11:14.937834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:06.223280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689340+0000","last_change":"2023-10-11T06:11:13.697891+0000","last_active":"2023-10-11T06:11:14.689340+0000","last_peered":"2023-10-11T06:11:14.689340+0000","last_clean":"2023-10-11T06:11:14.689340+0000","last_became_active":"2023-10-11T06:11:13.697630+0000","last_became_peered":"2023-10-11T06:11:13.697630+0000","last_unstale":"2023-10-11T06:11:14.689340+0000","last_undegraded":"2023-10-11T06:11:14.689340+0000","last_fullsized":"2023-10-11T06:11:14.689340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935004+0000","last_change":"2023-10-11T06:11:13.207215+0000","last_active":"2023-10-11T06:11:14.935004+0000","last_peered":"2023-10-11T06:11:14.935004+0000","last_clean":"2023-10-11T06:11:14.935004+0000","last_became_active":"2023-10-11T06:11:13.206865+0000","last_became_peered":"2023-10-11T06:11:13.206865+0000","last_unstale":"2023-10-11T06:11:14.935004+0000","last_undegraded":"2023-10-11T06:11:14.935004+0000","last_fullsized":"2023-10-11T06:11:14.935004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932758+0000","last_change":"2023-10-11T06:11:13.699849+0000","last_active":"2023-10-11T06:11:14.932758+0000","last_peered":"2023-10-11T06:11:14.932758+0000","last_clean":"2023-10-11T06:11:14.932758+0000","last_became_active":"2023-10-11T06:11:13.699499+0000","last_became_peered":"2023-10-11T06:11:13.699499+0000","last_unstale":"2023-10-11T06:11:14.932758+0000","last_undegraded":"2023-10-11T06:11:14.932758+0000","last_fullsized":"2023-10-11T06:11:14.932758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688855+0000","last_change":"2023-10-11T06:11:13.209002+0000","last_active":"2023-10-11T06:11:14.688855+0000","last_peered":"2023-10-11T06:11:14.688855+0000","last_clean":"2023-10-11T06:11:14.688855+0000","last_became_active":"2023-10-11T06:11:13.208587+0000","last_became_peered":"2023-10-11T06:11:13.208587+0000","last_unstale":"2023-10-11T06:11:14.688855+0000","last_undegraded":"2023-10-11T06:11:14.688855+0000","last_fullsized":"2023-10-11T06:11:14.688855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937054+0000","last_change":"2023-10-11T06:11:13.698914+0000","last_active":"2023-10-11T06:11:14.937054+0000","last_peered":"2023-10-11T06:11:14.937054+0000","last_clean":"2023-10-11T06:11:14.937054+0000","last_became_active":"2023-10-11T06:11:13.697742+0000","last_became_peered":"2023-10-11T06:11:13.697742+0000","last_unstale":"2023-10-11T06:11:14.937054+0000","last_undegraded":"2023-10-11T06:11:14.937054+0000","last_fullsized":"2023-10-11T06:11:14.937054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689617+0000","last_change":"2023-10-11T06:11:13.197306+0000","last_active":"2023-10-11T06:11:14.689617+0000","last_peered":"2023-10-11T06:11:14.689617+0000","last_clean":"2023-10-11T06:11:14.689617+0000","last_became_active":"2023-10-11T06:11:13.196154+0000","last_became_peered":"2023-10-11T06:11:13.196154+0000","last_unstale":"2023-10-11T06:11:14.689617+0000","last_undegraded":"2023-10-11T06:11:14.689617+0000","last_fullsized":"2023-10-11T06:11:14.689617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937482+0000","last_change":"2023-10-11T06:11:13.694445+0000","last_active":"2023-10-11T06:11:14.937482+0000","last_peered":"2023-10-11T06:11:14.937482+0000","last_clean":"2023-10-11T06:11:14.937482+0000","last_became_active":"2023-10-11T06:11:13.694063+0000","last_became_peered":"2023-10-11T06:11:13.694063+0000","last_unstale":"2023-10-11T06:11:14.937482+0000","last_undegraded":"2023-10-11T06:11:14.937482+0000","last_fullsized":"2023-10-11T06:11:14.937482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936267+0000","last_change":"2023-10-11T06:11:13.212919+0000","last_active":"2023-10-11T06:11:14.936267+0000","last_peered":"2023-10-11T06:11:14.936267+0000","last_clean":"2023-10-11T06:11:14.936267+0000","last_became_active":"2023-10-11T06:11:13.212064+0000","last_became_peered":"2023-10-11T06:11:13.212064+0000","last_unstale":"2023-10-11T06:11:14.936267+0000","last_undegraded":"2023-10-11T06:11:14.936267+0000","last_fullsized":"2023-10-11T06:11:14.936267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936642+0000","last_change":"2023-10-11T06:11:13.692017+0000","last_active":"2023-10-11T06:11:14.936642+0000","last_peered":"2023-10-11T06:11:14.936642+0000","last_clean":"2023-10-11T06:11:14.936642+0000","last_became_active":"2023-10-11T06:11:13.690806+0000","last_became_peered":"2023-10-11T06:11:13.690806+0000","last_unstale":"2023-10-11T06:11:14.936642+0000","last_undegraded":"2023-10-11T06:11:14.936642+0000","last_fullsized":"2023-10-11T06:11:14.936642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688241+0000","last_change":"2023-10-11T06:11:13.213340+0000","last_active":"2023-10-11T06:11:14.688241+0000","last_peered":"2023-10-11T06:11:14.688241+0000","last_clean":"2023-10-11T06:11:14.688241+0000","last_became_active":"2023-10-11T06:11:13.213031+0000","last_became_peered":"2023-10-11T06:11:13.213031+0000","last_unstale":"2023-10-11T06:11:14.688241+0000","last_undegraded":"2023-10-11T06:11:14.688241+0000","last_fullsized":"2023-10-11T06:11:14.688241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934847+0000","last_change":"2023-10-11T06:11:13.694560+0000","last_active":"2023-10-11T06:11:14.934847+0000","last_peered":"2023-10-11T06:11:14.934847+0000","last_clean":"2023-10-11T06:11:14.934847+0000","last_became_active":"2023-10-11T06:11:13.694226+0000","last_became_peered":"2023-10-11T06:11:13.694226+0000","last_unstale":"2023-10-11T06:11:14.934847+0000","last_undegraded":"2023-10-11T06:11:14.934847+0000","last_fullsized":"2023-10-11T06:11:14.934847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697572+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:11:13.697572+0000","last_peered":"2023-10-11T06:11:13.697572+0000","last_clean":"2023-10-11T06:11:13.697572+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:11:13.697572+0000","last_undegraded":"2023-10-11T06:11:13.697572+0000","last_fullsized":"2023-10-11T06:11:13.697572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.686018+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:11:13.686018+0000","last_peered":"2023-10-11T06:11:13.686018+0000","last_clean":"2023-10-11T06:11:13.686018+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:11:13.686018+0000","last_undegraded":"2023-10-11T06:11:13.686018+0000","last_fullsized":"2023-10-11T06:11:13.686018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688223+0000","last_change":"2023-10-11T06:11:13.208661+0000","last_active":"2023-10-11T06:11:14.688223+0000","last_peered":"2023-10-11T06:11:14.688223+0000","last_clean":"2023-10-11T06:11:14.688223+0000","last_became_active":"2023-10-11T06:11:13.208190+0000","last_became_peered":"2023-10-11T06:11:13.208190+0000","last_unstale":"2023-10-11T06:11:14.688223+0000","last_undegraded":"2023-10-11T06:11:14.688223+0000","last_fullsized":"2023-10-11T06:11:14.688223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689821+0000","last_change":"2023-10-11T06:11:13.695542+0000","last_active":"2023-10-11T06:11:14.689821+0000","last_peered":"2023-10-11T06:11:14.689821+0000","last_clean":"2023-10-11T06:11:14.689821+0000","last_became_active":"2023-10-11T06:11:13.695257+0000","last_became_peered":"2023-10-11T06:11:13.695257+0000","last_unstale":"2023-10-11T06:11:14.689821+0000","last_undegraded":"2023-10-11T06:11:14.689821+0000","last_fullsized":"2023-10-11T06:11:14.689821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:04.854693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697461+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:11:13.697461+0000","last_peered":"2023-10-11T06:11:13.697461+0000","last_clean":"2023-10-11T06:11:13.697461+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:11:13.697461+0000","last_undegraded":"2023-10-11T06:11:13.697461+0000","last_fullsized":"2023-10-11T06:11:13.697461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932509+0000","last_change":"2023-10-11T06:11:13.215225+0000","last_active":"2023-10-11T06:11:14.932509+0000","last_peered":"2023-10-11T06:11:14.932509+0000","last_clean":"2023-10-11T06:11:14.932509+0000","last_became_active":"2023-10-11T06:11:13.214888+0000","last_became_peered":"2023-10-11T06:11:13.214888+0000","last_unstale":"2023-10-11T06:11:14.932509+0000","last_undegraded":"2023-10-11T06:11:14.932509+0000","last_fullsized":"2023-10-11T06:11:14.932509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933557+0000","last_change":"2023-10-11T06:11:13.704745+0000","last_active":"2023-10-11T06:11:14.933557+0000","last_peered":"2023-10-11T06:11:14.933557+0000","last_clean":"2023-10-11T06:11:14.933557+0000","last_became_active":"2023-10-11T06:11:13.704504+0000","last_became_peered":"2023-10-11T06:11:13.704504+0000","last_unstale":"2023-10-11T06:11:14.933557+0000","last_undegraded":"2023-10-11T06:11:14.933557+0000","last_fullsized":"2023-10-11T06:11:14.933557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.694273+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:11:13.694273+0000","last_peered":"2023-10-11T06:11:13.694273+0000","last_clean":"2023-10-11T06:11:13.694273+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:11:13.694273+0000","last_undegraded":"2023-10-11T06:11:13.694273+0000","last_fullsized":"2023-10-11T06:11:13.694273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934122+0000","last_change":"2023-10-11T06:11:13.212259+0000","last_active":"2023-10-11T06:11:14.934122+0000","last_peered":"2023-10-11T06:11:14.934122+0000","last_clean":"2023-10-11T06:11:14.934122+0000","last_became_active":"2023-10-11T06:11:13.211994+0000","last_became_peered":"2023-10-11T06:11:13.211994+0000","last_unstale":"2023-10-11T06:11:14.934122+0000","last_undegraded":"2023-10-11T06:11:14.934122+0000","last_fullsized":"2023-10-11T06:11:14.934122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934946+0000","last_change":"2023-10-11T06:11:13.700897+0000","last_active":"2023-10-11T06:11:14.934946+0000","last_peered":"2023-10-11T06:11:14.934946+0000","last_clean":"2023-10-11T06:11:14.934946+0000","last_became_active":"2023-10-11T06:11:13.700631+0000","last_became_peered":"2023-10-11T06:11:13.700631+0000","last_unstale":"2023-10-11T06:11:14.934946+0000","last_undegraded":"2023-10-11T06:11:14.934946+0000","last_fullsized":"2023-10-11T06:11:14.934946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.688833+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:11:13.688833+0000","last_peered":"2023-10-11T06:11:13.688833+0000","last_clean":"2023-10-11T06:11:13.688833+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:11:13.688833+0000","last_undegraded":"2023-10-11T06:11:13.688833+0000","last_fullsized":"2023-10-11T06:11:13.688833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936462+0000","last_change":"2023-10-11T06:11:13.218760+0000","last_active":"2023-10-11T06:11:14.936462+0000","last_peered":"2023-10-11T06:11:14.936462+0000","last_clean":"2023-10-11T06:11:14.936462+0000","last_became_active":"2023-10-11T06:11:13.218429+0000","last_became_peered":"2023-10-11T06:11:13.218429+0000","last_unstale":"2023-10-11T06:11:14.936462+0000","last_undegraded":"2023-10-11T06:11:14.936462+0000","last_fullsized":"2023-10-11T06:11:14.936462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934619+0000","last_change":"2023-10-11T06:11:13.703570+0000","last_active":"2023-10-11T06:11:14.934619+0000","last_peered":"2023-10-11T06:11:14.934619+0000","last_clean":"2023-10-11T06:11:14.934619+0000","last_became_active":"2023-10-11T06:11:13.703305+0000","last_became_peered":"2023-10-11T06:11:13.703305+0000","last_unstale":"2023-10-11T06:11:14.934619+0000","last_undegraded":"2023-10-11T06:11:14.934619+0000","last_fullsized":"2023-10-11T06:11:14.934619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936650+0000","last_change":"2023-10-11T06:11:13.208687+0000","last_active":"2023-10-11T06:11:14.936650+0000","last_peered":"2023-10-11T06:11:14.936650+0000","last_clean":"2023-10-11T06:11:14.936650+0000","last_became_active":"2023-10-11T06:11:13.207444+0000","last_became_peered":"2023-10-11T06:11:13.207444+0000","last_unstale":"2023-10-11T06:11:14.936650+0000","last_undegraded":"2023-10-11T06:11:14.936650+0000","last_fullsized":"2023-10-11T06:11:14.936650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.677406+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:11:13.677406+0000","last_peered":"2023-10-11T06:11:13.677406+0000","last_clean":"2023-10-11T06:11:13.677406+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:11:13.677406+0000","last_undegraded":"2023-10-11T06:11:13.677406+0000","last_fullsized":"2023-10-11T06:11:13.677406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934834+0000","last_change":"2023-10-11T06:11:13.700806+0000","last_active":"2023-10-11T06:11:14.934834+0000","last_peered":"2023-10-11T06:11:14.934834+0000","last_clean":"2023-10-11T06:11:14.934834+0000","last_became_active":"2023-10-11T06:11:13.700595+0000","last_became_peered":"2023-10-11T06:11:13.700595+0000","last_unstale":"2023-10-11T06:11:14.934834+0000","last_undegraded":"2023-10-11T06:11:14.934834+0000","last_fullsized":"2023-10-11T06:11:14.934834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934850+0000","last_change":"2023-10-11T06:11:13.207586+0000","last_active":"2023-10-11T06:11:14.934850+0000","last_peered":"2023-10-11T06:11:14.934850+0000","last_clean":"2023-10-11T06:11:14.934850+0000","last_became_active":"2023-10-11T06:11:13.207303+0000","last_became_peered":"2023-10-11T06:11:13.207303+0000","last_unstale":"2023-10-11T06:11:14.934850+0000","last_undegraded":"2023-10-11T06:11:14.934850+0000","last_fullsized":"2023-10-11T06:11:14.934850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.696997+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:11:13.696997+0000","last_peered":"2023-10-11T06:11:13.696997+0000","last_clean":"2023-10-11T06:11:13.696997+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:11:13.696997+0000","last_undegraded":"2023-10-11T06:11:13.696997+0000","last_fullsized":"2023-10-11T06:11:13.696997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932868+0000","last_change":"2023-10-11T06:11:13.700313+0000","last_active":"2023-10-11T06:11:14.932868+0000","last_peered":"2023-10-11T06:11:14.932868+0000","last_clean":"2023-10-11T06:11:14.932868+0000","last_became_active":"2023-10-11T06:11:13.700001+0000","last_became_peered":"2023-10-11T06:11:13.700001+0000","last_unstale":"2023-10-11T06:11:14.932868+0000","last_undegraded":"2023-10-11T06:11:14.932868+0000","last_fullsized":"2023-10-11T06:11:14.932868+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690266+0000","last_change":"2023-10-11T06:11:13.214171+0000","last_active":"2023-10-11T06:11:14.690266+0000","last_peered":"2023-10-11T06:11:14.690266+0000","last_clean":"2023-10-11T06:11:14.690266+0000","last_became_active":"2023-10-11T06:11:13.213902+0000","last_became_peered":"2023-10-11T06:11:13.213902+0000","last_unstale":"2023-10-11T06:11:14.690266+0000","last_undegraded":"2023-10-11T06:11:14.690266+0000","last_fullsized":"2023-10-11T06:11:14.690266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.678254+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:11:13.678254+0000","last_peered":"2023-10-11T06:11:13.678254+0000","last_clean":"2023-10-11T06:11:13.678254+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:11:13.678254+0000","last_undegraded":"2023-10-11T06:11:13.678254+0000","last_fullsized":"2023-10-11T06:11:13.678254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687734+0000","last_change":"2023-10-11T06:11:13.709945+0000","last_active":"2023-10-11T06:11:14.687734+0000","last_peered":"2023-10-11T06:11:14.687734+0000","last_clean":"2023-10-11T06:11:14.687734+0000","last_became_active":"2023-10-11T06:11:13.709607+0000","last_became_peered":"2023-10-11T06:11:13.709607+0000","last_unstale":"2023-10-11T06:11:14.687734+0000","last_undegraded":"2023-10-11T06:11:14.687734+0000","last_fullsized":"2023-10-11T06:11:14.687734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933748+0000","last_change":"2023-10-11T06:11:13.213811+0000","last_active":"2023-10-11T06:11:14.933748+0000","last_peered":"2023-10-11T06:11:14.933748+0000","last_clean":"2023-10-11T06:11:14.933748+0000","last_became_active":"2023-10-11T06:11:13.213434+0000","last_became_peered":"2023-10-11T06:11:13.213434+0000","last_unstale":"2023-10-11T06:11:14.933748+0000","last_undegraded":"2023-10-11T06:11:14.933748+0000","last_fullsized":"2023-10-11T06:11:14.933748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.680148+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:11:13.680148+0000","last_peered":"2023-10-11T06:11:13.680148+0000","last_clean":"2023-10-11T06:11:13.680148+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:11:13.680148+0000","last_undegraded":"2023-10-11T06:11:13.680148+0000","last_fullsized":"2023-10-11T06:11:13.680148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936024+0000","last_change":"2023-10-11T06:11:13.700733+0000","last_active":"2023-10-11T06:11:14.936024+0000","last_peered":"2023-10-11T06:11:14.936024+0000","last_clean":"2023-10-11T06:11:14.936024+0000","last_became_active":"2023-10-11T06:11:13.700219+0000","last_became_peered":"2023-10-11T06:11:13.700219+0000","last_unstale":"2023-10-11T06:11:14.936024+0000","last_undegraded":"2023-10-11T06:11:14.936024+0000","last_fullsized":"2023-10-11T06:11:14.936024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936212+0000","last_change":"2023-10-11T06:11:13.691032+0000","last_active":"2023-10-11T06:11:14.936212+0000","last_peered":"2023-10-11T06:11:14.936212+0000","last_clean":"2023-10-11T06:11:14.936212+0000","last_became_active":"2023-10-11T06:11:13.690737+0000","last_became_peered":"2023-10-11T06:11:13.690737+0000","last_unstale":"2023-10-11T06:11:14.936212+0000","last_undegraded":"2023-10-11T06:11:14.936212+0000","last_fullsized":"2023-10-11T06:11:14.936212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.938254+0000","last_change":"2023-10-11T06:11:13.217477+0000","last_active":"2023-10-11T06:11:14.938254+0000","last_peered":"2023-10-11T06:11:14.938254+0000","last_clean":"2023-10-11T06:11:14.938254+0000","last_became_active":"2023-10-11T06:11:13.216485+0000","last_became_peered":"2023-10-11T06:11:13.216485+0000","last_unstale":"2023-10-11T06:11:14.938254+0000","last_undegraded":"2023-10-11T06:11:14.938254+0000","last_fullsized":"2023-10-11T06:11:14.938254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934689+0000","last_change":"2023-10-11T06:11:13.691615+0000","last_active":"2023-10-11T06:11:14.934689+0000","last_peered":"2023-10-11T06:11:14.934689+0000","last_clean":"2023-10-11T06:11:14.934689+0000","last_became_active":"2023-10-11T06:11:13.690979+0000","last_became_peered":"2023-10-11T06:11:13.690979+0000","last_unstale":"2023-10-11T06:11:14.934689+0000","last_undegraded":"2023-10-11T06:11:14.934689+0000","last_fullsized":"2023-10-11T06:11:14.934689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937582+0000","last_change":"2023-10-11T06:11:13.220058+0000","last_active":"2023-10-11T06:11:14.937582+0000","last_peered":"2023-10-11T06:11:14.937582+0000","last_clean":"2023-10-11T06:11:14.937582+0000","last_became_active":"2023-10-11T06:11:13.219554+0000","last_became_peered":"2023-10-11T06:11:13.219554+0000","last_unstale":"2023-10-11T06:11:14.937582+0000","last_undegraded":"2023-10-11T06:11:14.937582+0000","last_fullsized":"2023-10-11T06:11:14.937582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:48.717818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936220+0000","last_change":"2023-10-11T06:11:13.217619+0000","last_active":"2023-10-11T06:11:14.936220+0000","last_peered":"2023-10-11T06:11:14.936220+0000","last_clean":"2023-10-11T06:11:14.936220+0000","last_became_active":"2023-10-11T06:11:13.217261+0000","last_became_peered":"2023-10-11T06:11:13.217261+0000","last_unstale":"2023-10-11T06:11:14.936220+0000","last_undegraded":"2023-10-11T06:11:14.936220+0000","last_fullsized":"2023-10-11T06:11:14.936220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:34.256594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688528+0000","last_change":"2023-10-11T06:11:13.709235+0000","last_active":"2023-10-11T06:11:14.688528+0000","last_peered":"2023-10-11T06:11:14.688528+0000","last_clean":"2023-10-11T06:11:14.688528+0000","last_became_active":"2023-10-11T06:11:13.708979+0000","last_became_peered":"2023-10-11T06:11:13.708979+0000","last_unstale":"2023-10-11T06:11:14.688528+0000","last_undegraded":"2023-10-11T06:11:14.688528+0000","last_fullsized":"2023-10-11T06:11:14.688528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:24:24.143432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933598+0000","last_change":"2023-10-11T06:11:13.213756+0000","last_active":"2023-10-11T06:11:14.933598+0000","last_peered":"2023-10-11T06:11:14.933598+0000","last_clean":"2023-10-11T06:11:14.933598+0000","last_became_active":"2023-10-11T06:11:13.213456+0000","last_became_peered":"2023-10-11T06:11:13.213456+0000","last_unstale":"2023-10-11T06:11:14.933598+0000","last_undegraded":"2023-10-11T06:11:14.933598+0000","last_fullsized":"2023-10-11T06:11:14.933598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688932+0000","last_change":"2023-10-11T06:11:13.698101+0000","last_active":"2023-10-11T06:11:14.688932+0000","last_peered":"2023-10-11T06:11:14.688932+0000","last_clean":"2023-10-11T06:11:14.688932+0000","last_became_active":"2023-10-11T06:11:13.696880+0000","last_became_peered":"2023-10-11T06:11:13.696880+0000","last_unstale":"2023-10-11T06:11:14.688932+0000","last_undegraded":"2023-10-11T06:11:14.688932+0000","last_fullsized":"2023-10-11T06:11:14.688932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690335+0000","last_change":"2023-10-11T06:11:13.701258+0000","last_active":"2023-10-11T06:11:14.690335+0000","last_peered":"2023-10-11T06:11:14.690335+0000","last_clean":"2023-10-11T06:11:14.690335+0000","last_became_active":"2023-10-11T06:11:13.700995+0000","last_became_peered":"2023-10-11T06:11:13.700995+0000","last_unstale":"2023-10-11T06:11:14.690335+0000","last_undegraded":"2023-10-11T06:11:14.690335+0000","last_fullsized":"2023-10-11T06:11:14.690335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:44.575547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687997+0000","last_change":"2023-10-11T06:11:13.210038+0000","last_active":"2023-10-11T06:11:14.687997+0000","last_peered":"2023-10-11T06:11:14.687997+0000","last_clean":"2023-10-11T06:11:14.687997+0000","last_became_active":"2023-10-11T06:11:13.209702+0000","last_became_peered":"2023-10-11T06:11:13.209702+0000","last_unstale":"2023-10-11T06:11:14.687997+0000","last_undegraded":"2023-10-11T06:11:14.687997+0000","last_fullsized":"2023-10-11T06:11:14.687997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933487+0000","last_change":"2023-10-11T06:11:13.707123+0000","last_active":"2023-10-11T06:11:14.933487+0000","last_peered":"2023-10-11T06:11:14.933487+0000","last_clean":"2023-10-11T06:11:14.933487+0000","last_became_active":"2023-10-11T06:11:13.706856+0000","last_became_peered":"2023-10-11T06:11:13.706856+0000","last_unstale":"2023-10-11T06:11:14.933487+0000","last_undegraded":"2023-10-11T06:11:14.933487+0000","last_fullsized":"2023-10-11T06:11:14.933487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934110+0000","last_change":"2023-10-11T06:11:13.206009+0000","last_active":"2023-10-11T06:11:14.934110+0000","last_peered":"2023-10-11T06:11:14.934110+0000","last_clean":"2023-10-11T06:11:14.934110+0000","last_became_active":"2023-10-11T06:11:13.205671+0000","last_became_peered":"2023-10-11T06:11:13.205671+0000","last_unstale":"2023-10-11T06:11:14.934110+0000","last_undegraded":"2023-10-11T06:11:14.934110+0000","last_fullsized":"2023-10-11T06:11:14.934110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688523+0000","last_change":"2023-10-11T06:11:13.697631+0000","last_active":"2023-10-11T06:11:14.688523+0000","last_peered":"2023-10-11T06:11:14.688523+0000","last_clean":"2023-10-11T06:11:14.688523+0000","last_became_active":"2023-10-11T06:11:13.696974+0000","last_became_peered":"2023-10-11T06:11:13.696974+0000","last_unstale":"2023-10-11T06:11:14.688523+0000","last_undegraded":"2023-10-11T06:11:14.688523+0000","last_fullsized":"2023-10-11T06:11:14.688523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936125+0000","last_change":"2023-10-11T06:11:13.221421+0000","last_active":"2023-10-11T06:11:14.936125+0000","last_peered":"2023-10-11T06:11:14.936125+0000","last_clean":"2023-10-11T06:11:14.936125+0000","last_became_active":"2023-10-11T06:11:13.221114+0000","last_became_peered":"2023-10-11T06:11:13.221114+0000","last_unstale":"2023-10-11T06:11:14.936125+0000","last_undegraded":"2023-10-11T06:11:14.936125+0000","last_fullsized":"2023-10-11T06:11:14.936125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688237+0000","last_change":"2023-10-11T06:11:13.699277+0000","last_active":"2023-10-11T06:11:14.688237+0000","last_peered":"2023-10-11T06:11:14.688237+0000","last_clean":"2023-10-11T06:11:14.688237+0000","last_became_active":"2023-10-11T06:11:13.699042+0000","last_became_peered":"2023-10-11T06:11:13.699042+0000","last_unstale":"2023-10-11T06:11:14.688237+0000","last_undegraded":"2023-10-11T06:11:14.688237+0000","last_fullsized":"2023-10-11T06:11:14.688237+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933247+0000","last_change":"2023-10-11T06:11:13.218429+0000","last_active":"2023-10-11T06:11:14.933247+0000","last_peered":"2023-10-11T06:11:14.933247+0000","last_clean":"2023-10-11T06:11:14.933247+0000","last_became_active":"2023-10-11T06:11:13.218150+0000","last_became_peered":"2023-10-11T06:11:13.218150+0000","last_unstale":"2023-10-11T06:11:14.933247+0000","last_undegraded":"2023-10-11T06:11:14.933247+0000","last_fullsized":"2023-10-11T06:11:14.933247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935952+0000","last_change":"2023-10-11T06:11:13.700494+0000","last_active":"2023-10-11T06:11:14.935952+0000","last_peered":"2023-10-11T06:11:14.935952+0000","last_clean":"2023-10-11T06:11:14.935952+0000","last_became_active":"2023-10-11T06:11:13.700182+0000","last_became_peered":"2023-10-11T06:11:13.700182+0000","last_unstale":"2023-10-11T06:11:14.935952+0000","last_undegraded":"2023-10-11T06:11:14.935952+0000","last_fullsized":"2023-10-11T06:11:14.935952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934742+0000","last_change":"2023-10-11T06:11:13.221465+0000","last_active":"2023-10-11T06:11:14.934742+0000","last_peered":"2023-10-11T06:11:14.934742+0000","last_clean":"2023-10-11T06:11:14.934742+0000","last_became_active":"2023-10-11T06:11:13.221160+0000","last_became_peered":"2023-10-11T06:11:13.221160+0000","last_unstale":"2023-10-11T06:11:14.934742+0000","last_undegraded":"2023-10-11T06:11:14.934742+0000","last_fullsized":"2023-10-11T06:11:14.934742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935425+0000","last_change":"2023-10-11T06:11:13.690241+0000","last_active":"2023-10-11T06:11:14.935425+0000","last_peered":"2023-10-11T06:11:14.935425+0000","last_clean":"2023-10-11T06:11:14.935425+0000","last_became_active":"2023-10-11T06:11:13.689766+0000","last_became_peered":"2023-10-11T06:11:13.689766+0000","last_unstale":"2023-10-11T06:11:14.935425+0000","last_undegraded":"2023-10-11T06:11:14.935425+0000","last_fullsized":"2023-10-11T06:11:14.935425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:46:46.784397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936532+0000","last_change":"2023-10-11T06:11:13.212684+0000","last_active":"2023-10-11T06:11:14.936532+0000","last_peered":"2023-10-11T06:11:14.936532+0000","last_clean":"2023-10-11T06:11:14.936532+0000","last_became_active":"2023-10-11T06:11:13.212366+0000","last_became_peered":"2023-10-11T06:11:13.212366+0000","last_unstale":"2023-10-11T06:11:14.936532+0000","last_undegraded":"2023-10-11T06:11:14.936532+0000","last_fullsized":"2023-10-11T06:11:14.936532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688632+0000","last_change":"2023-10-11T06:11:13.706859+0000","last_active":"2023-10-11T06:11:14.688632+0000","last_peered":"2023-10-11T06:11:14.688632+0000","last_clean":"2023-10-11T06:11:14.688632+0000","last_became_active":"2023-10-11T06:11:13.706584+0000","last_became_peered":"2023-10-11T06:11:13.706584+0000","last_unstale":"2023-10-11T06:11:14.688632+0000","last_undegraded":"2023-10-11T06:11:14.688632+0000","last_fullsized":"2023-10-11T06:11:14.688632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688832+0000","last_change":"2023-10-11T06:11:13.200955+0000","last_active":"2023-10-11T06:11:14.688832+0000","last_peered":"2023-10-11T06:11:14.688832+0000","last_clean":"2023-10-11T06:11:14.688832+0000","last_became_active":"2023-10-11T06:11:13.200617+0000","last_became_peered":"2023-10-11T06:11:13.200617+0000","last_unstale":"2023-10-11T06:11:14.688832+0000","last_undegraded":"2023-10-11T06:11:14.688832+0000","last_fullsized":"2023-10-11T06:11:14.688832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935118+0000","last_change":"2023-10-11T06:11:13.689555+0000","last_active":"2023-10-11T06:11:14.935118+0000","last_peered":"2023-10-11T06:11:14.935118+0000","last_clean":"2023-10-11T06:11:14.935118+0000","last_became_active":"2023-10-11T06:11:13.689252+0000","last_became_peered":"2023-10-11T06:11:13.689252+0000","last_unstale":"2023-10-11T06:11:14.935118+0000","last_undegraded":"2023-10-11T06:11:14.935118+0000","last_fullsized":"2023-10-11T06:11:14.935118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:49.157560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933504+0000","last_change":"2023-10-11T06:11:13.213633+0000","last_active":"2023-10-11T06:11:14.933504+0000","last_peered":"2023-10-11T06:11:14.933504+0000","last_clean":"2023-10-11T06:11:14.933504+0000","last_became_active":"2023-10-11T06:11:13.213082+0000","last_became_peered":"2023-10-11T06:11:13.213082+0000","last_unstale":"2023-10-11T06:11:14.933504+0000","last_undegraded":"2023-10-11T06:11:14.933504+0000","last_fullsized":"2023-10-11T06:11:14.933504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":679765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.681}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":88269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.732}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":87707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7170000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":678567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.73}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:11:18.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:11:18.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.491+0000 7f087f9e4640 1 Processor -- start 2023-10-11T06:11:18.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.492+0000 7f087f9e4640 1 -- start start 2023-10-11T06:11:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087f9e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f08780d0c20 0x7f08780af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087f9e4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08780d25e0 0x7f08780b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087f9e4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08780d2a40 con 0x7f087815c290 2023-10-11T06:11:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087f9e4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087806cd10 con 0x7f08780d25e0 2023-10-11T06:11:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087f9e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08780afb60 con 0x7f08780d0c20 2023-10-11T06:11:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087d780640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f087815c290 0x7f08780d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40652/0 (socket says 172.21.15.84:40652) 2023-10-11T06:11:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087d780640 1 -- 172.21.15.84:0/1154405696 learned_addr learned my addr 172.21.15.84:0/1154405696 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:18.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.493+0000 7f087df81640 1 --2- 172.21.15.84:0/1154405696 >> v2:172.21.15.84:3300/0 conn(0x7f08780d25e0 0x7f08780b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.494+0000 7f087df81640 1 -- 172.21.15.84:0/1154405696 >> v2:172.21.15.99:3300/0 conn(0x7f08780d0c20 msgr2=0x7f08780af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.494+0000 7f087cf7f640 1 --2- 172.21.15.84:0/1154405696 >> v2:172.21.15.99:3300/0 conn(0x7f08780d0c20 0x7f08780af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.494+0000 7f087df81640 1 --2- 172.21.15.84:0/1154405696 >> v2:172.21.15.99:3300/0 conn(0x7f08780d0c20 0x7f08780af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.494+0000 7f087df81640 1 -- 172.21.15.84:0/1154405696 >> v1:172.21.15.84:6789/0 conn(0x7f087815c290 legacy=0x7f08780d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.494+0000 7f087df81640 1 -- 172.21.15.84:0/1154405696 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780afe40 con 0x7f08780d25e0 2023-10-11T06:11:18.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.494+0000 7f0877fff640 1 -- 172.21.15.84:0/1154405696 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877125589 0 0) 0x7f08780d2a40 con 0x7f087815c290 2023-10-11T06:11:18.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.494+0000 7f087cf7f640 1 --2- 172.21.15.84:0/1154405696 >> v2:172.21.15.99:3300/0 conn(0x7f08780d0c20 0x7f08780af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:18.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.495+0000 7f087df81640 1 --2- 172.21.15.84:0/1154405696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f08780b0c00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f08780d2120 tx=0x7f087002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288513d54f9e676e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:18.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.495+0000 7f0877fff640 1 -- 172.21.15.84:0/1154405696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087002e070 con 0x7f08780d25e0 2023-10-11T06:11:18.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.495+0000 7f0877fff640 1 -- 172.21.15.84:0/1154405696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08700315a0 con 0x7f08780d25e0 2023-10-11T06:11:18.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.495+0000 7f0877fff640 1 -- 172.21.15.84:0/1154405696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870031930 con 0x7f08780d25e0 2023-10-11T06:11:18.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 -- 172.21.15.84:0/1154405696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 msgr2=0x7f08780b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1154405696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f08780b0c00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f08780d2120 tx=0x7f087002b840 comp rx=0 tx=0).stop 2023-10-11T06:11:18.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 -- 172.21.15.84:0/1154405696 shutdown_connections 2023-10-11T06:11:18.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1154405696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f08780b0c00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1154405696 >> v2:172.21.15.99:3300/0 conn(0x7f08780d0c20 0x7f08780af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 -- 172.21.15.84:0/1154405696 >> 172.21.15.84:0/1154405696 conn(0x7f0878071cc0 msgr2=0x7f087809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:18.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 -- 172.21.15.84:0/1154405696 shutdown_connections 2023-10-11T06:11:18.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.496+0000 7f087f9e4640 1 -- 172.21.15.84:0/1154405696 wait complete. 2023-10-11T06:11:18.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.498+0000 7f087f9e4640 1 Processor -- start 2023-10-11T06:11:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.498+0000 7f087f9e4640 1 -- start start 2023-10-11T06:11:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.499+0000 7f087f9e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08780d0c20 0x7f08781f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.499+0000 7f087f9e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f087814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.499+0000 7f087f9e4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f087815c290 0x7f087814ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087f9e4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781f5be0 con 0x7f08780d25e0 2023-10-11T06:11:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087f9e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781f5340 con 0x7f08780d0c20 2023-10-11T06:11:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087f9e4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08781f5620 con 0x7f087815c290 2023-10-11T06:11:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f087814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f087814e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37384/0 (socket says 172.21.15.84:37384) 2023-10-11T06:11:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 -- 172.21.15.84:0/1295030596 learned_addr learned my addr 172.21.15.84:0/1295030596 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087d780640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08780d0c20 0x7f08781f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087df81640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f087815c290 0x7f087814ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 -- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f087815c290 msgr2=0x7f087814ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f087815c290 0x7f087814ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 -- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08780d0c20 msgr2=0x7f08781f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08780d0c20 0x7f08781f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 -- 172.21.15.84:0/1295030596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0870009400 con 0x7f08780d25e0 2023-10-11T06:11:18.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.500+0000 7f087cf7f640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f087814e190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f086400c7f0 tx=0x7f086400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:18.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.501+0000 7f0875ffb640 1 -- 172.21.15.84:0/1295030596 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864007a30 con 0x7f08780d25e0 2023-10-11T06:11:18.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.501+0000 7f0875ffb640 1 -- 172.21.15.84:0/1295030596 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0864007b90 con 0x7f08780d25e0 2023-10-11T06:11:18.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.501+0000 7f0875ffb640 1 -- 172.21.15.84:0/1295030596 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086400ce40 con 0x7f08780d25e0 2023-10-11T06:11:18.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.501+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08781f5900 con 0x7f08780d25e0 2023-10-11T06:11:18.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.501+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087814e6d0 con 0x7f08780d25e0 2023-10-11T06:11:18.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.502+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087814deb0 con 0x7f08780d25e0 2023-10-11T06:11:18.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.503+0000 7f0875ffb640 1 -- 172.21.15.84:0/1295030596 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f08640113f0 con 0x7f08780d25e0 2023-10-11T06:11:18.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.504+0000 7f0875ffb640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085c06a860 0x7f085c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.505+0000 7f0875ffb640 1 -- 172.21.15.84:0/1295030596 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08640891f0 con 0x7f08780d25e0 2023-10-11T06:11:18.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.505+0000 7f087d780640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085c06a860 0x7f085c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.505+0000 7f087d780640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085c06a860 0x7f085c06cd10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f086c007ed0 tx=0x7f086c007e60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:18.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.509+0000 7f0875ffb640 1 -- 172.21.15.84:0/1295030596 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f087814deb0 con 0x7f08780d25e0 2023-10-11T06:11:18.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.692+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08781f5790 con 0x7f085c06a860 2023-10-11T06:11:18.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.701+0000 7f0875ffb640 1 -- 172.21.15.84:0/1295030596 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286589 (secure 0 0 0) 0x7f08781f5790 con 0x7f085c06a860 2023-10-11T06:11:18.706 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:18.708 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:11:18.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085c06a860 msgr2=0x7f085c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085c06a860 0x7f085c06cd10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f086c007ed0 tx=0x7f086c007e60 comp rx=0 tx=0).stop 2023-10-11T06:11:18.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 msgr2=0x7f087814e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f087814e190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f086400c7f0 tx=0x7f086400cbf0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 shutdown_connections 2023-10-11T06:11:18.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085c06a860 0x7f085c06cd10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f087815c290 0x7f087814ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08780d25e0 0x7f087814e190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 --2- 172.21.15.84:0/1295030596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08780d0c20 0x7f08781f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 >> 172.21.15.84:0/1295030596 conn(0x7f0878071cc0 msgr2=0x7f08780b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:18.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.707+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 shutdown_connections 2023-10-11T06:11:18.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.708+0000 7f087f9e4640 1 -- 172.21.15.84:0/1295030596 wait complete. 2023-10-11T06:11:18.726 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-11T06:11:16.771800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218728,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755992,"statfs":{"total":773094113280,"available":772870135808,"internally_reserved":0,"allocated":3186688,"data_stored":1886396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.603593"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688857+0000","last_change":"2023-10-11T06:11:13.702417+0000","last_active":"2023-10-11T06:11:14.688857+0000","last_peered":"2023-10-11T06:11:14.688857+0000","last_clean":"2023-10-11T06:11:14.688857+0000","last_became_active":"2023-10-11T06:11:13.702128+0000","last_became_peered":"2023-10-11T06:11:13.702128+0000","last_unstale":"2023-10-11T06:11:14.688857+0000","last_undegraded":"2023-10-11T06:11:14.688857+0000","last_fullsized":"2023-10-11T06:11:14.688857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:26.874292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935985+0000","last_change":"2023-10-11T06:11:13.701166+0000","last_active":"2023-10-11T06:11:14.935985+0000","last_peered":"2023-10-11T06:11:14.935985+0000","last_clean":"2023-10-11T06:11:14.935985+0000","last_became_active":"2023-10-11T06:11:13.700706+0000","last_became_peered":"2023-10-11T06:11:13.700706+0000","last_unstale":"2023-10-11T06:11:14.935985+0000","last_undegraded":"2023-10-11T06:11:14.935985+0000","last_fullsized":"2023-10-11T06:11:14.935985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937001+0000","last_change":"2023-10-11T06:11:13.701254+0000","last_active":"2023-10-11T06:11:14.937001+0000","last_peered":"2023-10-11T06:11:14.937001+0000","last_clean":"2023-10-11T06:11:14.937001+0000","last_became_active":"2023-10-11T06:11:13.700986+0000","last_became_peered":"2023-10-11T06:11:13.700986+0000","last_unstale":"2023-10-11T06:11:14.937001+0000","last_undegraded":"2023-10-11T06:11:14.937001+0000","last_fullsized":"2023-10-11T06:11:14.937001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:08:07.089082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689491+0000","last_change":"2023-10-11T06:11:13.692093+0000","last_active":"2023-10-11T06:11:14.689491+0000","last_peered":"2023-10-11T06:11:14.689491+0000","last_clean":"2023-10-11T06:11:14.689491+0000","last_became_active":"2023-10-11T06:11:13.691749+0000","last_became_peered":"2023-10-11T06:11:13.691749+0000","last_unstale":"2023-10-11T06:11:14.689491+0000","last_undegraded":"2023-10-11T06:11:14.689491+0000","last_fullsized":"2023-10-11T06:11:14.689491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933130+0000","last_change":"2023-10-11T06:11:13.706178+0000","last_active":"2023-10-11T06:11:14.933130+0000","last_peered":"2023-10-11T06:11:14.933130+0000","last_clean":"2023-10-11T06:11:14.933130+0000","last_became_active":"2023-10-11T06:11:13.705917+0000","last_became_peered":"2023-10-11T06:11:13.705917+0000","last_unstale":"2023-10-11T06:11:14.933130+0000","last_undegraded":"2023-10-11T06:11:14.933130+0000","last_fullsized":"2023-10-11T06:11:14.933130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935149+0000","last_change":"2023-10-11T06:11:13.702491+0000","last_active":"2023-10-11T06:11:14.935149+0000","last_peered":"2023-10-11T06:11:14.935149+0000","last_clean":"2023-10-11T06:11:14.935149+0000","last_became_active":"2023-10-11T06:11:13.701716+0000","last_became_peered":"2023-10-11T06:11:13.701716+0000","last_unstale":"2023-10-11T06:11:14.935149+0000","last_undegraded":"2023-10-11T06:11:14.935149+0000","last_fullsized":"2023-10-11T06:11:14.935149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:16:18.652379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934429+0000","last_change":"2023-10-11T06:11:13.713375+0000","last_active":"2023-10-11T06:11:14.934429+0000","last_peered":"2023-10-11T06:11:14.934429+0000","last_clean":"2023-10-11T06:11:14.934429+0000","last_became_active":"2023-10-11T06:11:13.712800+0000","last_became_peered":"2023-10-11T06:11:13.712800+0000","last_unstale":"2023-10-11T06:11:14.934429+0000","last_undegraded":"2023-10-11T06:11:14.934429+0000","last_fullsized":"2023-10-11T06:11:14.934429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934085+0000","last_change":"2023-10-11T06:11:13.700021+0000","last_active":"2023-10-11T06:11:14.934085+0000","last_peered":"2023-10-11T06:11:14.934085+0000","last_clean":"2023-10-11T06:11:14.934085+0000","last_became_active":"2023-10-11T06:11:13.699710+0000","last_became_peered":"2023-10-11T06:11:13.699710+0000","last_unstale":"2023-10-11T06:11:14.934085+0000","last_undegraded":"2023-10-11T06:11:14.934085+0000","last_fullsized":"2023-10-11T06:11:14.934085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933410+0000","last_change":"2023-10-11T06:11:13.705628+0000","last_active":"2023-10-11T06:11:14.933410+0000","last_peered":"2023-10-11T06:11:14.933410+0000","last_clean":"2023-10-11T06:11:14.933410+0000","last_became_active":"2023-10-11T06:11:13.705205+0000","last_became_peered":"2023-10-11T06:11:13.705205+0000","last_unstale":"2023-10-11T06:11:14.933410+0000","last_undegraded":"2023-10-11T06:11:14.933410+0000","last_fullsized":"2023-10-11T06:11:14.933410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936060+0000","last_change":"2023-10-11T06:11:13.700979+0000","last_active":"2023-10-11T06:11:14.936060+0000","last_peered":"2023-10-11T06:11:14.936060+0000","last_clean":"2023-10-11T06:11:14.936060+0000","last_became_active":"2023-10-11T06:11:13.700344+0000","last_became_peered":"2023-10-11T06:11:13.700344+0000","last_unstale":"2023-10-11T06:11:14.936060+0000","last_undegraded":"2023-10-11T06:11:14.936060+0000","last_fullsized":"2023-10-11T06:11:14.936060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937234+0000","last_change":"2023-10-11T06:11:13.707695+0000","last_active":"2023-10-11T06:11:14.937234+0000","last_peered":"2023-10-11T06:11:14.937234+0000","last_clean":"2023-10-11T06:11:14.937234+0000","last_became_active":"2023-10-11T06:11:13.707358+0000","last_became_peered":"2023-10-11T06:11:13.707358+0000","last_unstale":"2023-10-11T06:11:14.937234+0000","last_undegraded":"2023-10-11T06:11:14.937234+0000","last_fullsized":"2023-10-11T06:11:14.937234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:38:53.832203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688144+0000","last_change":"2023-10-11T06:11:13.705483+0000","last_active":"2023-10-11T06:11:14.688144+0000","last_peered":"2023-10-11T06:11:14.688144+0000","last_clean":"2023-10-11T06:11:14.688144+0000","last_became_active":"2023-10-11T06:11:13.705228+0000","last_became_peered":"2023-10-11T06:11:13.705228+0000","last_unstale":"2023-10-11T06:11:14.688144+0000","last_undegraded":"2023-10-11T06:11:14.688144+0000","last_fullsized":"2023-10-11T06:11:14.688144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690422+0000","last_change":"2023-10-11T06:11:13.701448+0000","last_active":"2023-10-11T06:11:14.690422+0000","last_peered":"2023-10-11T06:11:14.690422+0000","last_clean":"2023-10-11T06:11:14.690422+0000","last_became_active":"2023-10-11T06:11:13.701186+0000","last_became_peered":"2023-10-11T06:11:13.701186+0000","last_unstale":"2023-10-11T06:11:14.690422+0000","last_undegraded":"2023-10-11T06:11:14.690422+0000","last_fullsized":"2023-10-11T06:11:14.690422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:01.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935482+0000","last_change":"2023-10-11T06:11:13.703675+0000","last_active":"2023-10-11T06:11:14.935482+0000","last_peered":"2023-10-11T06:11:14.935482+0000","last_clean":"2023-10-11T06:11:14.935482+0000","last_became_active":"2023-10-11T06:11:13.703365+0000","last_became_peered":"2023-10-11T06:11:13.703365+0000","last_unstale":"2023-10-11T06:11:14.935482+0000","last_undegraded":"2023-10-11T06:11:14.935482+0000","last_fullsized":"2023-10-11T06:11:14.935482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934721+0000","last_change":"2023-10-11T06:11:13.713483+0000","last_active":"2023-10-11T06:11:14.934721+0000","last_peered":"2023-10-11T06:11:14.934721+0000","last_clean":"2023-10-11T06:11:14.934721+0000","last_became_active":"2023-10-11T06:11:13.712594+0000","last_became_peered":"2023-10-11T06:11:13.712594+0000","last_unstale":"2023-10-11T06:11:14.934721+0000","last_undegraded":"2023-10-11T06:11:14.934721+0000","last_fullsized":"2023-10-11T06:11:14.934721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935163+0000","last_change":"2023-10-11T06:11:13.704217+0000","last_active":"2023-10-11T06:11:14.935163+0000","last_peered":"2023-10-11T06:11:14.935163+0000","last_clean":"2023-10-11T06:11:14.935163+0000","last_became_active":"2023-10-11T06:11:13.702922+0000","last_became_peered":"2023-10-11T06:11:13.702922+0000","last_unstale":"2023-10-11T06:11:14.935163+0000","last_undegraded":"2023-10-11T06:11:14.935163+0000","last_fullsized":"2023-10-11T06:11:14.935163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:10.963337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933536+0000","last_change":"2023-10-11T06:11:13.702292+0000","last_active":"2023-10-11T06:11:14.933536+0000","last_peered":"2023-10-11T06:11:14.933536+0000","last_clean":"2023-10-11T06:11:14.933536+0000","last_became_active":"2023-10-11T06:11:13.702060+0000","last_became_peered":"2023-10-11T06:11:13.702060+0000","last_unstale":"2023-10-11T06:11:14.933536+0000","last_undegraded":"2023-10-11T06:11:14.933536+0000","last_fullsized":"2023-10-11T06:11:14.933536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936185+0000","last_change":"2023-10-11T06:11:13.694348+0000","last_active":"2023-10-11T06:11:14.936185+0000","last_peered":"2023-10-11T06:11:14.936185+0000","last_clean":"2023-10-11T06:11:14.936185+0000","last_became_active":"2023-10-11T06:11:13.694074+0000","last_became_peered":"2023-10-11T06:11:13.694074+0000","last_unstale":"2023-10-11T06:11:14.936185+0000","last_undegraded":"2023-10-11T06:11:14.936185+0000","last_fullsized":"2023-10-11T06:11:14.936185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689126+0000","last_change":"2023-10-11T06:11:13.693722+0000","last_active":"2023-10-11T06:11:14.689126+0000","last_peered":"2023-10-11T06:11:14.689126+0000","last_clean":"2023-10-11T06:11:14.689126+0000","last_became_active":"2023-10-11T06:11:13.693338+0000","last_became_peered":"2023-10-11T06:11:13.693338+0000","last_unstale":"2023-10-11T06:11:14.689126+0000","last_undegraded":"2023-10-11T06:11:14.689126+0000","last_fullsized":"2023-10-11T06:11:14.689126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936628+0000","last_change":"2023-10-11T06:11:13.692755+0000","last_active":"2023-10-11T06:11:14.936628+0000","last_peered":"2023-10-11T06:11:14.936628+0000","last_clean":"2023-10-11T06:11:14.936628+0000","last_became_active":"2023-10-11T06:11:13.692467+0000","last_became_peered":"2023-10-11T06:11:13.692467+0000","last_unstale":"2023-10-11T06:11:14.936628+0000","last_undegraded":"2023-10-11T06:11:14.936628+0000","last_fullsized":"2023-10-11T06:11:14.936628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:12.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934603+0000","last_change":"2023-10-11T06:11:13.712835+0000","last_active":"2023-10-11T06:11:14.934603+0000","last_peered":"2023-10-11T06:11:14.934603+0000","last_clean":"2023-10-11T06:11:14.934603+0000","last_became_active":"2023-10-11T06:11:13.712498+0000","last_became_peered":"2023-10-11T06:11:13.712498+0000","last_unstale":"2023-10-11T06:11:14.934603+0000","last_undegraded":"2023-10-11T06:11:14.934603+0000","last_fullsized":"2023-10-11T06:11:14.934603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932549+0000","last_change":"2023-10-11T06:11:13.700139+0000","last_active":"2023-10-11T06:11:14.932549+0000","last_peered":"2023-10-11T06:11:14.932549+0000","last_clean":"2023-10-11T06:11:14.932549+0000","last_became_active":"2023-10-11T06:11:13.699888+0000","last_became_peered":"2023-10-11T06:11:13.699888+0000","last_unstale":"2023-10-11T06:11:14.932549+0000","last_undegraded":"2023-10-11T06:11:14.932549+0000","last_fullsized":"2023-10-11T06:11:14.932549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687599+0000","last_change":"2023-10-11T06:11:13.694139+0000","last_active":"2023-10-11T06:11:14.687599+0000","last_peered":"2023-10-11T06:11:14.687599+0000","last_clean":"2023-10-11T06:11:14.687599+0000","last_became_active":"2023-10-11T06:11:13.693870+0000","last_became_peered":"2023-10-11T06:11:13.693870+0000","last_unstale":"2023-10-11T06:11:14.687599+0000","last_undegraded":"2023-10-11T06:11:14.687599+0000","last_fullsized":"2023-10-11T06:11:14.687599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934975+0000","last_change":"2023-10-11T06:11:13.713679+0000","last_active":"2023-10-11T06:11:14.934975+0000","last_peered":"2023-10-11T06:11:14.934975+0000","last_clean":"2023-10-11T06:11:14.934975+0000","last_became_active":"2023-10-11T06:11:13.712476+0000","last_became_peered":"2023-10-11T06:11:13.712476+0000","last_unstale":"2023-10-11T06:11:14.934975+0000","last_undegraded":"2023-10-11T06:11:14.934975+0000","last_fullsized":"2023-10-11T06:11:14.934975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934737+0000","last_change":"2023-10-11T06:11:13.693952+0000","last_active":"2023-10-11T06:11:14.934737+0000","last_peered":"2023-10-11T06:11:14.934737+0000","last_clean":"2023-10-11T06:11:14.934737+0000","last_became_active":"2023-10-11T06:11:13.693582+0000","last_became_peered":"2023-10-11T06:11:13.693582+0000","last_unstale":"2023-10-11T06:11:14.934737+0000","last_undegraded":"2023-10-11T06:11:14.934737+0000","last_fullsized":"2023-10-11T06:11:14.934737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934890+0000","last_change":"2023-10-11T06:11:13.694225+0000","last_active":"2023-10-11T06:11:14.934890+0000","last_peered":"2023-10-11T06:11:14.934890+0000","last_clean":"2023-10-11T06:11:14.934890+0000","last_became_active":"2023-10-11T06:11:13.693965+0000","last_became_peered":"2023-10-11T06:11:13.693965+0000","last_unstale":"2023-10-11T06:11:14.934890+0000","last_undegraded":"2023-10-11T06:11:14.934890+0000","last_fullsized":"2023-10-11T06:11:14.934890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:07.237906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935270+0000","last_change":"2023-10-11T06:11:13.710202+0000","last_active":"2023-10-11T06:11:14.935270+0000","last_peered":"2023-10-11T06:11:14.935270+0000","last_clean":"2023-10-11T06:11:14.935270+0000","last_became_active":"2023-10-11T06:11:13.709963+0000","last_became_peered":"2023-10-11T06:11:13.709963+0000","last_unstale":"2023-10-11T06:11:14.935270+0000","last_undegraded":"2023-10-11T06:11:14.935270+0000","last_fullsized":"2023-10-11T06:11:14.935270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935088+0000","last_change":"2023-10-11T06:11:13.709433+0000","last_active":"2023-10-11T06:11:14.935088+0000","last_peered":"2023-10-11T06:11:14.935088+0000","last_clean":"2023-10-11T06:11:14.935088+0000","last_became_active":"2023-10-11T06:11:13.709162+0000","last_became_peered":"2023-10-11T06:11:13.709162+0000","last_unstale":"2023-10-11T06:11:14.935088+0000","last_undegraded":"2023-10-11T06:11:14.935088+0000","last_fullsized":"2023-10-11T06:11:14.935088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688453+0000","last_change":"2023-10-11T06:11:13.709986+0000","last_active":"2023-10-11T06:11:14.688453+0000","last_peered":"2023-10-11T06:11:14.688453+0000","last_clean":"2023-10-11T06:11:14.688453+0000","last_became_active":"2023-10-11T06:11:13.709545+0000","last_became_peered":"2023-10-11T06:11:13.709545+0000","last_unstale":"2023-10-11T06:11:14.688453+0000","last_undegraded":"2023-10-11T06:11:14.688453+0000","last_fullsized":"2023-10-11T06:11:14.688453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935870+0000","last_change":"2023-10-11T06:11:13.690029+0000","last_active":"2023-10-11T06:11:14.935870+0000","last_peered":"2023-10-11T06:11:14.935870+0000","last_clean":"2023-10-11T06:11:14.935870+0000","last_became_active":"2023-10-11T06:11:13.689452+0000","last_became_peered":"2023-10-11T06:11:13.689452+0000","last_unstale":"2023-10-11T06:11:14.935870+0000","last_undegraded":"2023-10-11T06:11:14.935870+0000","last_fullsized":"2023-10-11T06:11:14.935870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:44:24.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687703+0000","last_change":"2023-10-11T06:11:13.697848+0000","last_active":"2023-10-11T06:11:14.687703+0000","last_peered":"2023-10-11T06:11:14.687703+0000","last_clean":"2023-10-11T06:11:14.687703+0000","last_became_active":"2023-10-11T06:11:13.697611+0000","last_became_peered":"2023-10-11T06:11:13.697611+0000","last_unstale":"2023-10-11T06:11:14.687703+0000","last_undegraded":"2023-10-11T06:11:14.687703+0000","last_fullsized":"2023-10-11T06:11:14.687703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689328+0000","last_change":"2023-10-11T06:11:13.696436+0000","last_active":"2023-10-11T06:11:14.689328+0000","last_peered":"2023-10-11T06:11:14.689328+0000","last_clean":"2023-10-11T06:11:14.689328+0000","last_became_active":"2023-10-11T06:11:13.696110+0000","last_became_peered":"2023-10-11T06:11:13.696110+0000","last_unstale":"2023-10-11T06:11:14.689328+0000","last_undegraded":"2023-10-11T06:11:14.689328+0000","last_fullsized":"2023-10-11T06:11:14.689328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689195+0000","last_change":"2023-10-11T06:11:13.697298+0000","last_active":"2023-10-11T06:11:14.689195+0000","last_peered":"2023-10-11T06:11:14.689195+0000","last_clean":"2023-10-11T06:11:14.689195+0000","last_became_active":"2023-10-11T06:11:13.697002+0000","last_became_peered":"2023-10-11T06:11:13.697002+0000","last_unstale":"2023-10-11T06:11:14.689195+0000","last_undegraded":"2023-10-11T06:11:14.689195+0000","last_fullsized":"2023-10-11T06:11:14.689195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937247+0000","last_change":"2023-10-11T06:11:13.221800+0000","last_active":"2023-10-11T06:11:14.937247+0000","last_peered":"2023-10-11T06:11:14.937247+0000","last_clean":"2023-10-11T06:11:14.937247+0000","last_became_active":"2023-10-11T06:11:13.221375+0000","last_became_peered":"2023-10-11T06:11:13.221375+0000","last_unstale":"2023-10-11T06:11:14.937247+0000","last_undegraded":"2023-10-11T06:11:14.937247+0000","last_fullsized":"2023-10-11T06:11:14.937247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:57.513899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935532+0000","last_change":"2023-10-11T06:11:13.706371+0000","last_active":"2023-10-11T06:11:14.935532+0000","last_peered":"2023-10-11T06:11:14.935532+0000","last_clean":"2023-10-11T06:11:14.935532+0000","last_became_active":"2023-10-11T06:11:13.706038+0000","last_became_peered":"2023-10-11T06:11:13.706038+0000","last_unstale":"2023-10-11T06:11:14.935532+0000","last_undegraded":"2023-10-11T06:11:14.935532+0000","last_fullsized":"2023-10-11T06:11:14.935532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932620+0000","last_change":"2023-10-11T06:11:13.217589+0000","last_active":"2023-10-11T06:11:14.932620+0000","last_peered":"2023-10-11T06:11:14.932620+0000","last_clean":"2023-10-11T06:11:14.932620+0000","last_became_active":"2023-10-11T06:11:13.217168+0000","last_became_peered":"2023-10-11T06:11:13.217168+0000","last_unstale":"2023-10-11T06:11:14.932620+0000","last_undegraded":"2023-10-11T06:11:14.932620+0000","last_fullsized":"2023-10-11T06:11:14.932620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933735+0000","last_change":"2023-10-11T06:11:13.703040+0000","last_active":"2023-10-11T06:11:14.933735+0000","last_peered":"2023-10-11T06:11:14.933735+0000","last_clean":"2023-10-11T06:11:14.933735+0000","last_became_active":"2023-10-11T06:11:13.702687+0000","last_became_peered":"2023-10-11T06:11:13.702687+0000","last_unstale":"2023-10-11T06:11:14.933735+0000","last_undegraded":"2023-10-11T06:11:14.933735+0000","last_fullsized":"2023-10-11T06:11:14.933735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934835+0000","last_change":"2023-10-11T06:11:13.221450+0000","last_active":"2023-10-11T06:11:14.934835+0000","last_peered":"2023-10-11T06:11:14.934835+0000","last_clean":"2023-10-11T06:11:14.934835+0000","last_became_active":"2023-10-11T06:11:13.221110+0000","last_became_peered":"2023-10-11T06:11:13.221110+0000","last_unstale":"2023-10-11T06:11:14.934835+0000","last_undegraded":"2023-10-11T06:11:14.934835+0000","last_fullsized":"2023-10-11T06:11:14.934835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:48.142934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690013+0000","last_change":"2023-10-11T06:11:13.710650+0000","last_active":"2023-10-11T06:11:14.690013+0000","last_peered":"2023-10-11T06:11:14.690013+0000","last_clean":"2023-10-11T06:11:14.690013+0000","last_became_active":"2023-10-11T06:11:13.710414+0000","last_became_peered":"2023-10-11T06:11:13.710414+0000","last_unstale":"2023-10-11T06:11:14.690013+0000","last_undegraded":"2023-10-11T06:11:14.690013+0000","last_fullsized":"2023-10-11T06:11:14.690013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689695+0000","last_change":"2023-10-11T06:11:13.209784+0000","last_active":"2023-10-11T06:11:14.689695+0000","last_peered":"2023-10-11T06:11:14.689695+0000","last_clean":"2023-10-11T06:11:14.689695+0000","last_became_active":"2023-10-11T06:11:13.209455+0000","last_became_peered":"2023-10-11T06:11:13.209455+0000","last_unstale":"2023-10-11T06:11:14.689695+0000","last_undegraded":"2023-10-11T06:11:14.689695+0000","last_fullsized":"2023-10-11T06:11:14.689695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937957+0000","last_change":"2023-10-11T06:11:13.221554+0000","last_active":"2023-10-11T06:11:14.937957+0000","last_peered":"2023-10-11T06:11:14.937957+0000","last_clean":"2023-10-11T06:11:14.937957+0000","last_became_active":"2023-10-11T06:11:13.221257+0000","last_became_peered":"2023-10-11T06:11:13.221257+0000","last_unstale":"2023-10-11T06:11:14.937957+0000","last_undegraded":"2023-10-11T06:11:14.937957+0000","last_fullsized":"2023-10-11T06:11:14.937957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:43.560921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934456+0000","last_change":"2023-10-11T06:11:13.706727+0000","last_active":"2023-10-11T06:11:14.934456+0000","last_peered":"2023-10-11T06:11:14.934456+0000","last_clean":"2023-10-11T06:11:14.934456+0000","last_became_active":"2023-10-11T06:11:13.706016+0000","last_became_peered":"2023-10-11T06:11:13.706016+0000","last_unstale":"2023-10-11T06:11:14.934456+0000","last_undegraded":"2023-10-11T06:11:14.934456+0000","last_fullsized":"2023-10-11T06:11:14.934456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937828+0000","last_change":"2023-10-11T06:11:13.222531+0000","last_active":"2023-10-11T06:11:14.937828+0000","last_peered":"2023-10-11T06:11:14.937828+0000","last_clean":"2023-10-11T06:11:14.937828+0000","last_became_active":"2023-10-11T06:11:13.222126+0000","last_became_peered":"2023-10-11T06:11:13.222126+0000","last_unstale":"2023-10-11T06:11:14.937828+0000","last_undegraded":"2023-10-11T06:11:14.937828+0000","last_fullsized":"2023-10-11T06:11:14.937828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:03.840067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935577+0000","last_change":"2023-10-11T06:11:13.700622+0000","last_active":"2023-10-11T06:11:14.935577+0000","last_peered":"2023-10-11T06:11:14.935577+0000","last_clean":"2023-10-11T06:11:14.935577+0000","last_became_active":"2023-10-11T06:11:13.699582+0000","last_became_peered":"2023-10-11T06:11:13.699582+0000","last_unstale":"2023-10-11T06:11:14.935577+0000","last_undegraded":"2023-10-11T06:11:14.935577+0000","last_fullsized":"2023-10-11T06:11:14.935577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937834+0000","last_change":"2023-10-11T06:11:13.220125+0000","last_active":"2023-10-11T06:11:14.937834+0000","last_peered":"2023-10-11T06:11:14.937834+0000","last_clean":"2023-10-11T06:11:14.937834+0000","last_became_active":"2023-10-11T06:11:13.219575+0000","last_became_peered":"2023-10-11T06:11:13.219575+0000","last_unstale":"2023-10-11T06:11:14.937834+0000","last_undegraded":"2023-10-11T06:11:14.937834+0000","last_fullsized":"2023-10-11T06:11:14.937834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:06.223280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689340+0000","last_change":"2023-10-11T06:11:13.697891+0000","last_active":"2023-10-11T06:11:14.689340+0000","last_peered":"2023-10-11T06:11:14.689340+0000","last_clean":"2023-10-11T06:11:14.689340+0000","last_became_active":"2023-10-11T06:11:13.697630+0000","last_became_peered":"2023-10-11T06:11:13.697630+0000","last_unstale":"2023-10-11T06:11:14.689340+0000","last_undegraded":"2023-10-11T06:11:14.689340+0000","last_fullsized":"2023-10-11T06:11:14.689340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935004+0000","last_change":"2023-10-11T06:11:13.207215+0000","last_active":"2023-10-11T06:11:14.935004+0000","last_peered":"2023-10-11T06:11:14.935004+0000","last_clean":"2023-10-11T06:11:14.935004+0000","last_became_active":"2023-10-11T06:11:13.206865+0000","last_became_peered":"2023-10-11T06:11:13.206865+0000","last_unstale":"2023-10-11T06:11:14.935004+0000","last_undegraded":"2023-10-11T06:11:14.935004+0000","last_fullsized":"2023-10-11T06:11:14.935004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932758+0000","last_change":"2023-10-11T06:11:13.699849+0000","last_active":"2023-10-11T06:11:14.932758+0000","last_peered":"2023-10-11T06:11:14.932758+0000","last_clean":"2023-10-11T06:11:14.932758+0000","last_became_active":"2023-10-11T06:11:13.699499+0000","last_became_peered":"2023-10-11T06:11:13.699499+0000","last_unstale":"2023-10-11T06:11:14.932758+0000","last_undegraded":"2023-10-11T06:11:14.932758+0000","last_fullsized":"2023-10-11T06:11:14.932758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688855+0000","last_change":"2023-10-11T06:11:13.209002+0000","last_active":"2023-10-11T06:11:14.688855+0000","last_peered":"2023-10-11T06:11:14.688855+0000","last_clean":"2023-10-11T06:11:14.688855+0000","last_became_active":"2023-10-11T06:11:13.208587+0000","last_became_peered":"2023-10-11T06:11:13.208587+0000","last_unstale":"2023-10-11T06:11:14.688855+0000","last_undegraded":"2023-10-11T06:11:14.688855+0000","last_fullsized":"2023-10-11T06:11:14.688855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937054+0000","last_change":"2023-10-11T06:11:13.698914+0000","last_active":"2023-10-11T06:11:14.937054+0000","last_peered":"2023-10-11T06:11:14.937054+0000","last_clean":"2023-10-11T06:11:14.937054+0000","last_became_active":"2023-10-11T06:11:13.697742+0000","last_became_peered":"2023-10-11T06:11:13.697742+0000","last_unstale":"2023-10-11T06:11:14.937054+0000","last_undegraded":"2023-10-11T06:11:14.937054+0000","last_fullsized":"2023-10-11T06:11:14.937054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689617+0000","last_change":"2023-10-11T06:11:13.197306+0000","last_active":"2023-10-11T06:11:14.689617+0000","last_peered":"2023-10-11T06:11:14.689617+0000","last_clean":"2023-10-11T06:11:14.689617+0000","last_became_active":"2023-10-11T06:11:13.196154+0000","last_became_peered":"2023-10-11T06:11:13.196154+0000","last_unstale":"2023-10-11T06:11:14.689617+0000","last_undegraded":"2023-10-11T06:11:14.689617+0000","last_fullsized":"2023-10-11T06:11:14.689617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937482+0000","last_change":"2023-10-11T06:11:13.694445+0000","last_active":"2023-10-11T06:11:14.937482+0000","last_peered":"2023-10-11T06:11:14.937482+0000","last_clean":"2023-10-11T06:11:14.937482+0000","last_became_active":"2023-10-11T06:11:13.694063+0000","last_became_peered":"2023-10-11T06:11:13.694063+0000","last_unstale":"2023-10-11T06:11:14.937482+0000","last_undegraded":"2023-10-11T06:11:14.937482+0000","last_fullsized":"2023-10-11T06:11:14.937482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936267+0000","last_change":"2023-10-11T06:11:13.212919+0000","last_active":"2023-10-11T06:11:14.936267+0000","last_peered":"2023-10-11T06:11:14.936267+0000","last_clean":"2023-10-11T06:11:14.936267+0000","last_became_active":"2023-10-11T06:11:13.212064+0000","last_became_peered":"2023-10-11T06:11:13.212064+0000","last_unstale":"2023-10-11T06:11:14.936267+0000","last_undegraded":"2023-10-11T06:11:14.936267+0000","last_fullsized":"2023-10-11T06:11:14.936267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936642+0000","last_change":"2023-10-11T06:11:13.692017+0000","last_active":"2023-10-11T06:11:14.936642+0000","last_peered":"2023-10-11T06:11:14.936642+0000","last_clean":"2023-10-11T06:11:14.936642+0000","last_became_active":"2023-10-11T06:11:13.690806+0000","last_became_peered":"2023-10-11T06:11:13.690806+0000","last_unstale":"2023-10-11T06:11:14.936642+0000","last_undegraded":"2023-10-11T06:11:14.936642+0000","last_fullsized":"2023-10-11T06:11:14.936642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688241+0000","last_change":"2023-10-11T06:11:13.213340+0000","last_active":"2023-10-11T06:11:14.688241+0000","last_peered":"2023-10-11T06:11:14.688241+0000","last_clean":"2023-10-11T06:11:14.688241+0000","last_became_active":"2023-10-11T06:11:13.213031+0000","last_became_peered":"2023-10-11T06:11:13.213031+0000","last_unstale":"2023-10-11T06:11:14.688241+0000","last_undegraded":"2023-10-11T06:11:14.688241+0000","last_fullsized":"2023-10-11T06:11:14.688241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934847+0000","last_change":"2023-10-11T06:11:13.694560+0000","last_active":"2023-10-11T06:11:14.934847+0000","last_peered":"2023-10-11T06:11:14.934847+0000","last_clean":"2023-10-11T06:11:14.934847+0000","last_became_active":"2023-10-11T06:11:13.694226+0000","last_became_peered":"2023-10-11T06:11:13.694226+0000","last_unstale":"2023-10-11T06:11:14.934847+0000","last_undegraded":"2023-10-11T06:11:14.934847+0000","last_fullsized":"2023-10-11T06:11:14.934847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697572+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:11:13.697572+0000","last_peered":"2023-10-11T06:11:13.697572+0000","last_clean":"2023-10-11T06:11:13.697572+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:11:13.697572+0000","last_undegraded":"2023-10-11T06:11:13.697572+0000","last_fullsized":"2023-10-11T06:11:13.697572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.686018+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:11:13.686018+0000","last_peered":"2023-10-11T06:11:13.686018+0000","last_clean":"2023-10-11T06:11:13.686018+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:11:13.686018+0000","last_undegraded":"2023-10-11T06:11:13.686018+0000","last_fullsized":"2023-10-11T06:11:13.686018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688223+0000","last_change":"2023-10-11T06:11:13.208661+0000","last_active":"2023-10-11T06:11:14.688223+0000","last_peered":"2023-10-11T06:11:14.688223+0000","last_clean":"2023-10-11T06:11:14.688223+0000","last_became_active":"2023-10-11T06:11:13.208190+0000","last_became_peered":"2023-10-11T06:11:13.208190+0000","last_unstale":"2023-10-11T06:11:14.688223+0000","last_undegraded":"2023-10-11T06:11:14.688223+0000","last_fullsized":"2023-10-11T06:11:14.688223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689821+0000","last_change":"2023-10-11T06:11:13.695542+0000","last_active":"2023-10-11T06:11:14.689821+0000","last_peered":"2023-10-11T06:11:14.689821+0000","last_clean":"2023-10-11T06:11:14.689821+0000","last_became_active":"2023-10-11T06:11:13.695257+0000","last_became_peered":"2023-10-11T06:11:13.695257+0000","last_unstale":"2023-10-11T06:11:14.689821+0000","last_undegraded":"2023-10-11T06:11:14.689821+0000","last_fullsized":"2023-10-11T06:11:14.689821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:04.854693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697461+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:11:13.697461+0000","last_peered":"2023-10-11T06:11:13.697461+0000","last_clean":"2023-10-11T06:11:13.697461+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:11:13.697461+0000","last_undegraded":"2023-10-11T06:11:13.697461+0000","last_fullsized":"2023-10-11T06:11:13.697461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932509+0000","last_change":"2023-10-11T06:11:13.215225+0000","last_active":"2023-10-11T06:11:14.932509+0000","last_peered":"2023-10-11T06:11:14.932509+0000","last_clean":"2023-10-11T06:11:14.932509+0000","last_became_active":"2023-10-11T06:11:13.214888+0000","last_became_peered":"2023-10-11T06:11:13.214888+0000","last_unstale":"2023-10-11T06:11:14.932509+0000","last_undegraded":"2023-10-11T06:11:14.932509+0000","last_fullsized":"2023-10-11T06:11:14.932509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933557+0000","last_change":"2023-10-11T06:11:13.704745+0000","last_active":"2023-10-11T06:11:14.933557+0000","last_peered":"2023-10-11T06:11:14.933557+0000","last_clean":"2023-10-11T06:11:14.933557+0000","last_became_active":"2023-10-11T06:11:13.704504+0000","last_became_peered":"2023-10-11T06:11:13.704504+0000","last_unstale":"2023-10-11T06:11:14.933557+0000","last_undegraded":"2023-10-11T06:11:14.933557+0000","last_fullsized":"2023-10-11T06:11:14.933557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.694273+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:11:13.694273+0000","last_peered":"2023-10-11T06:11:13.694273+0000","last_clean":"2023-10-11T06:11:13.694273+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:11:13.694273+0000","last_undegraded":"2023-10-11T06:11:13.694273+0000","last_fullsized":"2023-10-11T06:11:13.694273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934122+0000","last_change":"2023-10-11T06:11:13.212259+0000","last_active":"2023-10-11T06:11:14.934122+0000","last_peered":"2023-10-11T06:11:14.934122+0000","last_clean":"2023-10-11T06:11:14.934122+0000","last_became_active":"2023-10-11T06:11:13.211994+0000","last_became_peered":"2023-10-11T06:11:13.211994+0000","last_unstale":"2023-10-11T06:11:14.934122+0000","last_undegraded":"2023-10-11T06:11:14.934122+0000","last_fullsized":"2023-10-11T06:11:14.934122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934946+0000","last_change":"2023-10-11T06:11:13.700897+0000","last_active":"2023-10-11T06:11:14.934946+0000","last_peered":"2023-10-11T06:11:14.934946+0000","last_clean":"2023-10-11T06:11:14.934946+0000","last_became_active":"2023-10-11T06:11:13.700631+0000","last_became_peered":"2023-10-11T06:11:13.700631+0000","last_unstale":"2023-10-11T06:11:14.934946+0000","last_undegraded":"2023-10-11T06:11:14.934946+0000","last_fullsized":"2023-10-11T06:11:14.934946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.688833+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:11:13.688833+0000","last_peered":"2023-10-11T06:11:13.688833+0000","last_clean":"2023-10-11T06:11:13.688833+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:11:13.688833+0000","last_undegraded":"2023-10-11T06:11:13.688833+0000","last_fullsized":"2023-10-11T06:11:13.688833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936462+0000","last_change":"2023-10-11T06:11:13.218760+0000","last_active":"2023-10-11T06:11:14.936462+0000","last_peered":"2023-10-11T06:11:14.936462+0000","last_clean":"2023-10-11T06:11:14.936462+0000","last_became_active":"2023-10-11T06:11:13.218429+0000","last_became_peered":"2023-10-11T06:11:13.218429+0000","last_unstale":"2023-10-11T06:11:14.936462+0000","last_undegraded":"2023-10-11T06:11:14.936462+0000","last_fullsized":"2023-10-11T06:11:14.936462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934619+0000","last_change":"2023-10-11T06:11:13.703570+0000","last_active":"2023-10-11T06:11:14.934619+0000","last_peered":"2023-10-11T06:11:14.934619+0000","last_clean":"2023-10-11T06:11:14.934619+0000","last_became_active":"2023-10-11T06:11:13.703305+0000","last_became_peered":"2023-10-11T06:11:13.703305+0000","last_unstale":"2023-10-11T06:11:14.934619+0000","last_undegraded":"2023-10-11T06:11:14.934619+0000","last_fullsized":"2023-10-11T06:11:14.934619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936650+0000","last_change":"2023-10-11T06:11:13.208687+0000","last_active":"2023-10-11T06:11:14.936650+0000","last_peered":"2023-10-11T06:11:14.936650+0000","last_clean":"2023-10-11T06:11:14.936650+0000","last_became_active":"2023-10-11T06:11:13.207444+0000","last_became_peered":"2023-10-11T06:11:13.207444+0000","last_unstale":"2023-10-11T06:11:14.936650+0000","last_undegraded":"2023-10-11T06:11:14.936650+0000","last_fullsized":"2023-10-11T06:11:14.936650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.677406+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:11:13.677406+0000","last_peered":"2023-10-11T06:11:13.677406+0000","last_clean":"2023-10-11T06:11:13.677406+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:11:13.677406+0000","last_undegraded":"2023-10-11T06:11:13.677406+0000","last_fullsized":"2023-10-11T06:11:13.677406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934834+0000","last_change":"2023-10-11T06:11:13.700806+0000","last_active":"2023-10-11T06:11:14.934834+0000","last_peered":"2023-10-11T06:11:14.934834+0000","last_clean":"2023-10-11T06:11:14.934834+0000","last_became_active":"2023-10-11T06:11:13.700595+0000","last_became_peered":"2023-10-11T06:11:13.700595+0000","last_unstale":"2023-10-11T06:11:14.934834+0000","last_undegraded":"2023-10-11T06:11:14.934834+0000","last_fullsized":"2023-10-11T06:11:14.934834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934850+0000","last_change":"2023-10-11T06:11:13.207586+0000","last_active":"2023-10-11T06:11:14.934850+0000","last_peered":"2023-10-11T06:11:14.934850+0000","last_clean":"2023-10-11T06:11:14.934850+0000","last_became_active":"2023-10-11T06:11:13.207303+0000","last_became_peered":"2023-10-11T06:11:13.207303+0000","last_unstale":"2023-10-11T06:11:14.934850+0000","last_undegraded":"2023-10-11T06:11:14.934850+0000","last_fullsized":"2023-10-11T06:11:14.934850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.696997+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:11:13.696997+0000","last_peered":"2023-10-11T06:11:13.696997+0000","last_clean":"2023-10-11T06:11:13.696997+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:11:13.696997+0000","last_undegraded":"2023-10-11T06:11:13.696997+0000","last_fullsized":"2023-10-11T06:11:13.696997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932868+0000","last_change":"2023-10-11T06:11:13.700313+0000","last_active":"2023-10-11T06:11:14.932868+0000","last_peered":"2023-10-11T06:11:14.932868+0000","last_clean":"2023-10-11T06:11:14.932868+0000","last_became_active":"2023-10-11T06:11:13.700001+0000","last_became_peered":"2023-10-11T06:11:13.700001+0000","last_unstale":"2023-10-11T06:11:14.932868+0000","last_undegraded":"2023-10-11T06:11:14.932868+0000","last_fullsized":"2023-10-11T06:11:14.932868+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690266+0000","last_change":"2023-10-11T06:11:13.214171+0000","last_active":"2023-10-11T06:11:14.690266+0000","last_peered":"2023-10-11T06:11:14.690266+0000","last_clean":"2023-10-11T06:11:14.690266+0000","last_became_active":"2023-10-11T06:11:13.213902+0000","last_became_peered":"2023-10-11T06:11:13.213902+0000","last_unstale":"2023-10-11T06:11:14.690266+0000","last_undegraded":"2023-10-11T06:11:14.690266+0000","last_fullsized":"2023-10-11T06:11:14.690266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.678254+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:11:13.678254+0000","last_peered":"2023-10-11T06:11:13.678254+0000","last_clean":"2023-10-11T06:11:13.678254+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:11:13.678254+0000","last_undegraded":"2023-10-11T06:11:13.678254+0000","last_fullsized":"2023-10-11T06:11:13.678254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687734+0000","last_change":"2023-10-11T06:11:13.709945+0000","last_active":"2023-10-11T06:11:14.687734+0000","last_peered":"2023-10-11T06:11:14.687734+0000","last_clean":"2023-10-11T06:11:14.687734+0000","last_became_active":"2023-10-11T06:11:13.709607+0000","last_became_peered":"2023-10-11T06:11:13.709607+0000","last_unstale":"2023-10-11T06:11:14.687734+0000","last_undegraded":"2023-10-11T06:11:14.687734+0000","last_fullsized":"2023-10-11T06:11:14.687734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933748+0000","last_change":"2023-10-11T06:11:13.213811+0000","last_active":"2023-10-11T06:11:14.933748+0000","last_peered":"2023-10-11T06:11:14.933748+0000","last_clean":"2023-10-11T06:11:14.933748+0000","last_became_active":"2023-10-11T06:11:13.213434+0000","last_became_peered":"2023-10-11T06:11:13.213434+0000","last_unstale":"2023-10-11T06:11:14.933748+0000","last_undegraded":"2023-10-11T06:11:14.933748+0000","last_fullsized":"2023-10-11T06:11:14.933748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.680148+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:11:13.680148+0000","last_peered":"2023-10-11T06:11:13.680148+0000","last_clean":"2023-10-11T06:11:13.680148+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:11:13.680148+0000","last_undegraded":"2023-10-11T06:11:13.680148+0000","last_fullsized":"2023-10-11T06:11:13.680148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936024+0000","last_change":"2023-10-11T06:11:13.700733+0000","last_active":"2023-10-11T06:11:14.936024+0000","last_peered":"2023-10-11T06:11:14.936024+0000","last_clean":"2023-10-11T06:11:14.936024+0000","last_became_active":"2023-10-11T06:11:13.700219+0000","last_became_peered":"2023-10-11T06:11:13.700219+0000","last_unstale":"2023-10-11T06:11:14.936024+0000","last_undegraded":"2023-10-11T06:11:14.936024+0000","last_fullsized":"2023-10-11T06:11:14.936024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936212+0000","last_change":"2023-10-11T06:11:13.691032+0000","last_active":"2023-10-11T06:11:14.936212+0000","last_peered":"2023-10-11T06:11:14.936212+0000","last_clean":"2023-10-11T06:11:14.936212+0000","last_became_active":"2023-10-11T06:11:13.690737+0000","last_became_peered":"2023-10-11T06:11:13.690737+0000","last_unstale":"2023-10-11T06:11:14.936212+0000","last_undegraded":"2023-10-11T06:11:14.936212+0000","last_fullsized":"2023-10-11T06:11:14.936212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.938254+0000","last_change":"2023-10-11T06:11:13.217477+0000","last_active":"2023-10-11T06:11:14.938254+0000","last_peered":"2023-10-11T06:11:14.938254+0000","last_clean":"2023-10-11T06:11:14.938254+0000","last_became_active":"2023-10-11T06:11:13.216485+0000","last_became_peered":"2023-10-11T06:11:13.216485+0000","last_unstale":"2023-10-11T06:11:14.938254+0000","last_undegraded":"2023-10-11T06:11:14.938254+0000","last_fullsized":"2023-10-11T06:11:14.938254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934689+0000","last_change":"2023-10-11T06:11:13.691615+0000","last_active":"2023-10-11T06:11:14.934689+0000","last_peered":"2023-10-11T06:11:14.934689+0000","last_clean":"2023-10-11T06:11:14.934689+0000","last_became_active":"2023-10-11T06:11:13.690979+0000","last_became_peered":"2023-10-11T06:11:13.690979+0000","last_unstale":"2023-10-11T06:11:14.934689+0000","last_undegraded":"2023-10-11T06:11:14.934689+0000","last_fullsized":"2023-10-11T06:11:14.934689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937582+0000","last_change":"2023-10-11T06:11:13.220058+0000","last_active":"2023-10-11T06:11:14.937582+0000","last_peered":"2023-10-11T06:11:14.937582+0000","last_clean":"2023-10-11T06:11:14.937582+0000","last_became_active":"2023-10-11T06:11:13.219554+0000","last_became_peered":"2023-10-11T06:11:13.219554+0000","last_unstale":"2023-10-11T06:11:14.937582+0000","last_undegraded":"2023-10-11T06:11:14.937582+0000","last_fullsized":"2023-10-11T06:11:14.937582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:48.717818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936220+0000","last_change":"2023-10-11T06:11:13.217619+0000","last_active":"2023-10-11T06:11:14.936220+0000","last_peered":"2023-10-11T06:11:14.936220+0000","last_clean":"2023-10-11T06:11:14.936220+0000","last_became_active":"2023-10-11T06:11:13.217261+0000","last_became_peered":"2023-10-11T06:11:13.217261+0000","last_unstale":"2023-10-11T06:11:14.936220+0000","last_undegraded":"2023-10-11T06:11:14.936220+0000","last_fullsized":"2023-10-11T06:11:14.936220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:34.256594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688528+0000","last_change":"2023-10-11T06:11:13.709235+0000","last_active":"2023-10-11T06:11:14.688528+0000","last_peered":"2023-10-11T06:11:14.688528+0000","last_clean":"2023-10-11T06:11:14.688528+0000","last_became_active":"2023-10-11T06:11:13.708979+0000","last_became_peered":"2023-10-11T06:11:13.708979+0000","last_unstale":"2023-10-11T06:11:14.688528+0000","last_undegraded":"2023-10-11T06:11:14.688528+0000","last_fullsized":"2023-10-11T06:11:14.688528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:24:24.143432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933598+0000","last_change":"2023-10-11T06:11:13.213756+0000","last_active":"2023-10-11T06:11:14.933598+0000","last_peered":"2023-10-11T06:11:14.933598+0000","last_clean":"2023-10-11T06:11:14.933598+0000","last_became_active":"2023-10-11T06:11:13.213456+0000","last_became_peered":"2023-10-11T06:11:13.213456+0000","last_unstale":"2023-10-11T06:11:14.933598+0000","last_undegraded":"2023-10-11T06:11:14.933598+0000","last_fullsized":"2023-10-11T06:11:14.933598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688932+0000","last_change":"2023-10-11T06:11:13.698101+0000","last_active":"2023-10-11T06:11:14.688932+0000","last_peered":"2023-10-11T06:11:14.688932+0000","last_clean":"2023-10-11T06:11:14.688932+0000","last_became_active":"2023-10-11T06:11:13.696880+0000","last_became_peered":"2023-10-11T06:11:13.696880+0000","last_unstale":"2023-10-11T06:11:14.688932+0000","last_undegraded":"2023-10-11T06:11:14.688932+0000","last_fullsized":"2023-10-11T06:11:14.688932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690335+0000","last_change":"2023-10-11T06:11:13.701258+0000","last_active":"2023-10-11T06:11:14.690335+0000","last_peered":"2023-10-11T06:11:14.690335+0000","last_clean":"2023-10-11T06:11:14.690335+0000","last_became_active":"2023-10-11T06:11:13.700995+0000","last_became_peered":"2023-10-11T06:11:13.700995+0000","last_unstale":"2023-10-11T06:11:14.690335+0000","last_undegraded":"2023-10-11T06:11:14.690335+0000","last_fullsized":"2023-10-11T06:11:14.690335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:44.575547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687997+0000","last_change":"2023-10-11T06:11:13.210038+0000","last_active":"2023-10-11T06:11:14.687997+0000","last_peered":"2023-10-11T06:11:14.687997+0000","last_clean":"2023-10-11T06:11:14.687997+0000","last_became_active":"2023-10-11T06:11:13.209702+0000","last_became_peered":"2023-10-11T06:11:13.209702+0000","last_unstale":"2023-10-11T06:11:14.687997+0000","last_undegraded":"2023-10-11T06:11:14.687997+0000","last_fullsized":"2023-10-11T06:11:14.687997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933487+0000","last_change":"2023-10-11T06:11:13.707123+0000","last_active":"2023-10-11T06:11:14.933487+0000","last_peered":"2023-10-11T06:11:14.933487+0000","last_clean":"2023-10-11T06:11:14.933487+0000","last_became_active":"2023-10-11T06:11:13.706856+0000","last_became_peered":"2023-10-11T06:11:13.706856+0000","last_unstale":"2023-10-11T06:11:14.933487+0000","last_undegraded":"2023-10-11T06:11:14.933487+0000","last_fullsized":"2023-10-11T06:11:14.933487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934110+0000","last_change":"2023-10-11T06:11:13.206009+0000","last_active":"2023-10-11T06:11:14.934110+0000","last_peered":"2023-10-11T06:11:14.934110+0000","last_clean":"2023-10-11T06:11:14.934110+0000","last_became_active":"2023-10-11T06:11:13.205671+0000","last_became_peered":"2023-10-11T06:11:13.205671+0000","last_unstale":"2023-10-11T06:11:14.934110+0000","last_undegraded":"2023-10-11T06:11:14.934110+0000","last_fullsized":"2023-10-11T06:11:14.934110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688523+0000","last_change":"2023-10-11T06:11:13.697631+0000","last_active":"2023-10-11T06:11:14.688523+0000","last_peered":"2023-10-11T06:11:14.688523+0000","last_clean":"2023-10-11T06:11:14.688523+0000","last_became_active":"2023-10-11T06:11:13.696974+0000","last_became_peered":"2023-10-11T06:11:13.696974+0000","last_unstale":"2023-10-11T06:11:14.688523+0000","last_undegraded":"2023-10-11T06:11:14.688523+0000","last_fullsized":"2023-10-11T06:11:14.688523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936125+0000","last_change":"2023-10-11T06:11:13.221421+0000","last_active":"2023-10-11T06:11:14.936125+0000","last_peered":"2023-10-11T06:11:14.936125+0000","last_clean":"2023-10-11T06:11:14.936125+0000","last_became_active":"2023-10-11T06:11:13.221114+0000","last_became_peered":"2023-10-11T06:11:13.221114+0000","last_unstale":"2023-10-11T06:11:14.936125+0000","last_undegraded":"2023-10-11T06:11:14.936125+0000","last_fullsized":"2023-10-11T06:11:14.936125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688237+0000","last_change":"2023-10-11T06:11:13.699277+0000","last_active":"2023-10-11T06:11:14.688237+0000","last_peered":"2023-10-11T06:11:14.688237+0000","last_clean":"2023-10-11T06:11:14.688237+0000","last_became_active":"2023-10-11T06:11:13.699042+0000","last_became_peered":"2023-10-11T06:11:13.699042+0000","last_unstale":"2023-10-11T06:11:14.688237+0000","last_undegraded":"2023-10-11T06:11:14.688237+0000","last_fullsized":"2023-10-11T06:11:14.688237+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933247+0000","last_change":"2023-10-11T06:11:13.218429+0000","last_active":"2023-10-11T06:11:14.933247+0000","last_peered":"2023-10-11T06:11:14.933247+0000","last_clean":"2023-10-11T06:11:14.933247+0000","last_became_active":"2023-10-11T06:11:13.218150+0000","last_became_peered":"2023-10-11T06:11:13.218150+0000","last_unstale":"2023-10-11T06:11:14.933247+0000","last_undegraded":"2023-10-11T06:11:14.933247+0000","last_fullsized":"2023-10-11T06:11:14.933247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935952+0000","last_change":"2023-10-11T06:11:13.700494+0000","last_active":"2023-10-11T06:11:14.935952+0000","last_peered":"2023-10-11T06:11:14.935952+0000","last_clean":"2023-10-11T06:11:14.935952+0000","last_became_active":"2023-10-11T06:11:13.700182+0000","last_became_peered":"2023-10-11T06:11:13.700182+0000","last_unstale":"2023-10-11T06:11:14.935952+0000","last_undegraded":"2023-10-11T06:11:14.935952+0000","last_fullsized":"2023-10-11T06:11:14.935952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934742+0000","last_change":"2023-10-11T06:11:13.221465+0000","last_active":"2023-10-11T06:11:14.934742+0000","last_peered":"2023-10-11T06:11:14.934742+0000","last_clean":"2023-10-11T06:11:14.934742+0000","last_became_active":"2023-10-11T06:11:13.221160+0000","last_became_peered":"2023-10-11T06:11:13.221160+0000","last_unstale":"2023-10-11T06:11:14.934742+0000","last_undegraded":"2023-10-11T06:11:14.934742+0000","last_fullsized":"2023-10-11T06:11:14.934742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935425+0000","last_change":"2023-10-11T06:11:13.690241+0000","last_active":"2023-10-11T06:11:14.935425+0000","last_peered":"2023-10-11T06:11:14.935425+0000","last_clean":"2023-10-11T06:11:14.935425+0000","last_became_active":"2023-10-11T06:11:13.689766+0000","last_became_peered":"2023-10-11T06:11:13.689766+0000","last_unstale":"2023-10-11T06:11:14.935425+0000","last_undegraded":"2023-10-11T06:11:14.935425+0000","last_fullsized":"2023-10-11T06:11:14.935425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:46:46.784397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936532+0000","last_change":"2023-10-11T06:11:13.212684+0000","last_active":"2023-10-11T06:11:14.936532+0000","last_peered":"2023-10-11T06:11:14.936532+0000","last_clean":"2023-10-11T06:11:14.936532+0000","last_became_active":"2023-10-11T06:11:13.212366+0000","last_became_peered":"2023-10-11T06:11:13.212366+0000","last_unstale":"2023-10-11T06:11:14.936532+0000","last_undegraded":"2023-10-11T06:11:14.936532+0000","last_fullsized":"2023-10-11T06:11:14.936532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688632+0000","last_change":"2023-10-11T06:11:13.706859+0000","last_active":"2023-10-11T06:11:14.688632+0000","last_peered":"2023-10-11T06:11:14.688632+0000","last_clean":"2023-10-11T06:11:14.688632+0000","last_became_active":"2023-10-11T06:11:13.706584+0000","last_became_peered":"2023-10-11T06:11:13.706584+0000","last_unstale":"2023-10-11T06:11:14.688632+0000","last_undegraded":"2023-10-11T06:11:14.688632+0000","last_fullsized":"2023-10-11T06:11:14.688632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688832+0000","last_change":"2023-10-11T06:11:13.200955+0000","last_active":"2023-10-11T06:11:14.688832+0000","last_peered":"2023-10-11T06:11:14.688832+0000","last_clean":"2023-10-11T06:11:14.688832+0000","last_became_active":"2023-10-11T06:11:13.200617+0000","last_became_peered":"2023-10-11T06:11:13.200617+0000","last_unstale":"2023-10-11T06:11:14.688832+0000","last_undegraded":"2023-10-11T06:11:14.688832+0000","last_fullsized":"2023-10-11T06:11:14.688832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935118+0000","last_change":"2023-10-11T06:11:13.689555+0000","last_active":"2023-10-11T06:11:14.935118+0000","last_peered":"2023-10-11T06:11:14.935118+0000","last_clean":"2023-10-11T06:11:14.935118+0000","last_became_active":"2023-10-11T06:11:13.689252+0000","last_became_peered":"2023-10-11T06:11:13.689252+0000","last_unstale":"2023-10-11T06:11:14.935118+0000","last_undegraded":"2023-10-11T06:11:14.935118+0000","last_fullsized":"2023-10-11T06:11:14.935118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:49.157560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933504+0000","last_change":"2023-10-11T06:11:13.213633+0000","last_active":"2023-10-11T06:11:14.933504+0000","last_peered":"2023-10-11T06:11:14.933504+0000","last_clean":"2023-10-11T06:11:14.933504+0000","last_became_active":"2023-10-11T06:11:13.213082+0000","last_became_peered":"2023-10-11T06:11:13.213082+0000","last_unstale":"2023-10-11T06:11:14.933504+0000","last_undegraded":"2023-10-11T06:11:14.933504+0000","last_fullsized":"2023-10-11T06:11:14.933504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":679765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.681}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":88269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.732}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.897}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":87707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7170000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":678567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.73}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:11:18.732 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T06:11:18.732 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-11T06:11:18.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-11T06:11:18.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-11T06:11:18.733 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-11T06:11:18.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.987+0000 7f6559e77640 1 Processor -- start 2023-10-11T06:11:18.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.987+0000 7f6559e77640 1 -- start start 2023-10-11T06:11:18.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.989+0000 7f6559e77640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65540af4d0 0x7f65540d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.989+0000 7f6559e77640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65540af930 con 0x7f655415c290 2023-10-11T06:11:18.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.989+0000 7f6559e77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655406cd10 con 0x7f65540d0c20 2023-10-11T06:11:18.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.989+0000 7f6559e77640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65540afb60 con 0x7f65540af4d0 2023-10-11T06:11:18.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.989+0000 7f65537fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f655415c290 0x7f65540d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54684/0 (socket says 172.21.15.84:54684) 2023-10-11T06:11:18.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.989+0000 7f65537fe640 1 -- 172.21.15.84:0/1145089398 learned_addr learned my addr 172.21.15.84:0/1145089398 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:18.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.989+0000 7f6553fff640 1 --2- 172.21.15.84:0/1145089398 >> v2:172.21.15.84:3300/0 conn(0x7f65540af4d0 0x7f65540d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:18.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.990+0000 7f6552ffd640 1 -- 172.21.15.84:0/1145089398 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519483564 0 0) 0x7f65540af930 con 0x7f655415c290 2023-10-11T06:11:18.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.990+0000 7f6552ffd640 1 -- 172.21.15.84:0/1145089398 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6540003520 con 0x7f655415c290 2023-10-11T06:11:18.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.990+0000 7f6553fff640 1 -- 172.21.15.84:0/1145089398 >> v1:172.21.15.99:6789/0 conn(0x7f65540d0c20 legacy=0x7f65540b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.990+0000 7f6553fff640 1 -- 172.21.15.84:0/1145089398 >> v1:172.21.15.84:6789/0 conn(0x7f655415c290 legacy=0x7f65540d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.990+0000 7f6553fff640 1 -- 172.21.15.84:0/1145089398 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540afe40 con 0x7f65540af4d0 2023-10-11T06:11:18.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.991+0000 7f6553fff640 1 --2- 172.21.15.84:0/1145089398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65540af4d0 0x7f65540d25e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41f880193b8ca79 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:18.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.991+0000 7f6552ffd640 1 -- 172.21.15.84:0/1145089398 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c02cc60 con 0x7f65540af4d0 2023-10-11T06:11:18.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.991+0000 7f6552ffd640 1 -- 172.21.15.84:0/1145089398 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c02cdc0 con 0x7f65540af4d0 2023-10-11T06:11:18.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.991+0000 7f6552ffd640 1 -- 172.21.15.84:0/1145089398 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c034660 con 0x7f65540af4d0 2023-10-11T06:11:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.992+0000 7f6559e77640 1 -- 172.21.15.84:0/1145089398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65540af4d0 msgr2=0x7f65540d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.992+0000 7f6559e77640 1 --2- 172.21.15.84:0/1145089398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65540af4d0 0x7f65540d25e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c02beb0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.992+0000 7f6559e77640 1 -- 172.21.15.84:0/1145089398 shutdown_connections 2023-10-11T06:11:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.992+0000 7f6559e77640 1 --2- 172.21.15.84:0/1145089398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65540af4d0 0x7f65540d25e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:18.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.992+0000 7f6559e77640 1 -- 172.21.15.84:0/1145089398 >> 172.21.15.84:0/1145089398 conn(0x7f6554071cc0 msgr2=0x7f655409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:18.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.992+0000 7f6559e77640 1 -- 172.21.15.84:0/1145089398 shutdown_connections 2023-10-11T06:11:18.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.992+0000 7f6559e77640 1 -- 172.21.15.84:0/1145089398 wait complete. 2023-10-11T06:11:18.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.993+0000 7f6559e77640 1 Processor -- start 2023-10-11T06:11:18.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.994+0000 7f6559e77640 1 -- start start 2023-10-11T06:11:18.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6559e77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65540af4d0 0x7f6554157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6559e77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65540d0c20 0x7f6554154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6559e77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655415c290 0x7f65540b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:18.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6559e77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6554156c20 con 0x7f655415c290 2023-10-11T06:11:18.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6559e77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6554157580 con 0x7f65540af4d0 2023-10-11T06:11:18.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6559e77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6554155fa0 con 0x7f65540d0c20 2023-10-11T06:11:18.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655415c290 0x7f65540b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655415c290 0x7f65540b7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53734/0 (socket says 172.21.15.84:53734) 2023-10-11T06:11:19.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 -- 172.21.15.84:0/491063759 learned_addr learned my addr 172.21.15.84:0/491063759 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:19.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f65537fe640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65540af4d0 0x7f6554157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 -- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65540d0c20 msgr2=0x7f6554154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65540d0c20 0x7f6554154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 -- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65540af4d0 msgr2=0x7f6554157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65540af4d0 0x7f6554157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f6553fff640 1 -- 172.21.15.84:0/491063759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c009a70 con 0x7f655415c290 2023-10-11T06:11:19.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.995+0000 7f65537fe640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65540af4d0 0x7f6554157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:19.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.996+0000 7f6553fff640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655415c290 0x7f65540b7730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f654c009eb0 tx=0x7f654c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.996+0000 7f6550ff9640 1 -- 172.21.15.84:0/491063759 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c040650 con 0x7f655415c290 2023-10-11T06:11:19.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.996+0000 7f6550ff9640 1 -- 172.21.15.84:0/491063759 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c0407d0 con 0x7f655415c290 2023-10-11T06:11:19.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.996+0000 7f6550ff9640 1 -- 172.21.15.84:0/491063759 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c040b80 con 0x7f655415c290 2023-10-11T06:11:19.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.996+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6554156280 con 0x7f655415c290 2023-10-11T06:11:19.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.996+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6554156840 con 0x7f655415c290 2023-10-11T06:11:19.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.997+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6554156d60 con 0x7f655415c290 2023-10-11T06:11:19.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.998+0000 7f6550ff9640 1 -- 172.21.15.84:0/491063759 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f654c040ce0 con 0x7f655415c290 2023-10-11T06:11:19.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:18.999+0000 7f6550ff9640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f653406a830 0x7f653406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.000+0000 7f6550ff9640 1 -- 172.21.15.84:0/491063759 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f654c0abe20 con 0x7f655415c290 2023-10-11T06:11:19.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.000+0000 7f65537fe640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f653406a830 0x7f653406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.000+0000 7f65537fe640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f653406a830 0x7f653406cce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65440011e0 tx=0x7f6544009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.004+0000 7f6550ff9640 1 -- 172.21.15.84:0/491063759 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f6554156d60 con 0x7f655415c290 2023-10-11T06:11:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.242+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6554156d60 con 0x7f655415c290 2023-10-11T06:11:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.246+0000 7f6550ff9640 1 -- 172.21.15.84:0/491063759 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170491 (secure 0 0 0) 0x7f6554156d60 con 0x7f655415c290 2023-10-11T06:11:19.249 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:19.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f653406a830 msgr2=0x7f653406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f653406a830 0x7f653406cce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65440011e0 tx=0x7f6544009040 comp rx=0 tx=0).stop 2023-10-11T06:11:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655415c290 msgr2=0x7f65540b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655415c290 0x7f65540b7730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f654c009eb0 tx=0x7f654c004a40 comp rx=0 tx=0).stop 2023-10-11T06:11:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 shutdown_connections 2023-10-11T06:11:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f653406a830 0x7f653406cce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f655415c290 0x7f65540b7730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65540d0c20 0x7f6554154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 --2- 172.21.15.84:0/491063759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65540af4d0 0x7f6554157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 >> 172.21.15.84:0/491063759 conn(0x7f6554071cc0 msgr2=0x7f65540b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 shutdown_connections 2023-10-11T06:11:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.250+0000 7f6559e77640 1 -- 172.21.15.84:0/491063759 wait complete. 2023-10-11T06:11:19.268 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":5,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":87595},{"type":"v1","addr":"172.21.15.84:6801","nonce":87595}]},"active_addr":"172.21.15.84:6801/87595","active_change":"2023-10-11T06:10:50.748910+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2535123046}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2248303068}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":1660678151}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2757180865}]}]} 2023-10-11T06:11:19.272 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-11T06:11:19.273 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-11T06:11:19.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:19.337 INFO:tasks.ceph.mds.d.smithi099.stderr:2023-10-11T06:11:19.336+0000 7f9b2306b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T06:11:19.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.476+0000 7f6877d5c640 1 Processor -- start 2023-10-11T06:11:19.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.476+0000 7f6877d5c640 1 -- start start 2023-10-11T06:11:19.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f6877d5c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68700adb90 0x7f68700caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f6877d5c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f6877d5c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68700cb960 con 0x7f68700ce0d0 2023-10-11T06:11:19.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f6877d5c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687015a340 con 0x7f68700adb90 2023-10-11T06:11:19.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f6877d5c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700ce4a0 con 0x7f68700cd300 2023-10-11T06:11:19.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f68752f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68700adb90 0x7f68700caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f68752f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68700adb90 0x7f68700caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53750/0 (socket says 172.21.15.84:53750) 2023-10-11T06:11:19.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.478+0000 7f68752f7640 1 -- 172.21.15.84:0/918720301 learned_addr learned my addr 172.21.15.84:0/918720301 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:19.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.479+0000 7f68762f9640 1 --2- 172.21.15.84:0/918720301 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.479+0000 7f68752f7640 1 -- 172.21.15.84:0/918720301 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 msgr2=0x7f68700cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:19.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.479+0000 7f68752f7640 1 --2- 172.21.15.84:0/918720301 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.479+0000 7f68752f7640 1 -- 172.21.15.84:0/918720301 >> v1:172.21.15.99:6789/0 conn(0x7f68700ce0d0 legacy=0x7f68700c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:19.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.479+0000 7f68752f7640 1 -- 172.21.15.84:0/918720301 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700ce610 con 0x7f68700adb90 2023-10-11T06:11:19.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.480+0000 7f68752f7640 1 --2- 172.21.15.84:0/918720301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700adb90 0x7f68700caee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f687006e440 tx=0x7f686002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fd76bfc308fe2c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.480+0000 7f6874af6640 1 -- 172.21.15.84:0/918720301 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860031830 con 0x7f68700adb90 2023-10-11T06:11:19.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.480+0000 7f6874af6640 1 -- 172.21.15.84:0/918720301 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6860031990 con 0x7f68700adb90 2023-10-11T06:11:19.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.480+0000 7f6874af6640 1 -- 172.21.15.84:0/918720301 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860031d20 con 0x7f68700adb90 2023-10-11T06:11:19.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 -- 172.21.15.84:0/918720301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700adb90 msgr2=0x7f68700caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 --2- 172.21.15.84:0/918720301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700adb90 0x7f68700caee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f687006e440 tx=0x7f686002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 -- 172.21.15.84:0/918720301 shutdown_connections 2023-10-11T06:11:19.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 --2- 172.21.15.84:0/918720301 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 --2- 172.21.15.84:0/918720301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700adb90 0x7f68700caee0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 -- 172.21.15.84:0/918720301 >> 172.21.15.84:0/918720301 conn(0x7f6870069c70 msgr2=0x7f687006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:19.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 -- 172.21.15.84:0/918720301 shutdown_connections 2023-10-11T06:11:19.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.481+0000 7f6877d5c640 1 -- 172.21.15.84:0/918720301 wait complete. 2023-10-11T06:11:19.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.482+0000 7f6877d5c640 1 Processor -- start 2023-10-11T06:11:19.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.482+0000 7f6877d5c640 1 -- start start 2023-10-11T06:11:19.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.483+0000 7f6877d5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68700adb90 0x7f68700c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.483+0000 7f6877d5c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.483+0000 7f6877d5c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700ce0d0 0x7f68700aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f6877d5c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c4be0 con 0x7f68700ce0d0 2023-10-11T06:11:19.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f6877d5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700af330 con 0x7f68700adb90 2023-10-11T06:11:19.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f6877d5c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6870163f30 con 0x7f68700cd300 2023-10-11T06:11:19.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700ce0d0 0x7f68700aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700ce0d0 0x7f68700aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53760/0 (socket says 172.21.15.84:53760) 2023-10-11T06:11:19.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 -- 172.21.15.84:0/3549172203 learned_addr learned my addr 172.21.15.84:0/3549172203 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:19.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f6875af8640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68700adb90 0x7f68700c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68752f7640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 -- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 msgr2=0x7f68700c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 -- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68700adb90 msgr2=0x7f68700c1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68700adb90 0x7f68700c1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.484+0000 7f68762f9640 1 -- 172.21.15.84:0/3549172203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600094d0 con 0x7f68700ce0d0 2023-10-11T06:11:19.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.485+0000 7f68762f9640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700ce0d0 0x7f68700aea70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f686800cc30 tx=0x7f686800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.485+0000 7f68667fc640 1 -- 172.21.15.84:0/3549172203 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800ce40 con 0x7f68700ce0d0 2023-10-11T06:11:19.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.485+0000 7f68667fc640 1 -- 172.21.15.84:0/3549172203 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868003990 con 0x7f68700ce0d0 2023-10-11T06:11:19.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.485+0000 7f68667fc640 1 -- 172.21.15.84:0/3549172203 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868003d20 con 0x7f68700ce0d0 2023-10-11T06:11:19.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.485+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6870164210 con 0x7f68700ce0d0 2023-10-11T06:11:19.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.485+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68701647d0 con 0x7f68700ce0d0 2023-10-11T06:11:19.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.486+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68701593d0 con 0x7f68700ce0d0 2023-10-11T06:11:19.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.487+0000 7f68667fc640 1 -- 172.21.15.84:0/3549172203 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6868003e80 con 0x7f68700ce0d0 2023-10-11T06:11:19.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.488+0000 7f68667fc640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f685406a830 0x7f685406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.488+0000 7f68667fc640 1 -- 172.21.15.84:0/3549172203 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68680893f0 con 0x7f68700ce0d0 2023-10-11T06:11:19.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.488+0000 7f6875af8640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f685406a830 0x7f685406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.489+0000 7f6875af8640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f685406a830 0x7f685406cce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f686c004e50 tx=0x7f686c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.492+0000 7f68667fc640 1 -- 172.21.15.84:0/3549172203 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f68701593d0 con 0x7f68700ce0d0 2023-10-11T06:11:19.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.682+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68701593d0 con 0x7f68700ce0d0 2023-10-11T06:11:19.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.685+0000 7f68667fc640 1 -- 172.21.15.84:0/3549172203 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17763 (secure 0 0 0) 0x7f68701593d0 con 0x7f68700ce0d0 2023-10-11T06:11:19.686 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:19.687 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:14.671090+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:19.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f685406a830 msgr2=0x7f685406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f685406a830 0x7f685406cce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f686c004e50 tx=0x7f686c009040 comp rx=0 tx=0).stop 2023-10-11T06:11:19.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700ce0d0 msgr2=0x7f68700aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700ce0d0 0x7f68700aea70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f686800cc30 tx=0x7f686800a5c0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 shutdown_connections 2023-10-11T06:11:19.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f685406a830 0x7f685406cce0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68700ce0d0 0x7f68700aea70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68700cd300 0x7f68700c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 --2- 172.21.15.84:0/3549172203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68700adb90 0x7f68700c1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.687+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 >> 172.21.15.84:0/3549172203 conn(0x7f6870069c70 msgr2=0x7f6870153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:19.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.688+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 shutdown_connections 2023-10-11T06:11:19.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.688+0000 7f6877d5c640 1 -- 172.21.15.84:0/3549172203 wait complete. 2023-10-11T06:11:19.707 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-11T06:11:19.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:19.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.911+0000 7f1958bf6640 1 Processor -- start 2023-10-11T06:11:19.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.911+0000 7f1958bf6640 1 -- start start 2023-10-11T06:11:19.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1958bf6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f195415c290 0x7f19540af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1958bf6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19540d0c20 0x7f19540b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1958bf6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19540d2a30 con 0x7f19540d25e0 2023-10-11T06:11:19.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1958bf6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195406cd10 con 0x7f19540d0c20 2023-10-11T06:11:19.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1958bf6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19540afb60 con 0x7f195415c290 2023-10-11T06:11:19.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1952d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19540d0c20 0x7f19540b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1952d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19540d0c20 0x7f19540b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53764/0 (socket says 172.21.15.84:53764) 2023-10-11T06:11:19.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1952d9d640 1 -- 172.21.15.84:0/688676779 learned_addr learned my addr 172.21.15.84:0/688676779 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:19.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f195259c640 1 --2- 172.21.15.84:0/688676779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f195415c290 0x7f19540af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1952d9d640 1 -- 172.21.15.84:0/688676779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f195415c290 msgr2=0x7f19540af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:19.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1952d9d640 1 --2- 172.21.15.84:0/688676779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f195415c290 0x7f19540af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1952d9d640 1 -- 172.21.15.84:0/688676779 >> v1:172.21.15.99:6789/0 conn(0x7f19540d25e0 legacy=0x7f19540d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:19.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.913+0000 7f1952d9d640 1 -- 172.21.15.84:0/688676779 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19540afe40 con 0x7f19540d0c20 2023-10-11T06:11:19.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.914+0000 7f1952d9d640 1 --2- 172.21.15.84:0/688676779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b0c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f195406ceb0 tx=0x7f194c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0612ab8b4f68b89 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.915+0000 7f1951d9b640 1 -- 172.21.15.84:0/688676779 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c02c9c0 con 0x7f19540d0c20 2023-10-11T06:11:19.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.915+0000 7f1951d9b640 1 -- 172.21.15.84:0/688676779 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c02cb20 con 0x7f19540d0c20 2023-10-11T06:11:19.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.915+0000 7f1951d9b640 1 -- 172.21.15.84:0/688676779 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0343f0 con 0x7f19540d0c20 2023-10-11T06:11:19.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.915+0000 7f1958bf6640 1 -- 172.21.15.84:0/688676779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 msgr2=0x7f19540b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.915+0000 7f1958bf6640 1 --2- 172.21.15.84:0/688676779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b0c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f195406ceb0 tx=0x7f194c02be20 comp rx=0 tx=0).stop 2023-10-11T06:11:19.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.916+0000 7f1958bf6640 1 -- 172.21.15.84:0/688676779 shutdown_connections 2023-10-11T06:11:19.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.916+0000 7f1958bf6640 1 --2- 172.21.15.84:0/688676779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b0c00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.916+0000 7f1958bf6640 1 --2- 172.21.15.84:0/688676779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f195415c290 0x7f19540af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.916+0000 7f1958bf6640 1 -- 172.21.15.84:0/688676779 >> 172.21.15.84:0/688676779 conn(0x7f1954071cc0 msgr2=0x7f195409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:19.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.916+0000 7f1958bf6640 1 -- 172.21.15.84:0/688676779 shutdown_connections 2023-10-11T06:11:19.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.916+0000 7f1958bf6640 1 -- 172.21.15.84:0/688676779 wait complete. 2023-10-11T06:11:19.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.917+0000 7f1958bf6640 1 Processor -- start 2023-10-11T06:11:19.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.918+0000 7f1958bf6640 1 -- start start 2023-10-11T06:11:19.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.918+0000 7f1958bf6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f1958bf6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19540d25e0 0x7f19540b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f1958bf6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f195415c290 0x7f195409e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f1958bf6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19540ce300 con 0x7f19540d0c20 2023-10-11T06:11:19.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f1958bf6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954149750 con 0x7f195415c290 2023-10-11T06:11:19.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f1958bf6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19540ce530 con 0x7f19540d25e0 2023-10-11T06:11:19.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53774/0 (socket says 172.21.15.84:53774) 2023-10-11T06:11:19.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 -- 172.21.15.84:0/2566877408 learned_addr learned my addr 172.21.15.84:0/2566877408 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:19.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 -- 172.21.15.84:0/2566877408 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19540d25e0 msgr2=0x7f19540b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19540d25e0 0x7f19540b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 -- 172.21.15.84:0/2566877408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f195415c290 msgr2=0x7f195409e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:19.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f195415c290 0x7f195409e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:19.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 -- 172.21.15.84:0/2566877408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009b30 con 0x7f19540d0c20 2023-10-11T06:11:19.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.919+0000 7f195259c640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b6d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f194400d5f0 tx=0x7f194400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.920+0000 7f194affd640 1 -- 172.21.15.84:0/2566877408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440125f0 con 0x7f19540d0c20 2023-10-11T06:11:19.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.920+0000 7f194affd640 1 -- 172.21.15.84:0/2566877408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1944012750 con 0x7f19540d0c20 2023-10-11T06:11:19.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.920+0000 7f194affd640 1 -- 172.21.15.84:0/2566877408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944012b00 con 0x7f19540d0c20 2023-10-11T06:11:19.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.920+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1954149470 con 0x7f19540d0c20 2023-10-11T06:11:19.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.920+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19540ceab0 con 0x7f19540d0c20 2023-10-11T06:11:19.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.921+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1954073ed0 con 0x7f19540d0c20 2023-10-11T06:11:19.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.922+0000 7f194affd640 1 -- 172.21.15.84:0/2566877408 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1944012c60 con 0x7f19540d0c20 2023-10-11T06:11:19.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.923+0000 7f194affd640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f193406a830 0x7f193406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:19.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.923+0000 7f194affd640 1 -- 172.21.15.84:0/2566877408 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1944089540 con 0x7f19540d0c20 2023-10-11T06:11:19.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.923+0000 7f194bfff640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f193406a830 0x7f193406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:19.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.924+0000 7f194bfff640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f193406a830 0x7f193406cce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19540a0820 tx=0x7f193c0079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:19.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:19.927+0000 7f194affd640 1 -- 172.21.15.84:0/2566877408 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1944054720 con 0x7f19540d0c20 2023-10-11T06:11:20.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.117+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f195406c720 con 0x7f19540d0c20 2023-10-11T06:11:20.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.120+0000 7f194affd640 1 -- 172.21.15.84:0/2566877408 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17763 (secure 0 0 0) 0x7f1944057d40 con 0x7f19540d0c20 2023-10-11T06:11:20.122 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:20.122 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:14.671090+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:20.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f193406a830 msgr2=0x7f193406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f193406a830 0x7f193406cce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19540a0820 tx=0x7f193c0079b0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 msgr2=0x7f19540b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b6d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f194400d5f0 tx=0x7f194400dab0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 shutdown_connections 2023-10-11T06:11:20.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f193406a830 0x7f193406cce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f195415c290 0x7f195409e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19540d25e0 0x7f19540b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 --2- 172.21.15.84:0/2566877408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19540d0c20 0x7f19540b6d10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 >> 172.21.15.84:0/2566877408 conn(0x7f1954071cc0 msgr2=0x7f19540b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 shutdown_connections 2023-10-11T06:11:20.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.123+0000 7f1958bf6640 1 -- 172.21.15.84:0/2566877408 wait complete. 2023-10-11T06:11:20.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-11T06:11:20.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-11T06:11:20.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-11T06:11:20.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-11T06:11:20.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-11T06:11:20.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-11T06:11:20.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-11T06:11:20.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-11T06:11:20.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.446+0000 7f8e4ff1b640 1 Processor -- start 2023-10-11T06:11:20.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.446+0000 7f31e5ca7640 1 Processor -- start 2023-10-11T06:11:20.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.446+0000 7f31e5ca7640 1 -- start start 2023-10-11T06:11:20.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.446+0000 7f8e4ff1b640 1 -- start start 2023-10-11T06:11:20.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f8e4ff1b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e400de910 0x7f8e4004f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f8e4ff1b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e40036d70 0x7f8e40052cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f8e4ff1b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e40005f90 con 0x7f8e4004f770 2023-10-11T06:11:20.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f8e4ff1b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e400068f0 con 0x7f8e400de910 2023-10-11T06:11:20.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f8e4ff1b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40006230 con 0x7f8e40036d70 2023-10-11T06:11:20.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f8e4d4b6640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8e4004f770 0x7f8e40036660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54696/0 (socket says 172.21.15.84:54696) 2023-10-11T06:11:20.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f8e4d4b6640 1 -- 172.21.15.84:0/3226943050 learned_addr learned my addr 172.21.15.84:0/3226943050 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:20.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f31e5ca7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31d8036660 0x7f31d8036ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f31e5ca7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f31d8052cc0 0x7f31d80530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f31e5ca7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d8036ff0 con 0x7f31d80de910 2023-10-11T06:11:20.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f31e5ca7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d80068f0 con 0x7f31d8036660 2023-10-11T06:11:20.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.448+0000 7f31e5ca7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d8005f90 con 0x7f31d8052cc0 2023-10-11T06:11:20.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31deffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31d8036660 0x7f31d8036ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31deffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f31d8036660 0x7f31d8036ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53784/0 (socket says 172.21.15.84:53784) 2023-10-11T06:11:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31deffd640 1 -- 172.21.15.84:0/717853433 learned_addr learned my addr 172.21.15.84:0/717853433 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31dffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f31d8052cc0 0x7f31d80530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31deffd640 1 -- 172.21.15.84:0/717853433 >> v2:172.21.15.99:3300/0 conn(0x7f31d8052cc0 msgr2=0x7f31d80530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31deffd640 1 --2- 172.21.15.84:0/717853433 >> v2:172.21.15.99:3300/0 conn(0x7f31d8052cc0 0x7f31d80530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31deffd640 1 -- 172.21.15.84:0/717853433 >> v1:172.21.15.84:6789/0 conn(0x7f31d80de910 legacy=0x7f31d804f230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-11T06:11:20.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.449+0000 7f31deffd640 1 -- 172.21.15.84:0/717853433 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8006230 con 0x7f31d8036660 2023-10-11T06:11:20.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.451+0000 7f31deffd640 1 --2- 172.21.15.84:0/717853433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8036660 0x7f31d8036ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f31e0066ad0 tx=0x7f31e00ae460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e76bc17f0c8900 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.451+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e40036d70 0x7f8e40052cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.451+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325633561 0 0) 0x7f8e40005f90 con 0x7f8e4004f770 2023-10-11T06:11:20.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4dcb7640 1 --2- 172.21.15.84:0/3226943050 >> v2:172.21.15.99:3300/0 conn(0x7f8e400de910 0x7f8e4004f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e34003440 con 0x7f8e4004f770 2023-10-11T06:11:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f31de7fc640 1 -- 172.21.15.84:0/717853433 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00b5660 con 0x7f31d8036660 2023-10-11T06:11:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f31de7fc640 1 -- 172.21.15.84:0/717853433 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e00b1050 con 0x7f31d8036660 2023-10-11T06:11:20.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f31de7fc640 1 -- 172.21.15.84:0/717853433 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e00b59e0 con 0x7f31d8036660 2023-10-11T06:11:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 338527789 0 0) 0x7f8e34003440 con 0x7f8e4004f770 2023-10-11T06:11:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e40036d70 msgr2=0x7f8e40052cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e40036d70 0x7f8e40052cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 >> v2:172.21.15.99:3300/0 conn(0x7f8e400de910 msgr2=0x7f8e4004f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 --2- 172.21.15.84:0/3226943050 >> v2:172.21.15.99:3300/0 conn(0x7f8e400de910 0x7f8e4004f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40006510 con 0x7f8e4004f770 2023-10-11T06:11:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f8e38004cd0 con 0x7f8e4004f770 2023-10-11T06:11:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.452+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 >> v1:172.21.15.84:6789/0 conn(0x7f8e4004f770 legacy=0x7f8e40036660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f8e4ccb5640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34003a60 0x7f8e34003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f8e4ccb5640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400ac50 0x7f8e3400d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f8e4ccb5640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e3400d750 0x7f8e3400fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400068f0 con 0x7f8e3400d750 2023-10-11T06:11:20.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40006230 con 0x7f8e3400ac50 2023-10-11T06:11:20.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e34010510 con 0x7f8e34003a60 2023-10-11T06:11:20.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.454+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e3400d750 0x7f8e3400fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.454+0000 7f8e4dcb7640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34003a60 0x7f8e34003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f31e5ca7640 1 -- 172.21.15.84:0/717853433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8036660 msgr2=0x7f31d8036ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.453+0000 7f31e5ca7640 1 --2- 172.21.15.84:0/717853433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8036660 0x7f31d8036ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f31e0066ad0 tx=0x7f31e00ae460 comp rx=0 tx=0).stop 2023-10-11T06:11:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.455+0000 7f8e4d4b6640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400ac50 0x7f8e3400d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.455+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34003a60 msgr2=0x7f8e34003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.455+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34003a60 0x7f8e34003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.455+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400ac50 msgr2=0x7f8e3400d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.455+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400ac50 0x7f8e3400d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.455+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/3226943050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e34003440 con 0x7f8e3400d750 2023-10-11T06:11:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.455+0000 7f8e4d4b6640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400ac50 0x7f8e3400d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.456+0000 7f31e5ca7640 1 -- 172.21.15.84:0/717853433 shutdown_connections 2023-10-11T06:11:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.456+0000 7f31e5ca7640 1 --2- 172.21.15.84:0/717853433 >> v2:172.21.15.99:3300/0 conn(0x7f31d8052cc0 0x7f31d80530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.456+0000 7f31e5ca7640 1 --2- 172.21.15.84:0/717853433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d8036660 0x7f31d8036ab0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.456+0000 7f31e5ca7640 1 -- 172.21.15.84:0/717853433 >> 172.21.15.84:0/717853433 conn(0x7f31d8051660 msgr2=0x7f31d8051a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.457+0000 7f31e5ca7640 1 -- 172.21.15.84:0/717853433 shutdown_connections 2023-10-11T06:11:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.457+0000 7f31e5ca7640 1 -- 172.21.15.84:0/717853433 wait complete. 2023-10-11T06:11:20.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.457+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e3400d750 0x7f8e3400fbb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8e4004bc30 tx=0x7f8e340036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814d79bf05b43e63 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.457+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44032600 con 0x7f8e3400d750 2023-10-11T06:11:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.457+0000 7f8e4ccb5640 1 -- 172.21.15.84:0/3226943050 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4402e050 con 0x7f8e3400d750 2023-10-11T06:11:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.458+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/3226943050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e3400d750 msgr2=0x7f8e3400fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.458+0000 7f8e4ff1b640 1 --2- 172.21.15.84:0/3226943050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e3400d750 0x7f8e3400fbb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8e4004bc30 tx=0x7f8e340036f0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.458+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/3226943050 reap_dead start 2023-10-11T06:11:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.459+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/3226943050 shutdown_connections 2023-10-11T06:11:20.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.459+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/3226943050 >> 172.21.15.84:0/3226943050 conn(0x7f8e40051660 msgr2=0x7f8e40051a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.460+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/3226943050 shutdown_connections 2023-10-11T06:11:20.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.460+0000 7f31e5ca7640 1 Processor -- start 2023-10-11T06:11:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.460+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/3226943050 wait complete. 2023-10-11T06:11:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.463+0000 7f8e4ff1b640 1 Processor -- start 2023-10-11T06:11:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.463+0000 7f8e4ff1b640 1 -- start start 2023-10-11T06:11:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.460+0000 7f31e5ca7640 1 -- start start 2023-10-11T06:11:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.461+0000 7f31e5ca7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d8036660 0x7f31d80dfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.461+0000 7f31e5ca7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d8052cc0 0x7f31d81853a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31e5ca7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d80de910 0x7f31d817e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31deffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d8052cc0 0x7f31d81853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31deffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d8052cc0 0x7f31d81853a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:35284/0 (socket says 172.21.15.84:35284) 2023-10-11T06:11:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d80de910 0x7f31d817e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f8e4ff1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400d750 0x7f8e400df770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d80de910 0x7f31d817e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53806/0 (socket says 172.21.15.84:53806) 2023-10-11T06:11:20.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f8e4ff1b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e400e18b0 0x7f8e400d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f8e4ff1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e400cc810 0x7f8e400ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31deffd640 1 -- 172.21.15.84:0/2569884811 learned_addr learned my addr 172.21.15.84:0/2569884811 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f8e4ff1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400d7aa0 con 0x7f8e400cc810 2023-10-11T06:11:20.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f8e4dcb7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400d750 0x7f8e400df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f8e4dcb7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400d750 0x7f8e400df770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:35290/0 (socket says 172.21.15.84:35290) 2023-10-11T06:11:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.466+0000 7f8e4dcb7640 1 -- 172.21.15.84:0/1526938110 learned_addr learned my addr 172.21.15.84:0/1526938110 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8038a30 con 0x7f31d80de910 2023-10-11T06:11:20.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.466+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e400cc810 0x7f8e400ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d81858e0 con 0x7f31d8052cc0 2023-10-11T06:11:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.462+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d817ee00 con 0x7f31d8036660 2023-10-11T06:11:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.463+0000 7f31deffd640 1 -- 172.21.15.84:0/2569884811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d8036660 msgr2=0x7f31d80dfb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-11T06:11:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.463+0000 7f31deffd640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d8036660 0x7f31d80dfb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.463+0000 7f31deffd640 1 -- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d80de910 msgr2=0x7f31d817e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.463+0000 7f31deffd640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d80de910 0x7f31d817e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.463+0000 7f31deffd640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0072be0 con 0x7f31d8052cc0 2023-10-11T06:11:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.464+0000 7f31dffff640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31d80de910 0x7f31d817e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.464+0000 7f31deffd640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d8052cc0 0x7f31d81853a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31e002c840 tx=0x7f31e0068140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0096790 con 0x7f31d8052cc0 2023-10-11T06:11:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e00968f0 con 0x7f31d8052cc0 2023-10-11T06:11:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d8037db0 con 0x7f31d8052cc0 2023-10-11T06:11:20.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.465+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d8038370 con 0x7f31d8052cc0 2023-10-11T06:11:20.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.466+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e008e680 con 0x7f31d8052cc0 2023-10-11T06:11:20.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7f8274f16640 1 Processor -- start 2023-10-11T06:11:20.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7f8274f16640 1 -- start start 2023-10-11T06:11:20.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7fb529fd8640 1 Processor -- start 2023-10-11T06:11:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.466+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400cd1c0 con 0x7f8e3400d750 2023-10-11T06:11:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.466+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400e1d20 con 0x7f8e400e18b0 2023-10-11T06:11:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.466+0000 7f8e4d4b6640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e400e18b0 0x7f8e400d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/1526938110 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e400e18b0 msgr2=0x7f8e400d9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e400e18b0 0x7f8e400d9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/1526938110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400d750 msgr2=0x7f8e400df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e3400d750 0x7f8e400df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40006e90 con 0x7f8e400cc810 2023-10-11T06:11:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.468+0000 7f8e4e4b8640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e400cc810 0x7f8e400ccc80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e44032000 tx=0x7f8e44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7f8274f16640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8270160900 0x7f8270164cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31bc000fc0 con 0x7f31d8052cc0 2023-10-11T06:11:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44031ad0 con 0x7f8e400cc810 2023-10-11T06:11:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44031c30 con 0x7f8e400cc810 2023-10-11T06:11:20.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e400dfe50 con 0x7f8e400cc810 2023-10-11T06:11:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.467+0000 7fb529fd8640 1 -- start start 2023-10-11T06:11:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7fb529fd8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb51c0d97b0 0x7fb51c0e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7f8274f16640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82700cc6a0 0x7f82700b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7fb529fd8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0dfa80 con 0x7fb51c0d9b80 2023-10-11T06:11:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7fb529fd8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c050790 con 0x7fb51c0502f0 2023-10-11T06:11:20.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.470+0000 7f8274f16640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8270165230 con 0x7f82700ccaf0 2023-10-11T06:11:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.470+0000 7f8274f16640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827006e4d0 con 0x7f82700cc6a0 2023-10-11T06:11:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.470+0000 7f8274f16640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827006e7b0 con 0x7f8270160900 2023-10-11T06:11:20.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7fb529fd8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0e3520 con 0x7fb51c0d97b0 2023-10-11T06:11:20.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.469+0000 7f8e4ff1b640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e400e0410 con 0x7f8e400cc810 2023-10-11T06:11:20.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.470+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44004020 con 0x7f8e400cc810 2023-10-11T06:11:20.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.470+0000 7f826e59c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8270160900 0x7f8270164cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.470+0000 7f826e59c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8270160900 0x7f8270164cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53248/0 (socket says 172.21.15.84:53248) 2023-10-11T06:11:20.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.470+0000 7f826e59c640 1 -- 172.21.15.84:0/1563592793 learned_addr learned my addr 172.21.15.84:0/1563592793 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.471+0000 7fb523fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb51c0d9b80 0x7fb51c007b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54704/0 (socket says 172.21.15.84:54704) 2023-10-11T06:11:20.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.471+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e40006d10 con 0x7f8e400cc810 2023-10-11T06:11:20.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.471+0000 7fb523fff640 1 -- 172.21.15.84:0/2961248310 learned_addr learned my addr 172.21.15.84:0/2961248310 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:20.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7fb528fd6640 1 --2- 172.21.15.84:0/2961248310 >> v2:172.21.15.84:3300/0 conn(0x7fb51c0d97b0 0x7fb51c0e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7f826e59c640 1 -- 172.21.15.84:0/1563592793 >> v2:172.21.15.84:3300/0 conn(0x7f82700cc6a0 msgr2=0x7f82700b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7f826e59c640 1 --2- 172.21.15.84:0/1563592793 >> v2:172.21.15.84:3300/0 conn(0x7f82700cc6a0 0x7f82700b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7f826e59c640 1 -- 172.21.15.84:0/1563592793 >> v1:172.21.15.84:6789/0 conn(0x7f82700ccaf0 legacy=0x7f82700cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7f826e59c640 1 -- 172.21.15.84:0/1563592793 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82700cac60 con 0x7f8270160900 2023-10-11T06:11:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f31e008e7e0 con 0x7f31d8052cc0 2023-10-11T06:11:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7fb528fd6640 1 -- 172.21.15.84:0/2961248310 >> v1:172.21.15.99:6789/0 conn(0x7fb51c0502f0 legacy=0x7fb51c037640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7fb528fd6640 1 -- 172.21.15.84:0/2961248310 >> v1:172.21.15.84:6789/0 conn(0x7fb51c0d9b80 legacy=0x7fb51c007b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.472+0000 7fb528fd6640 1 -- 172.21.15.84:0/2961248310 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0e3800 con 0x7fb51c0d97b0 2023-10-11T06:11:20.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.474+0000 7f31cbfff640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f31c406a850 0x7f31c406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.473+0000 7f826e59c640 1 --2- 172.21.15.84:0/1563592793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8270160900 0x7f8270164cf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8268009e50 tx=0x7f826802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632db6ee0fd458c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.474+0000 7f826d59a640 1 -- 172.21.15.84:0/1563592793 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826802ca70 con 0x7f8270160900 2023-10-11T06:11:20.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.474+0000 7fb528fd6640 1 --2- 172.21.15.84:0/2961248310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d97b0 0x7fb51c0e2ed0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb51800ba40 tx=0x7fb51802e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418b316f2c8960ea server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.474+0000 7f826d59a640 1 -- 172.21.15.84:0/1563592793 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826802cbd0 con 0x7f8270160900 2023-10-11T06:11:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.474+0000 7f826d59a640 1 -- 172.21.15.84:0/1563592793 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680343f0 con 0x7f8270160900 2023-10-11T06:11:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7f8274f16640 1 -- 172.21.15.84:0/1563592793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8270160900 msgr2=0x7f8270164cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8e44004180 con 0x7f8e400cc810 2023-10-11T06:11:20.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7fb5237fe640 1 -- 172.21.15.84:0/2961248310 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51802ee60 con 0x7fb51c0d97b0 2023-10-11T06:11:20.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7fb5237fe640 1 -- 172.21.15.84:0/2961248310 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518036440 con 0x7fb51c0d97b0 2023-10-11T06:11:20.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7fb5237fe640 1 -- 172.21.15.84:0/2961248310 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180367d0 con 0x7fb51c0d97b0 2023-10-11T06:11:20.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.474+0000 7f31df7fe640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f31c406a850 0x7f31c406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31e0093070 con 0x7f31d8052cc0 2023-10-11T06:11:20.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7f31cbfff640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f31c4073100 0x7f31c4075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31c4075bc0 con 0x7f31c4073100 2023-10-11T06:11:20.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e0138350 con 0x7f31d8052cc0 2023-10-11T06:11:20.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f31df7fe640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f31c406a850 0x7f31c406cd00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f31d804b700 tx=0x7f31cc006b30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f31dffff640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f31c4073100 0x7f31c4075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.475+0000 7f8274f16640 1 --2- 172.21.15.84:0/1563592793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8270160900 0x7f8270164cf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8268009e50 tx=0x7f826802bed0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f8274f16640 1 -- 172.21.15.84:0/1563592793 shutdown_connections 2023-10-11T06:11:20.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f8274f16640 1 --2- 172.21.15.84:0/1563592793 >> v2:172.21.15.84:3300/0 conn(0x7f82700cc6a0 0x7f82700b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f8274f16640 1 --2- 172.21.15.84:0/1563592793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8270160900 0x7f8270164cf0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8268009e50 tx=0x7f826802bed0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f8274f16640 1 -- 172.21.15.84:0/1563592793 >> 172.21.15.84:0/1563592793 conn(0x7f827006bf30 msgr2=0x7f827006c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f8274f16640 1 -- 172.21.15.84:0/1563592793 shutdown_connections 2023-10-11T06:11:20.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f8274f16640 1 -- 172.21.15.84:0/1563592793 wait complete. 2023-10-11T06:11:20.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7f31dffff640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f31c4073100 0x7f31c4075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.477+0000 7f8e3e7fc640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2c06a900 0x7f8e2c06cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.477+0000 7fb529fd8640 1 -- 172.21.15.84:0/2961248310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d97b0 msgr2=0x7fb51c0e2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.477+0000 7fb529fd8640 1 --2- 172.21.15.84:0/2961248310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d97b0 0x7fb51c0e2ed0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb51800ba40 tx=0x7fb51802e0b0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.476+0000 7fbfe9aca640 1 Processor -- start 2023-10-11T06:11:20.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.477+0000 7fbfe9aca640 1 -- start start 2023-10-11T06:11:20.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.477+0000 7fb529fd8640 1 -- 172.21.15.84:0/2961248310 shutdown_connections 2023-10-11T06:11:20.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.477+0000 7fb529fd8640 1 --2- 172.21.15.84:0/2961248310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d97b0 0x7fb51c0e2ed0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.477+0000 7fb529fd8640 1 -- 172.21.15.84:0/2961248310 >> 172.21.15.84:0/2961248310 conn(0x7fb51c052640 msgr2=0x7fb51c052a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.478+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== osd.0 v2:172.21.15.84:6802/87856 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f31c4075bc0 con 0x7f31c4073100 2023-10-11T06:11:20.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.478+0000 7f8e4dcb7640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2c06a900 0x7f8e2c06cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.478+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e440ab160 con 0x7f8e400cc810 2023-10-11T06:11:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.478+0000 7f8e3e7fc640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f8e2c0731b0 0x7f8e2c0755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.478+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e2c075c70 con 0x7f8e2c0731b0 2023-10-11T06:11:20.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.478+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e440ab510 con 0x7f8e400cc810 2023-10-11T06:11:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.478+0000 7fb529fd8640 1 -- 172.21.15.84:0/2961248310 shutdown_connections 2023-10-11T06:11:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.479+0000 7fbfe9aca640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfe40d09d0 0x7fbfe40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.479+0000 7fbfe9aca640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe40b1e40 con 0x7fbfe40cecb0 2023-10-11T06:11:20.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.479+0000 7fb529fd8640 1 -- 172.21.15.84:0/2961248310 wait complete. 2023-10-11T06:11:20.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7fbfe9aca640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe406acc0 con 0x7fbfe40d3060 2023-10-11T06:11:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7f8e4d4b6640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f8e2c0731b0 0x7f8e2c0755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7fbfe2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfe40d09d0 0x7fbfe40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7fbfe2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfe40d09d0 0x7fbfe40b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53834/0 (socket says 172.21.15.84:53834) 2023-10-11T06:11:20.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3571776737 learned_addr learned my addr 172.21.15.84:0/3571776737 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.479+0000 7f8274f16640 1 Processor -- start 2023-10-11T06:11:20.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.479+0000 7f8274f16640 1 -- start start 2023-10-11T06:11:20.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3571776737 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe415c770 con 0x7fbfe40d09d0 2023-10-11T06:11:20.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7f8e4dcb7640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2c06a900 0x7f8e2c06cdb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e4804f820 tx=0x7f8e4806e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.480+0000 7fbfe37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbfe40cecb0 0x7fbfe40d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54730/0 (socket says 172.21.15.84:54730) 2023-10-11T06:11:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.481+0000 7f8274f16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82700cc6a0 0x7f8270164810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.481+0000 7f8274f16640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82700ccaf0 0x7f82701f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7f8274f16640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82701f9a70 0x7f82700af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7f8274f16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82700c22d0 con 0x7f82700cc6a0 2023-10-11T06:11:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7f8274f16640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8270163da0 con 0x7f82701f9a70 2023-10-11T06:11:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.481+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3571776737 >> v1:172.21.15.99:6789/0 conn(0x7fbfe40d3060 legacy=0x7fbfe40ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7f8274f16640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82700c4fc0 con 0x7f82700ccaf0 2023-10-11T06:11:20.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.481+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3571776737 >> v1:172.21.15.84:6789/0 conn(0x7fbfe40cecb0 legacy=0x7fbfe40d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.481+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3571776737 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe406afa0 con 0x7fbfe40d09d0 2023-10-11T06:11:20.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7fb529fd8640 1 Processor -- start 2023-10-11T06:11:20.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.483+0000 7f33b8cc4640 1 Processor -- start 2023-10-11T06:11:20.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.483+0000 7fb529fd8640 1 -- start start 2023-10-11T06:11:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.483+0000 7f33b8cc4640 1 -- start start 2023-10-11T06:11:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7f8e4d4b6640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f8e2c0731b0 0x7f8e2c0755c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.483+0000 7fbfe2ffd640 1 --2- 172.21.15.84:0/3571776737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40b1640 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009870 tx=0x7fbfd802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6ce3836fe773ea server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7f826dd9b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82700ccaf0 0x7f82701f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.483+0000 7f826dd9b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82700ccaf0 0x7f82701f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53260/0 (socket says 172.21.15.84:53260) 2023-10-11T06:11:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.483+0000 7f826dd9b640 1 -- 172.21.15.84:0/3967174629 learned_addr learned my addr 172.21.15.84:0/3967174629 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.482+0000 7f826e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82700cc6a0 0x7f8270164810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== osd.1 v2:172.21.15.84:6810/87859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8e2c075c70 con 0x7f8e2c0731b0 2023-10-11T06:11:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fb529fd8640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb51c0502f0 0x7fb51c04b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fb529fd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c0d97b0 0x7fb51c0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fb529fd8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d9b80 0x7fb51c0389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fb529fd8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0d0e20 con 0x7fb51c0d9b80 2023-10-11T06:11:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fb529fd8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c03a2e0 con 0x7fb51c0d97b0 2023-10-11T06:11:20.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fb529fd8640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c039240 con 0x7fb51c0502f0 2023-10-11T06:11:20.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7fb528fd6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb51c0502f0 0x7fb51c04b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fbfe27fc640 1 -- 172.21.15.84:0/3571776737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8030860 con 0x7fbfe40d09d0 2023-10-11T06:11:20.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fbfe27fc640 1 -- 172.21.15.84:0/3571776737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd80309c0 con 0x7fbfe40d09d0 2023-10-11T06:11:20.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7fbfe27fc640 1 -- 172.21.15.84:0/3571776737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8030d50 con 0x7fbfe40d09d0 2023-10-11T06:11:20.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7f33b8cc4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40c8c50 0x7f33b40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7f33b8cc4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b406e290 con 0x7f33b40b02f0 2023-10-11T06:11:20.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7f33b8cc4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b406e570 con 0x7f33b40b06c0 2023-10-11T06:11:20.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7f33b8cc4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40cae00 con 0x7f33b40c8c50 2023-10-11T06:11:20.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7f33b259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f33b40b02f0 0x7f33b415d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54736/0 (socket says 172.21.15.84:54736) 2023-10-11T06:11:20.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7f33b259c640 1 -- 172.21.15.84:0/1468116820 learned_addr learned my addr 172.21.15.84:0/1468116820 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:20.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f98a9b14640 1 Processor -- start 2023-10-11T06:11:20.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3571776737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 msgr2=0x7fbfe40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f826e59c640 1 -- 172.21.15.84:0/3967174629 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82700ccaf0 msgr2=0x7f82701f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f826e59c640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82700ccaf0 0x7f82701f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f826e59c640 1 -- 172.21.15.84:0/3967174629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82701f9a70 msgr2=0x7f82700af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f826e59c640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82701f9a70 0x7f82700af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f826e59c640 1 -- 172.21.15.84:0/3967174629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268009b30 con 0x7f82700cc6a0 2023-10-11T06:11:20.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7f826e59c640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82700cc6a0 0x7f8270164810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f827006e8f0 tx=0x7f8268004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.485+0000 7fbfe9aca640 1 --2- 172.21.15.84:0/3571776737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40b1640 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009870 tx=0x7fbfd802b9a0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.484+0000 7f98a9b14640 1 -- start start 2023-10-11T06:11:20.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7fb5297d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d9b80 0x7fb51c0389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f98a9b14640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f989c0e6c80 0x7f989c03be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7fb528fd6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb51c0502f0 0x7fb51c04b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53262/0 (socket says 172.21.15.84:53262) 2023-10-11T06:11:20.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7fb528fd6640 1 -- 172.21.15.84:0/458598264 learned_addr learned my addr 172.21.15.84:0/458598264 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7fb523fff640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c0d97b0 0x7fb51c0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f33b1d9b640 1 --2- 172.21.15.84:0/1468116820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40c8c50 0x7f33b40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fb528fd6640 1 -- 172.21.15.84:0/458598264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c0d97b0 msgr2=0x7fb51c0d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fb528fd6640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb51c0d97b0 0x7fb51c0d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fb528fd6640 1 -- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d9b80 msgr2=0x7fb51c0389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fb528fd6640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb51c0d9b80 0x7fb51c0389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fb528fd6640 1 -- 172.21.15.84:0/458598264 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51800b6a0 con 0x7fb51c0502f0 2023-10-11T06:11:20.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826803e450 con 0x7f82700cc6a0 2023-10-11T06:11:20.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826803e5b0 con 0x7f82700cc6a0 2023-10-11T06:11:20.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826803e940 con 0x7f82700cc6a0 2023-10-11T06:11:20.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82701fa060 con 0x7f82700cc6a0 2023-10-11T06:11:20.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82701642f0 con 0x7f82700cc6a0 2023-10-11T06:11:20.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3571776737 shutdown_connections 2023-10-11T06:11:20.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fbfe9aca640 1 --2- 172.21.15.84:0/3571776737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40b1640 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3571776737 >> 172.21.15.84:0/3571776737 conn(0x7fbfe4071cc0 msgr2=0x7fbfe409ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3571776737 shutdown_connections 2023-10-11T06:11:20.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f98a9b14640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0586b0 0x7f989c006c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f98a9b14640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f989c03aa70 0x7f989c03aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f98a9b14640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c03b430 con 0x7f989c03aa70 2023-10-11T06:11:20.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f98a9b14640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c056c30 con 0x7f989c0e6c80 2023-10-11T06:11:20.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.486+0000 7f98a9b14640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c053650 con 0x7f989c0586b0 2023-10-11T06:11:20.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f98a37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f989c0e6c80 0x7f989c03be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f98a37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f989c0e6c80 0x7f989c03be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:35318/0 (socket says 172.21.15.84:35318) 2023-10-11T06:11:20.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f98a37fe640 1 -- 172.21.15.84:0/1950863977 learned_addr learned my addr 172.21.15.84:0/1950863977 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f98a2ffd640 1 --2- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0586b0 0x7f989c006c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.487+0000 7f98a3fff640 1 --2- 172.21.15.84:0/1950863977 >> v2:172.21.15.84:3300/0 conn(0x7f989c03aa70 0x7f989c03aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.488+0000 7fb528fd6640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb51c0502f0 0x7fb51c04b8e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb51802e6c0 tx=0x7fb5180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.488+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518034b00 con 0x7fb51c0502f0 2023-10-11T06:11:20.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7fb529fd8640 1 -- 172.21.15.84:0/458598264 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c039520 con 0x7fb51c0502f0 2023-10-11T06:11:20.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7fb529fd8640 1 -- 172.21.15.84:0/458598264 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c039ae0 con 0x7fb51c0502f0 2023-10-11T06:11:20.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518034c60 con 0x7fb51c0502f0 2023-10-11T06:11:20.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518034b00 con 0x7fb51c0502f0 2023-10-11T06:11:20.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.488+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3571776737 wait complete. 2023-10-11T06:11:20.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.488+0000 7f33b1d9b640 1 -- 172.21.15.84:0/1468116820 >> v1:172.21.15.99:6789/0 conn(0x7f33b40b06c0 legacy=0x7f33b40713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.488+0000 7f33b1d9b640 1 -- 172.21.15.84:0/1468116820 >> v1:172.21.15.84:6789/0 conn(0x7f33b40b02f0 legacy=0x7f33b415d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.488+0000 7f33b1d9b640 1 -- 172.21.15.84:0/1468116820 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40caa20 con 0x7f33b40c8c50 2023-10-11T06:11:20.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7f98a37fe640 1 -- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0586b0 msgr2=0x7f989c006c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7f98a37fe640 1 --2- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0586b0 0x7f989c006c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7f98a37fe640 1 -- 172.21.15.84:0/1950863977 >> v2:172.21.15.84:3300/0 conn(0x7f989c03aa70 msgr2=0x7f989c03aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7f98a37fe640 1 --2- 172.21.15.84:0/1950863977 >> v2:172.21.15.84:3300/0 conn(0x7f989c03aa70 0x7f989c03aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.489+0000 7f98a37fe640 1 -- 172.21.15.84:0/1950863977 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0538c0 con 0x7f989c0e6c80 2023-10-11T06:11:20.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.490+0000 7fbfe9aca640 1 Processor -- start 2023-10-11T06:11:20.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.491+0000 7f98a3fff640 1 --2- 172.21.15.84:0/1950863977 >> v2:172.21.15.84:3300/0 conn(0x7f989c03aa70 0x7f989c03aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:20.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.491+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f827006e160 con 0x7f82700cc6a0 2023-10-11T06:11:20.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.491+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f826803eaa0 con 0x7f82700cc6a0 2023-10-11T06:11:20.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.491+0000 7fb529fd8640 1 -- 172.21.15.84:0/458598264 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb504000fc0 con 0x7fb51c0502f0 2023-10-11T06:11:20.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.492+0000 7fbfe9aca640 1 -- start start 2023-10-11T06:11:20.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.492+0000 7f98a37fe640 1 --2- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0e6c80 0x7f989c03be40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f989c03b730 tx=0x7f989002b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26cbc8e87eb7b7cd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.492+0000 7f33b1d9b640 1 --2- 172.21.15.84:0/1468116820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40c8c50 0x7f33b40cc680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009dc0 tx=0x7f33a402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9c3b358d76f30 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.492+0000 7f33b159a640 1 -- 172.21.15.84:0/1468116820 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a402cc40 con 0x7f33b40c8c50 2023-10-11T06:11:20.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.492+0000 7f98a27fc640 1 -- 172.21.15.84:0/1950863977 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890004230 con 0x7f989c0e6c80 2023-10-11T06:11:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7f33b159a640 1 -- 172.21.15.84:0/1468116820 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a402cda0 con 0x7f33b40c8c50 2023-10-11T06:11:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7f98a27fc640 1 -- 172.21.15.84:0/1950863977 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890004390 con 0x7f989c0e6c80 2023-10-11T06:11:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7f33b159a640 1 -- 172.21.15.84:0/1468116820 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4034660 con 0x7f33b40c8c50 2023-10-11T06:11:20.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7f98a27fc640 1 -- 172.21.15.84:0/1950863977 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890003810 con 0x7f989c0e6c80 2023-10-11T06:11:20.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7f825effd640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f825006a900 0x7f825006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7fbfe9aca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfe40cecb0 0x7fbfe40cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7fbfe9aca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.493+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82680abd90 con 0x7f82700cc6a0 2023-10-11T06:11:20.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f33b8cc4640 1 -- 172.21.15.84:0/1468116820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40c8c50 msgr2=0x7f33b40cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f33b8cc4640 1 --2- 172.21.15.84:0/1468116820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40c8c50 0x7f33b40cc680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009dc0 tx=0x7f33a402be90 comp rx=0 tx=0).stop 2023-10-11T06:11:20.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f826dd9b640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f825006a900 0x7f825006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fbfe2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f33b8cc4640 1 -- 172.21.15.84:0/1468116820 shutdown_connections 2023-10-11T06:11:20.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f33b8cc4640 1 --2- 172.21.15.84:0/1468116820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40c8c50 0x7f33b40cc680 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f33b8cc4640 1 -- 172.21.15.84:0/1468116820 >> 172.21.15.84:0/1468116820 conn(0x7f33b406bf30 msgr2=0x7f33b406c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fbfe2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53870/0 (socket says 172.21.15.84:53870) 2023-10-11T06:11:20.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3777440120 learned_addr learned my addr 172.21.15.84:0/3777440120 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f98a9b14640 1 -- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0e6c80 msgr2=0x7f989c03be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f98a9b14640 1 --2- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0e6c80 0x7f989c03be40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f989c03b730 tx=0x7f989002b880 comp rx=0 tx=0).stop 2023-10-11T06:11:20.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fbfe9aca640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfe40d3060 0x7fbfe40c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40c9600 con 0x7fbfe40d09d0 2023-10-11T06:11:20.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40cc080 con 0x7fbfe40cecb0 2023-10-11T06:11:20.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7f825effd640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7f82500731b0 0x7f82500755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40cb190 con 0x7fbfe40d3060 2023-10-11T06:11:20.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.492+0000 7fa79a42d640 1 Processor -- start 2023-10-11T06:11:20.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.492+0000 7fa79a42d640 1 -- start start 2023-10-11T06:11:20.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7fbfe3fff640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfe40d3060 0x7fbfe40c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fa79a42d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 0x7fa7940cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fa79a42d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940b0120 con 0x7fa794158910 2023-10-11T06:11:20.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fa79a42d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940b0400 con 0x7fa7940ce010 2023-10-11T06:11:20.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7fa79a42d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940b06e0 con 0x7fa7940c9420 2023-10-11T06:11:20.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7fa7989ca640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa7940ce010 0x7fa79415bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:53254/0 (socket says 172.21.15.84:53254) 2023-10-11T06:11:20.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7fa7989ca640 1 -- 172.21.15.84:0/2236194099 learned_addr learned my addr 172.21.15.84:0/2236194099 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:20.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7f826dd9b640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f825006a900 0x7f825006cdb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82700c8a90 tx=0x7f82600091c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7fa793fff640 1 --2- 172.21.15.84:0/2236194099 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 0x7fa7940cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 --> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8250075c70 con 0x7f82500731b0 2023-10-11T06:11:20.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82680af020 con 0x7f82700cc6a0 2023-10-11T06:11:20.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7fbfe37fe640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfe40cecb0 0x7fbfe40cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.495+0000 7f826ed9d640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7f82500731b0 0x7f82500755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fa792ffd640 1 -- 172.21.15.84:0/2236194099 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976323235 0 0) 0x7fa7940b0400 con 0x7fa7940ce010 2023-10-11T06:11:20.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fa792ffd640 1 -- 172.21.15.84:0/2236194099 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa77c003520 con 0x7fa7940ce010 2023-10-11T06:11:20.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfe40d3060 msgr2=0x7fbfe40c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fbfe2ffd640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfe40d3060 0x7fbfe40c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fa792ffd640 1 -- 172.21.15.84:0/2236194099 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211696143 0 0) 0x7fa7940b0120 con 0x7fa794158910 2023-10-11T06:11:20.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fa792ffd640 1 -- 172.21.15.84:0/2236194099 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7940b0400 con 0x7fa794158910 2023-10-11T06:11:20.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfe40cecb0 msgr2=0x7fbfe40cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fbfe2ffd640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfe40cecb0 0x7fbfe40cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fbfe2ffd640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80094d0 con 0x7fbfe40d09d0 2023-10-11T06:11:20.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fa793fff640 1 -- 172.21.15.84:0/2236194099 >> v1:172.21.15.99:6789/0 conn(0x7fa7940ce010 legacy=0x7fa79415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fa793fff640 1 -- 172.21.15.84:0/2236194099 >> v1:172.21.15.84:6789/0 conn(0x7fa794158910 legacy=0x7fa7940cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fa793fff640 1 -- 172.21.15.84:0/2236194099 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940b09c0 con 0x7fa7940c9420 2023-10-11T06:11:20.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb518040440 con 0x7fb51c0502f0 2023-10-11T06:11:20.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.497+0000 7fbfe3fff640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfe40d3060 0x7fbfe40c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.497+0000 7f826ed9d640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7f82500731b0 0x7f82500755c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.497+0000 7fbfe2ffd640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40c78e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009870 tx=0x7fbfd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7f98a9b14640 1 -- 172.21.15.84:0/1950863977 shutdown_connections 2023-10-11T06:11:20.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7f98a9b14640 1 --2- 172.21.15.84:0/1950863977 >> v2:172.21.15.84:3300/0 conn(0x7f989c03aa70 0x7f989c03aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7f98a9b14640 1 --2- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0586b0 0x7f989c006c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7f98a9b14640 1 --2- 172.21.15.84:0/1950863977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0e6c80 0x7f989c03be40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.496+0000 7f98a9b14640 1 -- 172.21.15.84:0/1950863977 >> 172.21.15.84:0/1950863977 conn(0x7f989c055a70 msgr2=0x7f989c055e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.494+0000 7f33b8cc4640 1 -- 172.21.15.84:0/1468116820 shutdown_connections 2023-10-11T06:11:20.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.497+0000 7f33b8cc4640 1 -- 172.21.15.84:0/1468116820 wait complete. 2023-10-11T06:11:20.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.498+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd803e040 con 0x7fbfe40d09d0 2023-10-11T06:11:20.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.498+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8039da0 con 0x7fbfe40d09d0 2023-10-11T06:11:20.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.498+0000 7fb5217fa640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb50806a8f0 0x7fb50806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.498+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8044630 con 0x7fbfe40d09d0 2023-10-11T06:11:20.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.498+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== osd.3 v2:172.21.15.84:6817/87864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8250075c70 con 0x7f82500731b0 2023-10-11T06:11:20.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.498+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe40cb470 con 0x7fbfe40d09d0 2023-10-11T06:11:20.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.498+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb5180ac430 con 0x7fb51c0502f0 2023-10-11T06:11:20.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.499+0000 7fb5217fa640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7fb5080730a0 0x7fb5080754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.499+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 --> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb508075b60 con 0x7fb5080730a0 2023-10-11T06:11:20.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.499+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb518034b00 con 0x7fb51c0502f0 2023-10-11T06:11:20.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.499+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe40cba30 con 0x7fbfe40d09d0 2023-10-11T06:11:20.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.500+0000 7f33b8cc4640 1 Processor -- start 2023-10-11T06:11:20.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.499+0000 7fb5297d7640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7fb5080730a0 0x7fb5080754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.500+0000 7f33b8cc4640 1 -- start start 2023-10-11T06:11:20.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.500+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfe406b560 con 0x7fbfe40d09d0 2023-10-11T06:11:20.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.500+0000 7fb5297d7640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7fb5080730a0 0x7fb5080754b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.500+0000 7fb523fff640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb50806a8f0 0x7fb50806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.501+0000 7f98a9b14640 1 -- 172.21.15.84:0/1950863977 shutdown_connections 2023-10-11T06:11:20.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.501+0000 7f98a9b14640 1 -- 172.21.15.84:0/1950863977 wait complete. 2023-10-11T06:11:20.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.501+0000 7fa793fff640 1 --2- 172.21.15.84:0/2236194099 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 0x7fa7940cb100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7880097d0 tx=0x7fa78802bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d0e8eaff93b06fc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.501+0000 7fa792ffd640 1 -- 172.21.15.84:0/2236194099 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78802c920 con 0x7fa7940c9420 2023-10-11T06:11:20.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.501+0000 7fa792ffd640 1 -- 172.21.15.84:0/2236194099 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78802ca80 con 0x7fa7940c9420 2023-10-11T06:11:20.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.502+0000 7f33b8cc4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40b02f0 0x7f33b41f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.502+0000 7f33b8cc4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b40b06c0 0x7f33b415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.502+0000 7fa79a42d640 1 -- 172.21.15.84:0/2236194099 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 msgr2=0x7fa7940cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.502+0000 7fa79a42d640 1 --2- 172.21.15.84:0/2236194099 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 0x7fa7940cb100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7880097d0 tx=0x7fa78802bb70 comp rx=0 tx=0).stop 2023-10-11T06:11:20.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7f33b8cc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b40c8c50 0x7f33b41f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7f33b8cc4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c41b0 con 0x7f33b40c8c50 2023-10-11T06:11:20.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7fa79a42d640 1 -- 172.21.15.84:0/2236194099 shutdown_connections 2023-10-11T06:11:20.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7fa79a42d640 1 --2- 172.21.15.84:0/2236194099 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 0x7fa7940cb100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7fa79a42d640 1 -- 172.21.15.84:0/2236194099 >> 172.21.15.84:0/2236194099 conn(0x7fa79406bf30 msgr2=0x7fa79406c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7f33b1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b40b06c0 0x7f33b415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7f33b1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b40b06c0 0x7f33b415ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:35328/0 (socket says 172.21.15.84:35328) 2023-10-11T06:11:20.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7f33b1d9b640 1 -- 172.21.15.84:0/3945874581 learned_addr learned my addr 172.21.15.84:0/3945874581 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41f5d80 con 0x7f33b40b06c0 2023-10-11T06:11:20.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.504+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c3600 con 0x7f33b40b02f0 2023-10-11T06:11:20.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.504+0000 7f98a9b14640 1 Processor -- start 2023-10-11T06:11:20.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.504+0000 7f98a9b14640 1 -- start start 2023-10-11T06:11:20.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.503+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbfd804d3b0 con 0x7fbfe40d09d0 2023-10-11T06:11:20.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.504+0000 7f33b2d9d640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b40c8c50 0x7f33b41f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.505+0000 7f33b259c640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40b02f0 0x7f33b41f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.505+0000 7f33b1d9b640 1 -- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40b02f0 msgr2=0x7f33b41f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.505+0000 7fbfcffff640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbfc806a880 0x7fbfc806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.505+0000 7f33b1d9b640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40b02f0 0x7f33b41f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.505+0000 7f33b1d9b640 1 -- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b40c8c50 msgr2=0x7f33b41f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.502+0000 7fb523fff640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb50806a8f0 0x7fb50806cda0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb51c03ba60 tx=0x7fb510009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.502+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== osd.2 v2:172.21.15.84:6816/87861 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb508075b60 con 0x7fb5080730a0 2023-10-11T06:11:20.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7fbfe37fe640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbfc806a880 0x7fbfc806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfd807a850 con 0x7fbfe40d09d0 2023-10-11T06:11:20.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7f98a9b14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c03aa70 0x7f989c0e7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7fa79a42d640 1 -- 172.21.15.84:0/2236194099 shutdown_connections 2023-10-11T06:11:20.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7fa79a42d640 1 -- 172.21.15.84:0/2236194099 wait complete. 2023-10-11T06:11:20.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 --> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31bc002cb0 con 0x7f31c4073100 2023-10-11T06:11:20.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7f33b1d9b640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b40c8c50 0x7f33b41f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7f33b1d9b640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4009a70 con 0x7f33b40b06c0 2023-10-11T06:11:20.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7f33b259c640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33b40b02f0 0x7f33b41f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:11:20.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7f33b2d9d640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33b40c8c50 0x7f33b41f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:20.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7f98a9b14640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0586b0 0x7f989c0ea590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f98a9b14640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0e6c80 0x7f989c0eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f98a9b14640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0e9430 con 0x7f989c03aa70 2023-10-11T06:11:20.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f31cbfff640 1 -- 172.21.15.84:0/2569884811 <== osd.0 v2:172.21.15.84:6802/87856 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f31bc002cb0 con 0x7f31c4073100 2023-10-11T06:11:20.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f98a9b14640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0e9660 con 0x7f989c0586b0 2023-10-11T06:11:20.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f98a9b14640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0e9940 con 0x7f989c0e6c80 2023-10-11T06:11:20.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7fbfcffff640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7fbfc8073d40 0x7fbfc80761d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.506+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc80768a0 con 0x7fbfc8073d40 2023-10-11T06:11:20.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7fbfe3fff640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7fbfc8073d40 0x7fbfc80761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd807a620 con 0x7fbfe40d09d0 2023-10-11T06:11:20.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c03aa70 0x7f989c0e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c03aa70 0x7f989c0e7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53880/0 (socket says 172.21.15.84:53880) 2023-10-11T06:11:20.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f98a37fe640 1 -- 172.21.15.84:0/4002559434 learned_addr learned my addr 172.21.15.84:0/4002559434 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.507+0000 7f33b1d9b640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b40b06c0 0x7f33b415ae40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000f80 tx=0x7f33a4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f98a3fff640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0e6c80 0x7f989c0eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f98a2ffd640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0586b0 0x7f989c0ea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f31c4073100 msgr2=0x7f31c4075510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f31e5ca7640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f31c4073100 0x7f31c4075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4040650 con 0x7f33b40b06c0 2023-10-11T06:11:20.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b415b380 con 0x7f33b40b06c0 2023-10-11T06:11:20.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f31c406a850 msgr2=0x7f31c406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f31e5ca7640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f31c406a850 0x7f31c406cd00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f31d804b700 tx=0x7f31cc006b30 comp rx=0 tx=0).stop 2023-10-11T06:11:20.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f98a37fe640 1 -- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0e6c80 msgr2=0x7f989c0eaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f98a37fe640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f989c0e6c80 0x7f989c0eaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f98a37fe640 1 -- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0586b0 msgr2=0x7f989c0ea590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d8052cc0 msgr2=0x7f31d81853a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f31e5ca7640 1 --2- 172.21.15.84:0/2569884811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d8052cc0 0x7f31d81853a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31e002c840 tx=0x7f31e0068140 comp rx=0 tx=0).stop 2023-10-11T06:11:20.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f98a37fe640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0586b0 0x7f989c0ea590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f31dffff640 1 -- 172.21.15.84:0/2569884811 reap_dead start 2023-10-11T06:11:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f98a37fe640 1 -- 172.21.15.84:0/4002559434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890009460 con 0x7f989c03aa70 2023-10-11T06:11:20.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7fbfe37fe640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbfc806a880 0x7fbfc806cd30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbfe41f9110 tx=0x7fbfdc009310 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.508+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b415b5b0 con 0x7f33b40b06c0 2023-10-11T06:11:20.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7fa79a42d640 1 Processor -- start 2023-10-11T06:11:20.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f98a2ffd640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f989c0586b0 0x7f989c0ea590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:20.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f98a37fe640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c03aa70 0x7f989c0e7260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f989002b8c0 tx=0x7f98900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a40407b0 con 0x7f33b40b06c0 2023-10-11T06:11:20.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.510+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890006e30 con 0x7f989c03aa70 2023-10-11T06:11:20.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.510+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890003e50 con 0x7f989c03aa70 2023-10-11T06:11:20.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.510+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890006260 con 0x7f989c03aa70 2023-10-11T06:11:20.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.510+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4040b60 con 0x7f33b40b06c0 2023-10-11T06:11:20.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.510+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c0e9c20 con 0x7f989c03aa70 2023-10-11T06:11:20.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.510+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c0e77a0 con 0x7f989c03aa70 2023-10-11T06:11:20.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.509+0000 7fa79a42d640 1 -- start start 2023-10-11T06:11:20.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.511+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3398000fc0 con 0x7f33b40b06c0 2023-10-11T06:11:20.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.511+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 shutdown_connections 2023-10-11T06:11:20.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.511+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 >> 172.21.15.84:0/2569884811 conn(0x7f31d8051660 msgr2=0x7f31d8033a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.511+0000 7fbfe3fff640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7fbfc8073d40 0x7fbfc80761d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.511+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 shutdown_connections 2023-10-11T06:11:20.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.512+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f989c007180 con 0x7f989c03aa70 2023-10-11T06:11:20.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.512+0000 7f31e5ca7640 1 -- 172.21.15.84:0/2569884811 wait complete. 2023-10-11T06:11:20.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.512+0000 7fa79a42d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 0x7fa7940c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.512+0000 7fa79a42d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940ce010 0x7fa794070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.512+0000 7fa79a42d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa794158910 0x7fa794071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fa79a42d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79406fe30 con 0x7fa7940ce010 2023-10-11T06:11:20.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fa79a42d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794070040 con 0x7fa794158910 2023-10-11T06:11:20.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fa79a42d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa794070320 con 0x7fa7940c9420 2023-10-11T06:11:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fa7937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940ce010 0x7fa794070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fa7937fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940ce010 0x7fa794070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53888/0 (socket says 172.21.15.84:53888) 2023-10-11T06:11:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fa7937fe640 1 -- 172.21.15.84:0/2334427519 learned_addr learned my addr 172.21.15.84:0/2334427519 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fa7989ca640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa794158910 0x7fa794071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.513+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== osd.4 v2:172.21.15.99:6800/88014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbfc80768a0 con 0x7fbfc8073d40 2023-10-11T06:11:20.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.514+0000 7fa7937fe640 1 -- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 msgr2=0x7fa7940c56d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-11T06:11:20.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.514+0000 7fa7937fe640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7940c9420 0x7fa7940c56d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.514+0000 7fa7937fe640 1 -- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa794158910 msgr2=0x7fa794071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.514+0000 7fa7937fe640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa794158910 0x7fa794071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.514+0000 7fa7937fe640 1 -- 172.21.15.84:0/2334427519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa788009400 con 0x7fa7940ce010 2023-10-11T06:11:20.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.514+0000 7fa7937fe640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940ce010 0x7fa794070e00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c7a0 tx=0x7fa78c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.512+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f33a4040cc0 con 0x7f33b40b06c0 2023-10-11T06:11:20.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7f33a2ffd640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f339406a930 0x7f339406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c003b70 con 0x7fa7940ce010 2023-10-11T06:11:20.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c003cd0 con 0x7fa7940ce010 2023-10-11T06:11:20.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00aa70 con 0x7fa7940ce010 2023-10-11T06:11:20.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33a40abe10 con 0x7f33b40b06c0 2023-10-11T06:11:20.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7f33a2ffd640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f33940731e0 0x7f33940755f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7f33b259c640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f339406a930 0x7f339406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.516+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3394075ca0 con 0x7f33940731e0 2023-10-11T06:11:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.514+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f98900063c0 con 0x7f989c03aa70 2023-10-11T06:11:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7f988ffff640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f988806a830 0x7f988806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.515+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f98900ab080 con 0x7f989c03aa70 2023-10-11T06:11:20.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.516+0000 7f988ffff640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7f98880730e0 0x7f98880754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.516+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 --> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9888075ba0 con 0x7f98880730e0 2023-10-11T06:11:20.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.516+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98900ad580 con 0x7f989c03aa70 2023-10-11T06:11:20.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.516+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33a40ae430 con 0x7f33b40b06c0 2023-10-11T06:11:20.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.516+0000 7f98a2ffd640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f988806a830 0x7f988806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.517+0000 7f33b259c640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f339406a930 0x7f339406cde0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005880 tx=0x7f33ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.517+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa794070600 con 0x7fa7940ce010 2023-10-11T06:11:20.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.517+0000 7f98a3fff640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7f98880730e0 0x7f98880754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.517+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7940c5c10 con 0x7fa7940ce010 2023-10-11T06:11:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.518+0000 7f33b2d9d640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f33940731e0 0x7f33940755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.518+0000 7f98a3fff640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7f98880730e0 0x7f98880754f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.518+0000 7f98a2ffd640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f988806a830 0x7f988806cce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9898006fd0 tx=0x7f9898008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.518+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 --> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e40006d10 con 0x7f8e2c0731b0 2023-10-11T06:11:20.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.519+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7940701b0 con 0x7fa7940ce010 2023-10-11T06:11:20.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.520+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa78c00abd0 con 0x7fa7940ce010 2023-10-11T06:11:20.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.520+0000 7f8e3e7fc640 1 -- 172.21.15.84:0/1526938110 <== osd.1 v2:172.21.15.84:6810/87859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e38004c30 con 0x7f8e2c0731b0 2023-10-11T06:11:20.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.521+0000 7fa790ff9640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa78006a900 0x7fa78006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.521+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa78c088ed0 con 0x7fa7940ce010 2023-10-11T06:11:20.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.521+0000 7fa790ff9640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7fa780073130 0x7fa780075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.521+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 --> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa780075bf0 con 0x7fa780073130 2023-10-11T06:11:20.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.521+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa78c00ae80 con 0x7fa7940ce010 2023-10-11T06:11:20.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.521+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== osd.6 v2:172.21.15.99:6820/88045 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9888075ba0 con 0x7f98880730e0 2023-10-11T06:11:20.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.520+0000 7f33b2d9d640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f33940731e0 0x7f33940755f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.521+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 --> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82700caf40 con 0x7f82500731b0 2023-10-11T06:11:20.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.522+0000 7fa793fff640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa78006a900 0x7fa78006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.522+0000 7fa7989ca640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7fa780073130 0x7fa780075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.522+0000 7f825effd640 1 -- 172.21.15.84:0/3967174629 <== osd.3 v2:172.21.15.84:6817/87864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8264009200 con 0x7f82500731b0 2023-10-11T06:11:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f8e2c0731b0 msgr2=0x7f8e2c0755c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7f8e2bfff640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f8e2c0731b0 0x7f8e2c0755c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7fa793fff640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa78006a900 0x7fa78006cdb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa788036000 tx=0x7fa788030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7f82500731b0 msgr2=0x7f82500755c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7f8274f16640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7f82500731b0 0x7f82500755c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== osd.5 v2:172.21.15.99:6808/88015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3394075ca0 con 0x7f33940731e0 2023-10-11T06:11:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f825006a900 msgr2=0x7f825006cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.523+0000 7f8274f16640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f825006a900 0x7f825006cdb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82700c8a90 tx=0x7f82600091c0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82700cc6a0 msgr2=0x7f8270164810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f8274f16640 1 --2- 172.21.15.84:0/3967174629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82700cc6a0 0x7f8270164810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f827006e8f0 tx=0x7f8268004a40 comp rx=0 tx=0).stop 2023-10-11T06:11:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f826ed9d640 1 -- 172.21.15.84:0/3967174629 reap_dead start 2023-10-11T06:11:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7fa7989ca640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7fa780073130 0x7fa780075540 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2c06a900 msgr2=0x7f8e2c06cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f8e2bfff640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2c06a900 0x7f8e2c06cdb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e4804f820 tx=0x7f8e4806e040 comp rx=0 tx=0).stop 2023-10-11T06:11:20.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e400cc810 msgr2=0x7f8e400ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f8e2bfff640 1 --2- 172.21.15.84:0/1526938110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e400cc810 0x7f8e400ccc80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e44032000 tx=0x7f8e44004730 comp rx=0 tx=0).stop 2023-10-11T06:11:20.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.524+0000 7f8e4e4b8640 1 -- 172.21.15.84:0/1526938110 reap_dead start 2023-10-11T06:11:20.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.525+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 shutdown_connections 2023-10-11T06:11:20.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.525+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 >> 172.21.15.84:0/3967174629 conn(0x7f827006bf30 msgr2=0x7f8270160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.526+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 shutdown_connections 2023-10-11T06:11:20.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.526+0000 7f8274f16640 1 -- 172.21.15.84:0/3967174629 wait complete. 2023-10-11T06:11:20.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.526+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 shutdown_connections 2023-10-11T06:11:20.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.526+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 >> 172.21.15.84:0/1526938110 conn(0x7f8e40051660 msgr2=0x7f8e4004aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.526+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== osd.7 v2:172.21.15.99:6816/88032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa780075bf0 con 0x7fa780073130 2023-10-11T06:11:20.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.527+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 shutdown_connections 2023-10-11T06:11:20.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.527+0000 7f8e2bfff640 1 -- 172.21.15.84:0/1526938110 wait complete. 2023-10-11T06:11:20.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.533+0000 7fb529fd8640 1 -- 172.21.15.84:0/458598264 --> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb504002cb0 con 0x7fb5080730a0 2023-10-11T06:11:20.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.534+0000 7fb5217fa640 1 -- 172.21.15.84:0/458598264 <== osd.2 v2:172.21.15.84:6816/87861 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb504002cb0 con 0x7fb5080730a0 2023-10-11T06:11:20.619 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-10-11T06:11:20.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-11T06:11:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.537+0000 7fb50effd640 1 -- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7fb5080730a0 msgr2=0x7fb5080754b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.537+0000 7fb50effd640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7fb5080730a0 0x7fb5080754b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.537+0000 7fb50effd640 1 -- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb50806a8f0 msgr2=0x7fb50806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.537+0000 7fb50effd640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb50806a8f0 0x7fb50806cda0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb51c03ba60 tx=0x7fb510009040 comp rx=0 tx=0).stop 2023-10-11T06:11:20.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.537+0000 7fb50effd640 1 -- 172.21.15.84:0/458598264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb51c0502f0 msgr2=0x7fb51c04b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.537+0000 7fb50effd640 1 --2- 172.21.15.84:0/458598264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb51c0502f0 0x7fb51c04b8e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb51802e6c0 tx=0x7fb5180046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.541+0000 7fb5297d7640 1 -- 172.21.15.84:0/458598264 reap_dead start 2023-10-11T06:11:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.541+0000 7fb50effd640 1 -- 172.21.15.84:0/458598264 shutdown_connections 2023-10-11T06:11:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.541+0000 7fb50effd640 1 -- 172.21.15.84:0/458598264 >> 172.21.15.84:0/458598264 conn(0x7fb51c052640 msgr2=0x7fb51c0dff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.541+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 --> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbfe406b280 con 0x7fbfc8073d40 2023-10-11T06:11:20.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.541+0000 7fb50effd640 1 -- 172.21.15.84:0/458598264 shutdown_connections 2023-10-11T06:11:20.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.541+0000 7fb50effd640 1 -- 172.21.15.84:0/458598264 wait complete. 2023-10-11T06:11:20.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.542+0000 7fbfcffff640 1 -- 172.21.15.84:0/3777440120 <== osd.4 v2:172.21.15.99:6800/88014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbfd0001790 con 0x7fbfc8073d40 2023-10-11T06:11:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.543+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7fbfc8073d40 msgr2=0x7fbfc80761d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe9aca640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7fbfc8073d40 0x7fbfc80761d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbfc806a880 msgr2=0x7fbfc806cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe9aca640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbfc806a880 0x7fbfc806cd30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbfe41f9110 tx=0x7fbfdc009310 comp rx=0 tx=0).stop 2023-10-11T06:11:20.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 msgr2=0x7fbfe40c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe9aca640 1 --2- 172.21.15.84:0/3777440120 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe40d09d0 0x7fbfe40c78e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009870 tx=0x7fbfd8004730 comp rx=0 tx=0).stop 2023-10-11T06:11:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe3fff640 1 -- 172.21.15.84:0/3777440120 reap_dead start 2023-10-11T06:11:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 shutdown_connections 2023-10-11T06:11:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.544+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 >> 172.21.15.84:0/3777440120 conn(0x7fbfe4071cc0 msgr2=0x7fbfe4072e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.545+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 shutdown_connections 2023-10-11T06:11:20.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.545+0000 7fbfe9aca640 1 -- 172.21.15.84:0/3777440120 wait complete. 2023-10-11T06:11:20.627 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:20.628 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-11T06:11:20.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.548+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 --> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3398002cb0 con 0x7f33940731e0 2023-10-11T06:11:20.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.550+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 --> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f989c053ba0 con 0x7f98880730e0 2023-10-11T06:11:20.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.550+0000 7f33a2ffd640 1 -- 172.21.15.84:0/3945874581 <== osd.5 v2:172.21.15.99:6808/88015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3398002cb0 con 0x7f33940731e0 2023-10-11T06:11:20.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.551+0000 7f988ffff640 1 -- 172.21.15.84:0/4002559434 <== osd.6 v2:172.21.15.99:6820/88045 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f98a404e040 con 0x7f98880730e0 2023-10-11T06:11:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f33940731e0 msgr2=0x7f33940755f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f33940731e0 0x7f33940755f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7f98880730e0 msgr2=0x7f98880754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f98a9b14640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7f98880730e0 0x7f98880754f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.631 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-10-11T06:11:20.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-11T06:11:20.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f988806a830 msgr2=0x7f988806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f339406a930 msgr2=0x7f339406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f339406a930 0x7f339406cde0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005880 tx=0x7f33ac009040 comp rx=0 tx=0).stop 2023-10-11T06:11:20.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f98a9b14640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f988806a830 0x7f988806cce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9898006fd0 tx=0x7f9898008040 comp rx=0 tx=0).stop 2023-10-11T06:11:20.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b40b06c0 msgr2=0x7f33b415ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 --2- 172.21.15.84:0/3945874581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33b40b06c0 0x7f33b415ae40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000f80 tx=0x7f33a4004a40 comp rx=0 tx=0).stop 2023-10-11T06:11:20.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c03aa70 msgr2=0x7f989c0e7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b2d9d640 1 -- 172.21.15.84:0/3945874581 reap_dead start 2023-10-11T06:11:20.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f98a9b14640 1 --2- 172.21.15.84:0/4002559434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f989c03aa70 0x7f989c0e7260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f989002b8c0 tx=0x7f98900046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f98a3fff640 1 -- 172.21.15.84:0/4002559434 reap_dead start 2023-10-11T06:11:20.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 shutdown_connections 2023-10-11T06:11:20.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.552+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 >> 172.21.15.84:0/3945874581 conn(0x7f33b406bf30 msgr2=0x7f33b40ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.553+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 shutdown_connections 2023-10-11T06:11:20.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.553+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 >> 172.21.15.84:0/4002559434 conn(0x7f989c055a70 msgr2=0x7f989c037f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.553+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 shutdown_connections 2023-10-11T06:11:20.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.553+0000 7f98a9b14640 1 -- 172.21.15.84:0/4002559434 wait complete. 2023-10-11T06:11:20.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.554+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 shutdown_connections 2023-10-11T06:11:20.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.556+0000 7f33b8cc4640 1 -- 172.21.15.84:0/3945874581 wait complete. 2023-10-11T06:11:20.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.561+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 --> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7940c97f0 con 0x7fa780073130 2023-10-11T06:11:20.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.562+0000 7fa790ff9640 1 -- 172.21.15.84:0/2334427519 <== osd.7 v2:172.21.15.99:6816/88032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa78400b930 con 0x7fa780073130 2023-10-11T06:11:20.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.563+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7fa780073130 msgr2=0x7fa780075540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.563+0000 7fa79a42d640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7fa780073130 0x7fa780075540 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.563+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa78006a900 msgr2=0x7fa78006cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.563+0000 7fa79a42d640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa78006a900 0x7fa78006cdb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa788036000 tx=0x7fa788030000 comp rx=0 tx=0).stop 2023-10-11T06:11:20.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.563+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940ce010 msgr2=0x7fa794070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.563+0000 7fa79a42d640 1 --2- 172.21.15.84:0/2334427519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7940ce010 0x7fa794070e00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c7a0 tx=0x7fa78c00cc60 comp rx=0 tx=0).stop 2023-10-11T06:11:20.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.563+0000 7fa7989ca640 1 -- 172.21.15.84:0/2334427519 reap_dead start 2023-10-11T06:11:20.641 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:20.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-11T06:11:20.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.564+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 shutdown_connections 2023-10-11T06:11:20.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.564+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 >> 172.21.15.84:0/2334427519 conn(0x7fa79406bf30 msgr2=0x7fa7940ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.564+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 shutdown_connections 2023-10-11T06:11:20.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.564+0000 7fa79a42d640 1 -- 172.21.15.84:0/2334427519 wait complete. 2023-10-11T06:11:20.643 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:20.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-11T06:11:20.644 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:20.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-11T06:11:20.646 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-10-11T06:11:20.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-11T06:11:20.647 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:20.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-11T06:11:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.962+0000 7f5aaa1ef640 1 Processor -- start 2023-10-11T06:11:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.962+0000 7f5aaa1ef640 1 -- start start 2023-10-11T06:11:20.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.964+0000 7f5aaa1ef640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5a9c050d60 0x7f5a9c101320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.964+0000 7f5aaa1ef640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a9c036630 0x7f5a9c036a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.964+0000 7f5aaa1ef640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c101860 con 0x7f5a9c0fa7b0 2023-10-11T06:11:20.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.964+0000 7f5aaa1ef640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c04ff50 con 0x7f5a9c050d60 2023-10-11T06:11:20.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.964+0000 7f5aaa1ef640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c04f920 con 0x7f5a9c036630 2023-10-11T06:11:20.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.965+0000 7f5aa89ec640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5a9c050d60 0x7f5a9c101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.965+0000 7f5aa89ec640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5a9c050d60 0x7f5a9c101320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53904/0 (socket says 172.21.15.84:53904) 2023-10-11T06:11:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.965+0000 7f5aa89ec640 1 -- 172.21.15.84:0/91881450 learned_addr learned my addr 172.21.15.84:0/91881450 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.965+0000 7f5aa99ee640 1 --2- 172.21.15.84:0/91881450 >> v2:172.21.15.99:3300/0 conn(0x7f5a9c036630 0x7f5a9c036a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.966+0000 7f5aa89ec640 1 -- 172.21.15.84:0/91881450 >> v2:172.21.15.99:3300/0 conn(0x7f5a9c036630 msgr2=0x7f5a9c036a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.966+0000 7f5aa89ec640 1 --2- 172.21.15.84:0/91881450 >> v2:172.21.15.99:3300/0 conn(0x7f5a9c036630 0x7f5a9c036a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.966+0000 7f5aa89ec640 1 -- 172.21.15.84:0/91881450 >> v1:172.21.15.99:6789/0 conn(0x7f5a9c0fa7b0 legacy=0x7f5a9c006820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-11T06:11:20.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.966+0000 7f5aa89ec640 1 -- 172.21.15.84:0/91881450 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c036f70 con 0x7f5a9c050d60 2023-10-11T06:11:20.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.967+0000 7f5aa89ec640 1 --2- 172.21.15.84:0/91881450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c101320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0486b0 tx=0x7f5a940344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d89be84a655fb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.968+0000 7f5a9bfff640 1 -- 172.21.15.84:0/91881450 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9403b600 con 0x7f5a9c050d60 2023-10-11T06:11:20.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.968+0000 7f5a9bfff640 1 -- 172.21.15.84:0/91881450 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a94037050 con 0x7f5a9c050d60 2023-10-11T06:11:20.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.968+0000 7f5a9bfff640 1 -- 172.21.15.84:0/91881450 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9403b980 con 0x7f5a9c050d60 2023-10-11T06:11:20.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.969+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/91881450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 msgr2=0x7f5a9c101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.969+0000 7f5aaa1ef640 1 --2- 172.21.15.84:0/91881450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c101320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0486b0 tx=0x7f5a940344c0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f7388e0e640 1 Processor -- start 2023-10-11T06:11:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f7388e0e640 1 -- start start 2023-10-11T06:11:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.975+0000 7f7388e0e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 0x7f73840b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.975+0000 7f7388e0e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73840d3060 0x7f73840d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.975+0000 7f7388e0e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73840b1e40 con 0x7f73840d0800 2023-10-11T06:11:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.975+0000 7f7388e0e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840d0c80 con 0x7f73840d3060 2023-10-11T06:11:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.975+0000 7f7388e0e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738415c770 con 0x7f73840cecb0 2023-10-11T06:11:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/91881450 shutdown_connections 2023-10-11T06:11:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f5aaa1ef640 1 --2- 172.21.15.84:0/91881450 >> v2:172.21.15.99:3300/0 conn(0x7f5a9c036630 0x7f5a9c036a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f5aaa1ef640 1 --2- 172.21.15.84:0/91881450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c101320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/91881450 >> 172.21.15.84:0/91881450 conn(0x7f5a9c051630 msgr2=0x7f5a9c051a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:20.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.976+0000 7f7382d9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f73840d0800 0x7f73840cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:53274/0 (socket says 172.21.15.84:53274) 2023-10-11T06:11:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/91881450 shutdown_connections 2023-10-11T06:11:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.973+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/91881450 wait complete. 2023-10-11T06:11:20.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.976+0000 7f7382d9d640 1 -- 172.21.15.84:0/2949555621 learned_addr learned my addr 172.21.15.84:0/2949555621 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.976+0000 7f5aaa1ef640 1 Processor -- start 2023-10-11T06:11:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.976+0000 7f5aaa1ef640 1 -- start start 2023-10-11T06:11:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.977+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488402631 0 0) 0x7f73840b1e40 con 0x7f73840d0800 2023-10-11T06:11:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f5aaa1ef640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a9c036630 0x7f5a9c0f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f5aaa1ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c0f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f5aaa1ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a9c0fa7b0 0x7f5a9c0f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f5aaa1ef640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0fab80 con 0x7f5a9c050d60 2023-10-11T06:11:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f5aaa1ef640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0f8f70 con 0x7f5a9c0fa7b0 2023-10-11T06:11:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f5aaa1ef640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0f8680 con 0x7f5a9c036630 2023-10-11T06:11:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f736c003520 con 0x7f73840d0800 2023-10-11T06:11:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f7381d9b640 1 --2- 172.21.15.84:0/2949555621 >> v2:172.21.15.99:3300/0 conn(0x7f73840d3060 0x7f73840d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f738259c640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 0x7f73840b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 <== mon.1 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 325625520 0 0) 0x7f736c003520 con 0x7f73840d0800 2023-10-11T06:11:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.978+0000 7f5aa91ed640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a9c036630 0x7f5a9c0f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c0f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c0f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53906/0 (socket says 172.21.15.84:53906) 2023-10-11T06:11:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 -- 172.21.15.84:0/1879888966 learned_addr learned my addr 172.21.15.84:0/1879888966 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 msgr2=0x7f73840b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 0x7f73840b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa99ee640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a9c0fa7b0 0x7f5a9c0f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 >> v2:172.21.15.99:3300/0 conn(0x7f73840d3060 msgr2=0x7f73840d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 --2- 172.21.15.84:0/2949555621 >> v2:172.21.15.99:3300/0 conn(0x7f73840d3060 0x7f73840d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738406acc0 con 0x7f73840d0800 2023-10-11T06:11:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 -- 172.21.15.84:0/1879888966 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a9c036630 msgr2=0x7f5a9c0f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a9c036630 0x7f5a9c0f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 -- 172.21.15.84:0/1879888966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a9c0fa7b0 msgr2=0x7f5a9c0f06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a9c0fa7b0 0x7f5a9c0f06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f5aa89ec640 1 -- 172.21.15.84:0/1879888966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a94008550 con 0x7f5a9c050d60 2023-10-11T06:11:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 <== mon.1 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f737c001890 con 0x7f73840d0800 2023-10-11T06:11:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.979+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 >> v1:172.21.15.99:6789/0 conn(0x7f73840d0800 legacy=0x7f73840cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.980+0000 7f738159a640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f736c003d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.980+0000 7f738159a640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 0x7f736c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.980+0000 7f738159a640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f736c004d00 0x7f736c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.980+0000 7f738259c640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f736c003d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.981+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840d0c80 con 0x7f736c003970 2023-10-11T06:11:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.981+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738415c770 con 0x7f736c004d00 2023-10-11T06:11:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.981+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c00b980 con 0x7f73840cecb0 2023-10-11T06:11:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.981+0000 7f7382d9d640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f736c004d00 0x7f736c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.980+0000 7f5aa89ec640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c0f7f90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a94008c40 tx=0x7f5a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f5a99ffb640 1 -- 172.21.15.84:0/1879888966 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940182a0 con 0x7f5a9c050d60 2023-10-11T06:11:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f5a99ffb640 1 -- 172.21.15.84:0/1879888966 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a94018420 con 0x7f5a9c050d60 2023-10-11T06:11:20.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f5a99ffb640 1 -- 172.21.15.84:0/1879888966 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94044400 con 0x7f5a9c050d60 2023-10-11T06:11:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.981+0000 7f7381d9b640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 0x7f736c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f738259c640 1 -- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 msgr2=0x7f736c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f738259c640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73840cecb0 0x7f736c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f738259c640 1 -- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f736c004d00 msgr2=0x7f736c005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f738259c640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f736c004d00 0x7f736c005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.982+0000 7f738259c640 1 -- 172.21.15.84:0/2949555621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c003520 con 0x7f736c003970 2023-10-11T06:11:20.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.983+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/1879888966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c0f8920 con 0x7f5a9c050d60 2023-10-11T06:11:20.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.983+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/1879888966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c0f2940 con 0x7f5a9c050d60 2023-10-11T06:11:20.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.981+0000 7fbe2a78b640 1 Processor -- start 2023-10-11T06:11:20.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.981+0000 7fbe2a78b640 1 -- start start 2023-10-11T06:11:20.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.984+0000 7fbe2a78b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe1c0eb150 0x7fbe1c050d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.984+0000 7fbe2a78b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c036ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:20.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.984+0000 7f738259c640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f736c003d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7374009ab0 tx=0x7f736c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cece29b646f3d6e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:20.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.984+0000 7fbe2a78b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1c04f890 con 0x7fbe1c0ead80 2023-10-11T06:11:20.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.984+0000 7fbe2a78b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c037020 con 0x7fbe1c0eb150 2023-10-11T06:11:20.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.984+0000 7fbe2a78b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0e75e0 con 0x7fbe1c036660 2023-10-11T06:11:20.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7fbe28d28640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c036ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:20.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7fbe28d28640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c036ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53352/0 (socket says 172.21.15.84:53352) 2023-10-11T06:11:20.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7fbe28d28640 1 -- 172.21.15.84:0/1820464610 learned_addr learned my addr 172.21.15.84:0/1820464610 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7fe67a4a4640 1 Processor -- start 2023-10-11T06:11:21.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/1879888966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c0f9100 con 0x7f5a9c050d60 2023-10-11T06:11:21.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7fbe237fe640 1 --2- 172.21.15.84:0/1820464610 >> v2:172.21.15.84:3300/0 conn(0x7fbe1c0eb150 0x7fbe1c050d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374005990 con 0x7f736c003970 2023-10-11T06:11:21.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.986+0000 7f738159a640 1 -- 172.21.15.84:0/2949555621 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7374005af0 con 0x7f736c003970 2023-10-11T06:11:21.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7f7388e0e640 1 -- 172.21.15.84:0/2949555621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 msgr2=0x7f736c003d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7f7388e0e640 1 --2- 172.21.15.84:0/2949555621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f736c003d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7374009ab0 tx=0x7f736c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.986+0000 7fbe237fe640 1 -- 172.21.15.84:0/1820464610 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 msgr2=0x7fbe1c036ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.986+0000 7fbe237fe640 1 --2- 172.21.15.84:0/1820464610 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c036ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.986+0000 7fbe237fe640 1 -- 172.21.15.84:0/1820464610 >> v1:172.21.15.84:6789/0 conn(0x7fbe1c0ead80 legacy=0x7fbe1c0e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.986+0000 7fbe237fe640 1 -- 172.21.15.84:0/1820464610 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c04f2d0 con 0x7fbe1c0eb150 2023-10-11T06:11:21.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.985+0000 7fe67a4a4640 1 -- start start 2023-10-11T06:11:21.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7fe67a4a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c0de910 0x7fe66c04f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7fe67a4a4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe66c036d70 0x7fe66c052cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7fe67a4a4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c005f90 con 0x7fe66c04f770 2023-10-11T06:11:21.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7fe67a4a4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0068f0 con 0x7fe66c036d70 2023-10-11T06:11:21.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7fe67a4a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c006230 con 0x7fe66c0de910 2023-10-11T06:11:21.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.987+0000 7f5a99ffb640 1 -- 172.21.15.84:0/1879888966 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5a9403b400 con 0x7f5a9c050d60 2023-10-11T06:11:21.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.988+0000 7f7382d9d640 1 -- 172.21.15.84:0/2949555621 reap_dead start 2023-10-11T06:11:21.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.988+0000 7f7388e0e640 1 -- 172.21.15.84:0/2949555621 shutdown_connections 2023-10-11T06:11:21.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.988+0000 7f7388e0e640 1 -- 172.21.15.84:0/2949555621 >> 172.21.15.84:0/2949555621 conn(0x7f7384071cc0 msgr2=0x7f738409ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.988+0000 7fe673fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c0de910 0x7fe66c04f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.988+0000 7fe673fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c0de910 0x7fe66c04f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53354/0 (socket says 172.21.15.84:53354) 2023-10-11T06:11:21.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.988+0000 7fe673fff640 1 -- 172.21.15.84:0/1299839839 learned_addr learned my addr 172.21.15.84:0/1299839839 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.988+0000 7fe678a41640 1 --2- 172.21.15.84:0/1299839839 >> v2:172.21.15.84:3300/0 conn(0x7fe66c036d70 0x7fe66c052cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7f5a99ffb640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5a8806a930 0x7f5a8806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7f5a99ffb640 1 -- 172.21.15.84:0/1879888966 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a940aad70 con 0x7f5a9c050d60 2023-10-11T06:11:21.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7f7388e0e640 1 -- 172.21.15.84:0/2949555621 shutdown_connections 2023-10-11T06:11:21.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7fe678a41640 1 -- 172.21.15.84:0/1299839839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c0de910 msgr2=0x7fe66c04f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7fe678a41640 1 --2- 172.21.15.84:0/1299839839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c0de910 0x7fe66c04f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7fe678a41640 1 -- 172.21.15.84:0/1299839839 >> v1:172.21.15.84:6789/0 conn(0x7fe66c04f770 legacy=0x7fe66c036660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7fe678a41640 1 -- 172.21.15.84:0/1299839839 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c006510 con 0x7fe66c036d70 2023-10-11T06:11:21.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.989+0000 7f7388e0e640 1 -- 172.21.15.84:0/2949555621 wait complete. 2023-10-11T06:11:21.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.991+0000 7fe678a41640 1 --2- 172.21.15.84:0/1299839839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c036d70 0x7fe66c052cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe6680097a0 tx=0x7fe66802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72f354efbf0951c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.991+0000 7f7388e0e640 1 Processor -- start 2023-10-11T06:11:21.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.991+0000 7f7388e0e640 1 -- start start 2023-10-11T06:11:21.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.992+0000 7fbe237fe640 1 --2- 172.21.15.84:0/1820464610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0eb150 0x7fbe1c050d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbe100097a0 tx=0x7fbe1002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7eefb61d5d11ae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.992+0000 7f7388e0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f73840c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7388e0e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7384159f40 0x7f73840c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f738259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f73840c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f738259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f73840c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53944/0 (socket says 172.21.15.84:53944) 2023-10-11T06:11:21.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7388e0e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73840c9d70 0x7f73841f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7388e0e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840caf90 con 0x7f736c003970 2023-10-11T06:11:21.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7388e0e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840ca380 con 0x7f73840c9d70 2023-10-11T06:11:21.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7388e0e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840ca660 con 0x7f7384159f40 2023-10-11T06:11:21.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7381d9b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7384159f40 0x7f73840c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7382d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73840c9d70 0x7f73841f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7381d9b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7384159f40 0x7f73840c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53368/0 (socket says 172.21.15.84:53368) 2023-10-11T06:11:21.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.993+0000 7f7381d9b640 1 -- 172.21.15.84:0/1630802956 learned_addr learned my addr 172.21.15.84:0/1630802956 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.994+0000 7fa6dc8ee640 1 Processor -- start 2023-10-11T06:11:21.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.995+0000 7f5aa91ed640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5a8806a930 0x7f5a8806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.994+0000 7f7382d9d640 1 -- 172.21.15.84:0/1630802956 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7384159f40 msgr2=0x7f73840c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.994+0000 7f7382d9d640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7384159f40 0x7f73840c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.994+0000 7f7382d9d640 1 -- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 msgr2=0x7f73840c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.994+0000 7f7382d9d640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f73840c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.994+0000 7f7382d9d640 1 -- 172.21.15.84:0/1630802956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840ca940 con 0x7f73840c9d70 2023-10-11T06:11:21.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.995+0000 7f7382d9d640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73840c9d70 0x7f73841f4800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73840b87a0 tx=0x7f737c00ade0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.995+0000 7f7372ffd640 1 -- 172.21.15.84:0/1630802956 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c0118e0 con 0x7f73840c9d70 2023-10-11T06:11:21.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.995+0000 7f7372ffd640 1 -- 172.21.15.84:0/1630802956 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c011a40 con 0x7f73840c9d70 2023-10-11T06:11:21.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.995+0000 7f7372ffd640 1 -- 172.21.15.84:0/1630802956 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c011db0 con 0x7f73840c9d70 2023-10-11T06:11:21.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.995+0000 7f7388e0e640 1 -- 172.21.15.84:0/1630802956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73840cac20 con 0x7f73840c9d70 2023-10-11T06:11:21.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.996+0000 7f7388e0e640 1 -- 172.21.15.84:0/1630802956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73841f4d40 con 0x7f73840c9d70 2023-10-11T06:11:21.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.994+0000 7fa6dc8ee640 1 -- start start 2023-10-11T06:11:21.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.997+0000 7fa6dc8ee640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6d00dae50 0x7fa6d0051410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.997+0000 7fa6dc8ee640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa6d00571e0 0x7fa6d0051950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c005250 con 0x7f73840c9d70 2023-10-11T06:11:21.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fac7c300640 1 Processor -- start 2023-10-11T06:11:21.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fac7c300640 1 -- start start 2023-10-11T06:11:21.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7f5aa91ed640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5a8806a930 0x7f5a8806cde0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0006fd0 tx=0x7f5aa0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fa6d77fe640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6d00dae50 0x7fa6d0051410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fa6d77fe640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6d00dae50 0x7fa6d0051410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53378/0 (socket says 172.21.15.84:53378) 2023-10-11T06:11:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fa6d77fe640 1 -- 172.21.15.84:0/3052039709 learned_addr learned my addr 172.21.15.84:0/3052039709 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d0053ec0 con 0x7fa6d0054350 2023-10-11T06:11:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d0057620 con 0x7fa6d00571e0 2023-10-11T06:11:21.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.998+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00db220 con 0x7fa6d00dae50 2023-10-11T06:11:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.999+0000 7fe672ffd640 1 -- 172.21.15.84:0/1299839839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66802e070 con 0x7fe66c036d70 2023-10-11T06:11:21.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.999+0000 7f5a99ffb640 1 -- 172.21.15.84:0/1879888966 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5a940af070 con 0x7f5a9c050d60 2023-10-11T06:11:21.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.999+0000 7fe672ffd640 1 -- 172.21.15.84:0/1299839839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6680315a0 con 0x7fe66c036d70 2023-10-11T06:11:21.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.999+0000 7f7372ffd640 1 -- 172.21.15.84:0/1630802956 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f737c01b360 con 0x7f73840c9d70 2023-10-11T06:11:21.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fac7c300640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac6c0e2a30 0x7fac6c03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.999+0000 7fbe22ffd640 1 -- 172.21.15.84:0/1820464610 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1002e070 con 0x7fbe1c0eb150 2023-10-11T06:11:21.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.999+0000 7fbe22ffd640 1 -- 172.21.15.84:0/1820464610 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe100315a0 con 0x7fbe1c0eb150 2023-10-11T06:11:21.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:20.999+0000 7fbe22ffd640 1 -- 172.21.15.84:0/1820464610 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10031930 con 0x7fbe1c0eb150 2023-10-11T06:11:21.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1820464610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0eb150 msgr2=0x7fbe1c050d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1820464610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0eb150 0x7fbe1c050d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbe100097a0 tx=0x7fbe1002b840 comp rx=0 tx=0).stop 2023-10-11T06:11:21.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1820464610 shutdown_connections 2023-10-11T06:11:21.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1820464610 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c036ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1820464610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0eb150 0x7fbe1c050d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1820464610 >> 172.21.15.84:0/1820464610 conn(0x7fbe1c051660 msgr2=0x7fbe1c051a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fe67a4a4640 1 -- 172.21.15.84:0/1299839839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c036d70 msgr2=0x7fe66c052cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fe672ffd640 1 -- 172.21.15.84:0/1299839839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66802e070 con 0x7fe66c036d70 2023-10-11T06:11:21.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/1299839839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c036d70 0x7fe66c052cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe6680097a0 tx=0x7fe66802b840 comp rx=0 tx=0).stop 2023-10-11T06:11:21.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fe67a4a4640 1 -- 172.21.15.84:0/1299839839 shutdown_connections 2023-10-11T06:11:21.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/1299839839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c036d70 0x7fe66c052cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/1299839839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c0de910 0x7fe66c04f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fe67a4a4640 1 -- 172.21.15.84:0/1299839839 >> 172.21.15.84:0/1299839839 conn(0x7fe66c051660 msgr2=0x7fe66c051a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.000+0000 7fac7c300640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c038410 0x7fac6c0e2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7c300640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6c007190 con 0x7fac6c03af00 2023-10-11T06:11:21.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7c300640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c03b3a0 con 0x7fac6c0e2a30 2023-10-11T06:11:21.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7c300640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c0e7290 con 0x7fac6c038410 2023-10-11T06:11:21.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7989b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fac6c03af00 0x7fac6c037d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54790/0 (socket says 172.21.15.84:54790) 2023-10-11T06:11:21.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7a09c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac6c0e2a30 0x7fac6c03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7a89d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c038410 0x7fac6c0e2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1820464610 shutdown_connections 2023-10-11T06:11:21.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7f7372ffd640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f736406a860 0x7f736406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7f7372ffd640 1 -- 172.21.15.84:0/1630802956 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f737c088400 con 0x7f73840c9d70 2023-10-11T06:11:21.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fe67a4a4640 1 -- 172.21.15.84:0/1299839839 shutdown_connections 2023-10-11T06:11:21.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7a09c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac6c0e2a30 0x7fac6c03a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53946/0 (socket says 172.21.15.84:53946) 2023-10-11T06:11:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fac7a09c640 1 -- 172.21.15.84:0/3443001589 learned_addr learned my addr 172.21.15.84:0/3443001589 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.002+0000 7fac7a89d640 1 -- 172.21.15.84:0/3443001589 >> v2:172.21.15.84:3300/0 conn(0x7fac6c0e2a30 msgr2=0x7fac6c03a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.002+0000 7f738259c640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f736406a860 0x7f736406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.002+0000 7fac7a89d640 1 --2- 172.21.15.84:0/3443001589 >> v2:172.21.15.84:3300/0 conn(0x7fac6c0e2a30 0x7fac6c03a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.002+0000 7fac7a89d640 1 -- 172.21.15.84:0/3443001589 >> v1:172.21.15.84:6789/0 conn(0x7fac6c03af00 legacy=0x7fac6c037d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fe67a4a4640 1 -- 172.21.15.84:0/1299839839 wait complete. 2023-10-11T06:11:21.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.001+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1820464610 wait complete. 2023-10-11T06:11:21.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.003+0000 7f738259c640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f736406a860 0x7f736406cd10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7374035000 tx=0x7f7374030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.003+0000 7fa6d7fff640 1 --2- 172.21.15.84:0/3052039709 >> v2:172.21.15.99:3300/0 conn(0x7fa6d00571e0 0x7fa6d0051950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.002+0000 7fac7a89d640 1 -- 172.21.15.84:0/3443001589 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0e6680 con 0x7fac6c038410 2023-10-11T06:11:21.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.003+0000 7fac7989b640 1 --1- 172.21.15.84:0/3443001589 >> v1:172.21.15.84:6789/0 conn(0x7fac6c03af00 0x7fac6c037d00 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-11T06:11:21.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.003+0000 7fac7a09c640 1 --2- 172.21.15.84:0/3443001589 >> v2:172.21.15.84:3300/0 conn(0x7fac6c0e2a30 0x7fac6c03a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:21.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.003+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999536049 0 0) 0x7fa6d0053ec0 con 0x7fa6d0054350 2023-10-11T06:11:21.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.003+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c8003520 con 0x7fa6d0054350 2023-10-11T06:11:21.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.003+0000 7fac7a89d640 1 --2- 172.21.15.84:0/3443001589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c038410 0x7fac6c0e2e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fac7404edf0 tx=0x7fac740a5700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa08c5dba3dbeac server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802519243 0 0) 0x7fa6c8003520 con 0x7fa6d0054350 2023-10-11T06:11:21.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6d00dae50 msgr2=0x7fa6d0051410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6d00dae50 0x7fa6d0051410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 >> v2:172.21.15.99:3300/0 conn(0x7fa6d00571e0 msgr2=0x7fa6d0051950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 --2- 172.21.15.84:0/3052039709 >> v2:172.21.15.99:3300/0 conn(0x7fa6d00571e0 0x7fa6d0051950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0053590 con 0x7fa6d0054350 2023-10-11T06:11:21.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fa6c4001670 con 0x7fa6d0054350 2023-10-11T06:11:21.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fac7909a640 1 -- 172.21.15.84:0/3443001589 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac740aa710 con 0x7fac6c038410 2023-10-11T06:11:21.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fe67a4a4640 1 Processor -- start 2023-10-11T06:11:21.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fac7909a640 1 -- 172.21.15.84:0/3443001589 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac740a8050 con 0x7fac6c038410 2023-10-11T06:11:21.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fac7909a640 1 -- 172.21.15.84:0/3443001589 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac740aaa90 con 0x7fac6c038410 2023-10-11T06:11:21.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 >> v1:172.21.15.84:6789/0 conn(0x7fa6d0054350 legacy=0x7fa6d0056ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c80039a0 0x7fa6c8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6c80047c0 0x7fa6c8004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80051e0 0x7fa6c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0057620 con 0x7fa6c80039a0 2023-10-11T06:11:21.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80104c0 con 0x7fa6c80047c0 2023-10-11T06:11:21.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8010680 con 0x7fa6c80051e0 2023-10-11T06:11:21.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fa6d6ffd640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6c80047c0 0x7fa6c8004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fa6d7fff640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80051e0 0x7fa6c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fbe2a78b640 1 Processor -- start 2023-10-11T06:11:21.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fbe2a78b640 1 -- start start 2023-10-11T06:11:21.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fac7c300640 1 -- 172.21.15.84:0/3443001589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c038410 msgr2=0x7fac6c0e2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fac7c300640 1 --2- 172.21.15.84:0/3443001589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c038410 0x7fac6c0e2e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fac7404edf0 tx=0x7fac740a5700 comp rx=0 tx=0).stop 2023-10-11T06:11:21.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fa6d77fe640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c80039a0 0x7fa6c8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fac7c300640 1 -- 172.21.15.84:0/3443001589 shutdown_connections 2023-10-11T06:11:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fac7c300640 1 --2- 172.21.15.84:0/3443001589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c038410 0x7fac6c0e2e00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.005+0000 7fac7c300640 1 --2- 172.21.15.84:0/3443001589 >> v2:172.21.15.84:3300/0 conn(0x7fac6c0e2a30 0x7fac6c03a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fac7c300640 1 -- 172.21.15.84:0/3443001589 >> 172.21.15.84:0/3443001589 conn(0x7fac6c058f40 msgr2=0x7fac6c059340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.004+0000 7fe67a4a4640 1 -- start start 2023-10-11T06:11:21.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fe67a4a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c036d70 0x7fe66c0d3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fe67a4a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66c04f770 0x7fe66c0eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fbe2a78b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c038040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fe67a4a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0de910 0x7fe66c17c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fe67a4a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0dfff0 con 0x7fe66c0de910 2023-10-11T06:11:21.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fe67a4a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0d36a0 con 0x7fe66c04f770 2023-10-11T06:11:21.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fa6d6ffd640 1 -- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80051e0 msgr2=0x7fa6c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fa6d6ffd640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80051e0 0x7fa6c800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fa6d6ffd640 1 -- 172.21.15.84:0/3052039709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c80039a0 msgr2=0x7fa6c8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fa6d6ffd640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c80039a0 0x7fa6c8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.006+0000 7fa6d6ffd640 1 -- 172.21.15.84:0/3052039709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0053590 con 0x7fa6c80047c0 2023-10-11T06:11:21.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fbe2a78b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0ead80 0x7fbe1c053490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fe673fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c036d70 0x7fe66c0d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66c04f770 0x7fe66c0eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66c04f770 0x7fe66c0eabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:35430/0 (socket says 172.21.15.84:35430) 2023-10-11T06:11:21.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fe6737fe640 1 -- 172.21.15.84:0/2228822969 learned_addr learned my addr 172.21.15.84:0/2228822969 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c039890 con 0x7fe66c036d70 2023-10-11T06:11:21.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fac7c300640 1 -- 172.21.15.84:0/3443001589 shutdown_connections 2023-10-11T06:11:21.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fe678a41640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0de910 0x7fe66c17c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fac7c300640 1 -- 172.21.15.84:0/3443001589 wait complete. 2023-10-11T06:11:21.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7f59812b4640 1 Processor -- start 2023-10-11T06:11:21.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fa6d6ffd640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6c80047c0 0x7fa6c8004c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c160 tx=0x7fa6c80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=801db6f33d378a33 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fbe2a78b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c0eb150 0x7fbe1c0539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fbe2a78b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0ddc00 con 0x7fbe1c0ead80 2023-10-11T06:11:21.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fbe2a78b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c054000 con 0x7fbe1c0eb150 2023-10-11T06:11:21.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.007+0000 7fbe2a78b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0542e0 con 0x7fbe1c036660 2023-10-11T06:11:21.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fbe23fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c038040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fbe23fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c038040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53410/0 (socket says 172.21.15.84:53410) 2023-10-11T06:11:21.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fbe23fff640 1 -- 172.21.15.84:0/1650062696 learned_addr learned my addr 172.21.15.84:0/1650062696 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fe673fff640 1 -- 172.21.15.84:0/2228822969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66c04f770 msgr2=0x7fe66c0eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fe673fff640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66c04f770 0x7fe66c0eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fe673fff640 1 -- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0de910 msgr2=0x7fe66c17c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7f59812b4640 1 -- start start 2023-10-11T06:11:21.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fe673fff640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0de910 0x7fe66c17c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fe673fff640 1 -- 172.21.15.84:0/2228822969 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668009400 con 0x7fe66c036d70 2023-10-11T06:11:21.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fe678a41640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0de910 0x7fe66c17c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:11:21.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fbe237fe640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0ead80 0x7fbe1c053490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.008+0000 7fbe28d28640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c0eb150 0x7fbe1c0539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe23fff640 1 -- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c0eb150 msgr2=0x7fbe1c0539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe23fff640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c0eb150 0x7fbe1c0539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe23fff640 1 -- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0ead80 msgr2=0x7fbe1c053490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe23fff640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0ead80 0x7fbe1c053490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe23fff640 1 -- 172.21.15.84:0/1650062696 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe10009400 con 0x7fbe1c036660 2023-10-11T06:11:21.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fe673fff640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c036d70 0x7fe66c0d3160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe674068710 tx=0x7fe674067010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe23fff640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c038040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbe24063d20 tx=0x7fbe24067010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fe670ff9640 1 -- 172.21.15.84:0/2228822969 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6740733b0 con 0x7fe66c036d70 2023-10-11T06:11:21.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fe670ff9640 1 -- 172.21.15.84:0/2228822969 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67408e070 con 0x7fe66c036d70 2023-10-11T06:11:21.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c006e90 con 0x7fe66c036d70 2023-10-11T06:11:21.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe20ff9640 1 -- 172.21.15.84:0/1650062696 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240735b0 con 0x7fbe1c036660 2023-10-11T06:11:21.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.009+0000 7fbe20ff9640 1 -- 172.21.15.84:0/1650062696 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe24072360 con 0x7fbe1c036660 2023-10-11T06:11:21.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c0dfc80 con 0x7fe66c036d70 2023-10-11T06:11:21.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7f59812b4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f597403e870 0x7f59740384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7f59812b4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59740e2dd0 con 0x7f5974055220 2023-10-11T06:11:21.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7f59812b4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5974053a50 con 0x7f5974055670 2023-10-11T06:11:21.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7f59812b4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5974007230 con 0x7f597403e870 2023-10-11T06:11:21.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7f597affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f597403e870 0x7f59740384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7f597affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f597403e870 0x7f59740384d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:35462/0 (socket says 172.21.15.84:35462) 2023-10-11T06:11:21.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7f597affd640 1 -- 172.21.15.84:0/280833519 learned_addr learned my addr 172.21.15.84:0/280833519 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7fe670ff9640 1 -- 172.21.15.84:0/2228822969 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674073750 con 0x7fe66c036d70 2023-10-11T06:11:21.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7fbe20ff9640 1 -- 172.21.15.84:0/1650062696 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24073990 con 0x7fbe1c036660 2023-10-11T06:11:21.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.010+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c0387a0 con 0x7fbe1c036660 2023-10-11T06:11:21.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c038d60 con 0x7fbe1c036660 2023-10-11T06:11:21.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7fac7c300640 1 Processor -- start 2023-10-11T06:11:21.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7fac7c300640 1 -- start start 2023-10-11T06:11:21.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7f597affd640 1 -- 172.21.15.84:0/280833519 >> v1:172.21.15.99:6789/0 conn(0x7f5974055670 legacy=0x7f5974056b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7f597affd640 1 -- 172.21.15.84:0/280833519 >> v1:172.21.15.84:6789/0 conn(0x7f5974055220 legacy=0x7f597403aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7f597affd640 1 -- 172.21.15.84:0/280833519 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974006c70 con 0x7f597403e870 2023-10-11T06:11:21.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.011+0000 7f5979ffb640 1 -- 172.21.15.84:0/280833519 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972274656 0 0) 0x7f5970001110 con 0x7f5974055220 2023-10-11T06:11:21.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.012+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c0e7fa0 con 0x7fbe1c036660 2023-10-11T06:11:21.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.012+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe654005250 con 0x7fe66c036d70 2023-10-11T06:11:21.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.012+0000 7f3bc74d4640 1 Processor -- start 2023-10-11T06:11:21.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.012+0000 7f3bc74d4640 1 -- start start 2023-10-11T06:11:21.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.013+0000 7f597affd640 1 --2- 172.21.15.84:0/280833519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597403e870 0x7f59740384d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f596c009880 tx=0x7f596c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341cc7254a3a7a18 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.013+0000 7fe670ff9640 1 -- 172.21.15.84:0/2228822969 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe6740738b0 con 0x7fe66c036d70 2023-10-11T06:11:21.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.013+0000 7fac7c300640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac6c038410 0x7fac6c0e3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7c300640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c03af00 0x7fac6c039aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fbe20ff9640 1 -- 172.21.15.84:0/1650062696 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbe2409a390 con 0x7fbe1c036660 2023-10-11T06:11:21.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7a09c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac6c038410 0x7fac6c0e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7a09c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac6c038410 0x7fac6c0e3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53980/0 (socket says 172.21.15.84:53980) 2023-10-11T06:11:21.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7c300640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c0e2a30 0x7fac6c0d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7f3bc74d4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3bc0160800 0x7f3bc00d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7a09c640 1 -- 172.21.15.84:0/1471282107 learned_addr learned my addr 172.21.15.84:0/1471282107 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c17b970 con 0x7fac6c038410 2023-10-11T06:11:21.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c0e3cd0 con 0x7fac6c0e2a30 2023-10-11T06:11:21.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7f3bc74d4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7f3bc74d4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc00b0550 con 0x7f3bc00d0a50 2023-10-11T06:11:21.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7f3bc74d4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc0160bd0 con 0x7f3bc0160800 2023-10-11T06:11:21.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7f3bc74d4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc00d0ec0 con 0x7f3bc00af4d0 2023-10-11T06:11:21.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7989b640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c03af00 0x7fac6c039aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.014+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c03ac10 con 0x7fac6c03af00 2023-10-11T06:11:21.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7f3bc5270640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3bc0160800 0x7f3bc00d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7f3bc5270640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3bc0160800 0x7f3bc00d30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53982/0 (socket says 172.21.15.84:53982) 2023-10-11T06:11:21.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7fbe20ff9640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe0806a830 0x7fbe0806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7f3bc5270640 1 -- 172.21.15.84:0/2016773903 learned_addr learned my addr 172.21.15.84:0/2016773903 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7fbe20ff9640 1 -- 172.21.15.84:0/1650062696 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe24102b00 con 0x7fbe1c036660 2023-10-11T06:11:21.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7f3bc4a6f640 1 --2- 172.21.15.84:0/2016773903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7fe670ff9640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe65806a830 0x7fe65806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fe670ff9640 1 -- 172.21.15.84:0/2228822969 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe674102250 con 0x7fe66c036d70 2023-10-11T06:11:21.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fbe237fe640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe0806a830 0x7fbe0806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7f3bc5270640 1 -- 172.21.15.84:0/2016773903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 msgr2=0x7f3bc00d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7f3bc5270640 1 --2- 172.21.15.84:0/2016773903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7f3bc5270640 1 -- 172.21.15.84:0/2016773903 >> v1:172.21.15.84:6789/0 conn(0x7f3bc00d0a50 legacy=0x7f3bc006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7f3bc5270640 1 -- 172.21.15.84:0/2016773903 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc00af940 con 0x7f3bc0160800 2023-10-11T06:11:21.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fe6737fe640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe65806a830 0x7fe65806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.015+0000 7fac7a89d640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c0e2a30 0x7fac6c0d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fac7a09c640 1 -- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c03af00 msgr2=0x7fac6c039aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fac7a09c640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c03af00 0x7fac6c039aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fac7a09c640 1 -- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c0e2a30 msgr2=0x7fac6c0d9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fac7a09c640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c0e2a30 0x7fac6c0d9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fac7a09c640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac700094d0 con 0x7fac6c038410 2023-10-11T06:11:21.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.016+0000 7fac7a09c640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac6c038410 0x7fac6c0e3790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9b0 tx=0x7fac7000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.017+0000 7fbe237fe640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe0806a830 0x7fbe0806cce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe10036000 tx=0x7fbe10008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.017+0000 7fac7989b640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c03af00 0x7fac6c039aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:21.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.017+0000 7fe6737fe640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe65806a830 0x7fe65806cce0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0de320 tx=0x7fe660007ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.017+0000 7fac6affd640 1 -- 172.21.15.84:0/1471282107 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7000c9a0 con 0x7fac6c038410 2023-10-11T06:11:21.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.017+0000 7f3bc5270640 1 --2- 172.21.15.84:0/2016773903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc00d30f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1766c29e597855d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.017+0000 7fac6affd640 1 -- 172.21.15.84:0/1471282107 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7000cb00 con 0x7fac6c038410 2023-10-11T06:11:21.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.017+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c0e3290 con 0x7fac6c038410 2023-10-11T06:11:21.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.018+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c17be60 con 0x7fac6c038410 2023-10-11T06:11:21.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.018+0000 7fac6affd640 1 -- 172.21.15.84:0/1471282107 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7000ce70 con 0x7fac6c038410 2023-10-11T06:11:21.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.018+0000 7f3bbffff640 1 -- 172.21.15.84:0/2016773903 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4030840 con 0x7f3bc0160800 2023-10-11T06:11:21.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.018+0000 7f3bbffff640 1 -- 172.21.15.84:0/2016773903 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb40309a0 con 0x7f3bc0160800 2023-10-11T06:11:21.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.018+0000 7f3bbffff640 1 -- 172.21.15.84:0/2016773903 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4030d30 con 0x7f3bc0160800 2023-10-11T06:11:21.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7f3bc74d4640 1 -- 172.21.15.84:0/2016773903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 msgr2=0x7f3bc00d30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/2016773903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc00d30f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb402b980 comp rx=0 tx=0).stop 2023-10-11T06:11:21.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7f3bc74d4640 1 -- 172.21.15.84:0/2016773903 shutdown_connections 2023-10-11T06:11:21.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/2016773903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/2016773903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc00d30f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7f3bc74d4640 1 -- 172.21.15.84:0/2016773903 >> 172.21.15.84:0/2016773903 conn(0x7f3bc0071cc0 msgr2=0x7f3bc009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c402ca30 con 0x7fa6c80047c0 2023-10-11T06:11:21.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c402cbb0 con 0x7fa6c80047c0 2023-10-11T06:11:21.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7fa6d67fc640 1 -- 172.21.15.84:0/3052039709 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4034440 con 0x7fa6c80047c0 2023-10-11T06:11:21.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7f3bc74d4640 1 -- 172.21.15.84:0/2016773903 shutdown_connections 2023-10-11T06:11:21.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.020+0000 7f3bc74d4640 1 -- 172.21.15.84:0/2016773903 wait complete. 2023-10-11T06:11:21.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.020+0000 7f5979ffb640 1 -- 172.21.15.84:0/280833519 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c004230 con 0x7f597403e870 2023-10-11T06:11:21.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.020+0000 7f5979ffb640 1 -- 172.21.15.84:0/280833519 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c004390 con 0x7f597403e870 2023-10-11T06:11:21.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.020+0000 7f5979ffb640 1 -- 172.21.15.84:0/280833519 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c003810 con 0x7f597403e870 2023-10-11T06:11:21.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.020+0000 7f7372ffd640 1 -- 172.21.15.84:0/1630802956 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f737c053690 con 0x7f73840c9d70 2023-10-11T06:11:21.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.020+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6c80047c0 msgr2=0x7fa6c8004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.020+0000 7fa6dc8ee640 1 --2- 172.21.15.84:0/3052039709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6c80047c0 0x7fa6c8004c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c160 tx=0x7fa6c80057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7fa6d7fff640 1 -- 172.21.15.84:0/3052039709 reap_dead start 2023-10-11T06:11:21.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7f59812b4640 1 -- 172.21.15.84:0/280833519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597403e870 msgr2=0x7f59740384d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7f59812b4640 1 --2- 172.21.15.84:0/280833519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597403e870 0x7f59740384d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f596c009880 tx=0x7f596c02b8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7fac6affd640 1 -- 172.21.15.84:0/1471282107 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fac70013470 con 0x7fac6c038410 2023-10-11T06:11:21.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 shutdown_connections 2023-10-11T06:11:21.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 >> 172.21.15.84:0/3052039709 conn(0x7fa6d00388f0 msgr2=0x7fa6d0038cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7f59812b4640 1 -- 172.21.15.84:0/280833519 shutdown_connections 2023-10-11T06:11:21.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7f59812b4640 1 --2- 172.21.15.84:0/280833519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f597403e870 0x7f59740384d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7f59812b4640 1 -- 172.21.15.84:0/280833519 >> 172.21.15.84:0/280833519 conn(0x7f5974055a70 msgr2=0x7f5974055e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.019+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c0056c0 con 0x7fac6c038410 2023-10-11T06:11:21.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7f59812b4640 1 -- 172.21.15.84:0/280833519 shutdown_connections 2023-10-11T06:11:21.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.021+0000 7f59812b4640 1 -- 172.21.15.84:0/280833519 wait complete. 2023-10-11T06:11:21.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.022+0000 7f3bc74d4640 1 Processor -- start 2023-10-11T06:11:21.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.022+0000 7f3bc74d4640 1 -- start start 2023-10-11T06:11:21.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.023+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 shutdown_connections 2023-10-11T06:11:21.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.023+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/3052039709 wait complete. 2023-10-11T06:11:21.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.023+0000 7fac6affd640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fac5806a900 0x7fac5806cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.023+0000 7fac6affd640 1 -- 172.21.15.84:0/1471282107 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac70089130 con 0x7fac6c038410 2023-10-11T06:11:21.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.024+0000 7f59812b4640 1 Processor -- start 2023-10-11T06:11:21.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.024+0000 7fac7989b640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fac5806a900 0x7fac5806cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.024+0000 7f59812b4640 1 -- start start 2023-10-11T06:11:21.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.025+0000 7fac7989b640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fac5806a900 0x7fac5806cdb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0ddda0 tx=0x7fac64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.025+0000 7f3bc74d4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.025+0000 7f3bc74d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc00d0a50 0x7f3bc00c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.025+0000 7f3bc74d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc5270640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc5270640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53438/0 (socket says 172.21.15.84:53438) 2023-10-11T06:11:21.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc5a71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc5a71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc006d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53996/0 (socket says 172.21.15.84:53996) 2023-10-11T06:11:21.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc4a6f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc00d0a50 0x7f3bc00c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc5270640 1 -- 172.21.15.84:0/557570616 learned_addr learned my addr 172.21.15.84:0/557570616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc014bab0 con 0x7f3bc0160800 2023-10-11T06:11:21.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f59812b4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f597403e870 0x7f59740dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc014bda0 con 0x7f3bc00d0a50 2023-10-11T06:11:21.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc009c8f0 con 0x7f3bc00af4d0 2023-10-11T06:11:21.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7fa6dc8ee640 1 Processor -- start 2023-10-11T06:11:21.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f59812b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5974055220 0x7f5974039e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7f59812b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5974055670 0x7f597403a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.026+0000 7fa6dc8ee640 1 -- start start 2023-10-11T06:11:21.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f597affd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f597403e870 0x7f59740dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f597affd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f597403e870 0x7f59740dc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53442/0 (socket says 172.21.15.84:53442) 2023-10-11T06:11:21.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f597affd640 1 -- 172.21.15.84:0/2200813104 learned_addr learned my addr 172.21.15.84:0/2200813104 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f59812b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59740e4030 con 0x7f5974055220 2023-10-11T06:11:21.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59740de280 con 0x7f5974055670 2023-10-11T06:11:21.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59741855b0 con 0x7f597403e870 2023-10-11T06:11:21.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f597b7fe640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5974055670 0x7f597403a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f3bc5a71640 1 -- 172.21.15.84:0/557570616 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 msgr2=0x7f3bc00c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f3bc5a71640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f3bc5a71640 1 -- 172.21.15.84:0/557570616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc00d0a50 msgr2=0x7f3bc00c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f3bc5a71640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc00d0a50 0x7f3bc00c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f3bc5a71640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40094d0 con 0x7f3bc0160800 2023-10-11T06:11:21.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f3bc5a71640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc006d1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800cac0 tx=0x7f3bb800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.027+0000 7f597a7fc640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5974055220 0x7f5974039e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7f597affd640 1 -- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5974055670 msgr2=0x7f597403a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7f597affd640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5974055670 0x7f597403a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7f597affd640 1 -- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5974055220 msgr2=0x7f5974039e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7f597affd640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5974055220 0x7f5974039e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7f597affd640 1 -- 172.21.15.84:0/2200813104 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c009460 con 0x7f597403e870 2023-10-11T06:11:21.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7f597a7fc640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5974055220 0x7f5974039e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:21.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7f597affd640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f597403e870 0x7f59740dc3e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f596c0099a0 tx=0x7f596c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.028+0000 7fa6dc8ee640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80047c0 0x7fa6d003a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6dc8ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d004b640 0x7fa6d00d8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6dc8ee640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d000dda0 0x7fa6d000e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6d6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d004b640 0x7fa6d00d8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6dc8ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0177e50 con 0x7fa6d004b640 2023-10-11T06:11:21.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6dc8ee640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000e750 con 0x7fa6d000dda0 2023-10-11T06:11:21.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6d6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d004b640 0x7fa6d00d8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54008/0 (socket says 172.21.15.84:54008) 2023-10-11T06:11:21.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6dc8ee640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d003ae30 con 0x7fa6c80047c0 2023-10-11T06:11:21.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6d6ffd640 1 -- 172.21.15.84:0/2919121575 learned_addr learned my addr 172.21.15.84:0/2919121575 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6d77fe640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80047c0 0x7fa6d003a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.029+0000 7fa6d7fff640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d000dda0 0x7fa6d000e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.030+0000 7fa6d6ffd640 1 -- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80047c0 msgr2=0x7fa6d003a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.030+0000 7fa6d6ffd640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80047c0 0x7fa6d003a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.030+0000 7fa6d6ffd640 1 -- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d000dda0 msgr2=0x7fa6d000e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.030+0000 7fa6d6ffd640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d000dda0 0x7fa6d000e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.030+0000 7fa6d6ffd640 1 -- 172.21.15.84:0/2919121575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d01770b0 con 0x7fa6d004b640 2023-10-11T06:11:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.030+0000 7fa6d6ffd640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d004b640 0x7fa6d00d8bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c280 tx=0x7fa6c4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7fa6c3fff640 1 -- 172.21.15.84:0/2919121575 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40037f0 con 0x7fa6d004b640 2023-10-11T06:11:21.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7fac6affd640 1 -- 172.21.15.84:0/1471282107 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fac70054440 con 0x7fac6c038410 2023-10-11T06:11:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7f3bbdffb640 1 -- 172.21.15.84:0/557570616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8003b70 con 0x7f3bc0160800 2023-10-11T06:11:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc014c040 con 0x7f3bc0160800 2023-10-11T06:11:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7f3bbdffb640 1 -- 172.21.15.84:0/557570616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8003cd0 con 0x7f3bc0160800 2023-10-11T06:11:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc015f5c0 con 0x7f3bc0160800 2023-10-11T06:11:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.032+0000 7f3bbdffb640 1 -- 172.21.15.84:0/557570616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8003b70 con 0x7f3bc0160800 2023-10-11T06:11:21.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.032+0000 7fbe20ff9640 1 -- 172.21.15.84:0/1650062696 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fbe240cdef0 con 0x7fbe1c036660 2023-10-11T06:11:21.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7fa6c3fff640 1 -- 172.21.15.84:0/2919121575 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c4003950 con 0x7fa6d004b640 2023-10-11T06:11:21.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.031+0000 7fa6c3fff640 1 -- 172.21.15.84:0/2919121575 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003cc0 con 0x7fa6d004b640 2023-10-11T06:11:21.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.033+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc00d16f0 con 0x7f3bc0160800 2023-10-11T06:11:21.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.033+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d0177370 con 0x7fa6d004b640 2023-10-11T06:11:21.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.033+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d0177930 con 0x7fa6d004b640 2023-10-11T06:11:21.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.035+0000 7f3bbdffb640 1 -- 172.21.15.84:0/557570616 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3bb80050d0 con 0x7f3bc0160800 2023-10-11T06:11:21.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.036+0000 7f3bbdffb640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3ba406a920 0x7f3ba406cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.037+0000 7f3bbdffb640 1 -- 172.21.15.84:0/557570616 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3bb8089470 con 0x7f3bc0160800 2023-10-11T06:11:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.038+0000 7f3bc5270640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3ba406a920 0x7f3ba406cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.036+0000 7fa6c3fff640 1 -- 172.21.15.84:0/2919121575 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa6c4034440 con 0x7fa6d004b640 2023-10-11T06:11:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.038+0000 7fa6c3fff640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa6bc06a860 0x7fa6bc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.038+0000 7fa6d77fe640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa6bc06a860 0x7fa6bc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.039+0000 7f3bc5270640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3ba406a920 0x7f3ba406cdd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40097f0 tx=0x7f3bb4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.039+0000 7fa6d77fe640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa6bc06a860 0x7fa6bc06cd10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006cc0 tx=0x7fa6cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.040+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d0177af0 con 0x7fa6d004b640 2023-10-11T06:11:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.043+0000 7fa6c3fff640 1 -- 172.21.15.84:0/2919121575 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6c40798a0 con 0x7fa6d004b640 2023-10-11T06:11:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.043+0000 7f3bbdffb640 1 -- 172.21.15.84:0/557570616 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3bb8054650 con 0x7f3bc0160800 2023-10-11T06:11:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.045+0000 7f59637fe640 1 -- 172.21.15.84:0/2200813104 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c003bc0 con 0x7f597403e870 2023-10-11T06:11:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.045+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5974185890 con 0x7f597403e870 2023-10-11T06:11:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.046+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5974185e50 con 0x7f597403e870 2023-10-11T06:11:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.046+0000 7f59637fe640 1 -- 172.21.15.84:0/2200813104 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c03f070 con 0x7f597403e870 2023-10-11T06:11:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.047+0000 7f59637fe640 1 -- 172.21.15.84:0/2200813104 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c004500 con 0x7f597403e870 2023-10-11T06:11:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.046+0000 7fe670ff9640 1 -- 172.21.15.84:0/2228822969 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe6740cd580 con 0x7fe66c036d70 2023-10-11T06:11:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.048+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c005250 con 0x7f597403e870 2023-10-11T06:11:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.050+0000 7f59637fe640 1 -- 172.21.15.84:0/2200813104 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f596c041030 con 0x7f597403e870 2023-10-11T06:11:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.052+0000 7f59637fe640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f596806a860 0x7f596806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.052+0000 7f59637fe640 1 -- 172.21.15.84:0/2200813104 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f596c0ab610 con 0x7f597403e870 2023-10-11T06:11:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.053+0000 7f597a7fc640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f596806a860 0x7f596806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.054+0000 7f597a7fc640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f596806a860 0x7f596806cd10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f597403d1d0 tx=0x7f5970009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.054+0000 7fa6c3fff640 1 -- 172.21.15.84:0/2919121575 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa6c4046080 con 0x7fa6d004b640 2023-10-11T06:11:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.063+0000 7f59637fe640 1 -- 172.21.15.84:0/2200813104 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f596c076900 con 0x7f597403e870 2023-10-11T06:11:21.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.401+0000 7f5aaa1ef640 1 -- 172.21.15.84:0/1879888966 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5a9c005380 con 0x7f5a9c050d60 2023-10-11T06:11:21.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.402+0000 7f5a99ffb640 1 -- 172.21.15.84:0/1879888966 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a94075fd0 con 0x7f5a9c050d60 2023-10-11T06:11:21.403 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-10-11T06:11:21.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.406+0000 7f5a877fe640 1 -- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5a8806a930 msgr2=0x7f5a8806cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.406+0000 7f5a877fe640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5a8806a930 0x7f5a8806cde0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0006fd0 tx=0x7f5aa0008040 comp rx=0 tx=0).stop 2023-10-11T06:11:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.406+0000 7f5a877fe640 1 -- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 msgr2=0x7f5a9c0f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.406+0000 7f5a877fe640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c0f7f90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a94008c40 tx=0x7f5a940046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.407+0000 7f5a877fe640 1 -- 172.21.15.84:0/1879888966 shutdown_connections 2023-10-11T06:11:21.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.407+0000 7f5a877fe640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5a8806a930 0x7f5a8806cde0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.407+0000 7f5a877fe640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a9c0fa7b0 0x7f5a9c0f06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.407+0000 7f5a877fe640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a9c050d60 0x7f5a9c0f7f90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.407+0000 7f5a877fe640 1 --2- 172.21.15.84:0/1879888966 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a9c036630 0x7f5a9c0f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.407+0000 7f5a877fe640 1 -- 172.21.15.84:0/1879888966 >> 172.21.15.84:0/1879888966 conn(0x7f5a9c051630 msgr2=0x7f5a9c033890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.408+0000 7f5a877fe640 1 -- 172.21.15.84:0/1879888966 shutdown_connections 2023-10-11T06:11:21.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.408+0000 7f5a877fe640 1 -- 172.21.15.84:0/1879888966 wait complete. 2023-10-11T06:11:21.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.427+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fac6c0077c0 con 0x7fac6c038410 2023-10-11T06:11:21.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.434+0000 7fac6affd640 1 -- 172.21.15.84:0/1471282107 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fac70057a60 con 0x7fac6c038410 2023-10-11T06:11:21.436 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:21.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-10-11T06:11:21.437 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.437+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f735c005500 con 0x7f73840c9d70 2023-10-11T06:11:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.437+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fac5806a900 msgr2=0x7fac5806cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.437+0000 7fac7c300640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fac5806a900 0x7fac5806cdb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0ddda0 tx=0x7fac64009040 comp rx=0 tx=0).stop 2023-10-11T06:11:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.437+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac6c038410 msgr2=0x7fac6c0e3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.438+0000 7f7372ffd640 1 -- 172.21.15.84:0/1630802956 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f737c056cb0 con 0x7f73840c9d70 2023-10-11T06:11:21.440 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:21.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.437+0000 7fac7c300640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac6c038410 0x7fac6c0e3790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9b0 tx=0x7fac7000ee70 comp rx=0 tx=0).stop 2023-10-11T06:11:21.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 shutdown_connections 2023-10-11T06:11:21.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fac5806a900 0x7fac5806cdb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c0e2a30 0x7fac6c0d9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac6c03af00 0x7fac6c039aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 --2- 172.21.15.84:0/1471282107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac6c038410 0x7fac6c0e3790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 >> 172.21.15.84:0/1471282107 conn(0x7fac6c058f40 msgr2=0x7fac6c0e1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 shutdown_connections 2023-10-11T06:11:21.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.439+0000 7fac7c300640 1 -- 172.21.15.84:0/1471282107 wait complete. 2023-10-11T06:11:21.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f736406a860 msgr2=0x7f736406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f736406a860 0x7f736406cd10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7374035000 tx=0x7f7374030000 comp rx=0 tx=0).stop 2023-10-11T06:11:21.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73840c9d70 msgr2=0x7f73841f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73840c9d70 0x7f73841f4800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73840b87a0 tx=0x7f737c00ade0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 shutdown_connections 2023-10-11T06:11:21.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f736406a860 0x7f736406cd10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73840c9d70 0x7f73841f4800 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7384159f40 0x7f73840c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 --2- 172.21.15.84:0/1630802956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f736c003970 0x7f73840c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 >> 172.21.15.84:0/1630802956 conn(0x7f7384071cc0 msgr2=0x7f73840b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 shutdown_connections 2023-10-11T06:11:21.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.441+0000 7f7370ff9640 1 -- 172.21.15.84:0/1630802956 wait complete. 2023-10-11T06:11:21.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.445+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbe1c038f50 con 0x7fbe1c036660 2023-10-11T06:11:21.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.446+0000 7fbe20ff9640 1 -- 172.21.15.84:0/1650062696 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe240d1510 con 0x7fbe1c036660 2023-10-11T06:11:21.447 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-10-11T06:11:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.449+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe0806a830 msgr2=0x7fbe0806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.450+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe0806a830 0x7fbe0806cce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe10036000 tx=0x7fbe10008030 comp rx=0 tx=0).stop 2023-10-11T06:11:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.450+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 msgr2=0x7fbe1c038040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.450+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c038040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbe24063d20 tx=0x7fbe24067010 comp rx=0 tx=0).stop 2023-10-11T06:11:21.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.450+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 shutdown_connections 2023-10-11T06:11:21.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.450+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe0806a830 0x7fbe0806cce0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.450+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c0eb150 0x7fbe1c0539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.450+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe1c0ead80 0x7fbe1c053490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.451+0000 7fbe2a78b640 1 --2- 172.21.15.84:0/1650062696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe1c036660 0x7fbe1c038040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.451+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 >> 172.21.15.84:0/1650062696 conn(0x7fbe1c051660 msgr2=0x7fbe1c0e5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.451+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 shutdown_connections 2023-10-11T06:11:21.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.452+0000 7fbe2a78b640 1 -- 172.21.15.84:0/1650062696 wait complete. 2023-10-11T06:11:21.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.454+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe6540057d0 con 0x7fe66c036d70 2023-10-11T06:11:21.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.455+0000 7fe670ff9640 1 -- 172.21.15.84:0/2228822969 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe674097020 con 0x7fe66c036d70 2023-10-11T06:11:21.457 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:21.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.458+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe65806a830 msgr2=0x7fe65806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.458+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa6d004cd10 con 0x7fa6d004b640 2023-10-11T06:11:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.458+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe65806a830 0x7fe65806cce0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0de320 tx=0x7fe660007ef0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.458+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c036d70 msgr2=0x7fe66c0d3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.458+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c036d70 0x7fe66c0d3160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe674068710 tx=0x7fe674067010 comp rx=0 tx=0).stop 2023-10-11T06:11:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.458+0000 7fa6c3fff640 1 -- 172.21.15.84:0/2919121575 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6c4075be0 con 0x7fa6d004b640 2023-10-11T06:11:21.461 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-10-11T06:11:21.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 shutdown_connections 2023-10-11T06:11:21.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe65806a830 0x7fe65806cce0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe66c0de910 0x7fe66c17c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66c04f770 0x7fe66c0eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 --2- 172.21.15.84:0/2228822969 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe66c036d70 0x7fe66c0d3160 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 >> 172.21.15.84:0/2228822969 conn(0x7fe66c051660 msgr2=0x7fe66c0d3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 shutdown_connections 2023-10-11T06:11:21.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.459+0000 7fe67a4a4640 1 -- 172.21.15.84:0/2228822969 wait complete. 2023-10-11T06:11:21.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.461+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3bc014c560 con 0x7f3bc0160800 2023-10-11T06:11:21.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.461+0000 7f3bbdffb640 1 -- 172.21.15.84:0/557570616 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3bb8057c70 con 0x7f3bc0160800 2023-10-11T06:11:21.465 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:21.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.462+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa6bc06a860 msgr2=0x7fa6bc06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.462+0000 7fa6dc8ee640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa6bc06a860 0x7fa6bc06cd10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006cc0 tx=0x7fa6cc008040 comp rx=0 tx=0).stop 2023-10-11T06:11:21.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.462+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d004b640 msgr2=0x7fa6d00d8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.462+0000 7fa6dc8ee640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d004b640 0x7fa6d00d8bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c280 tx=0x7fa6c4001930 comp rx=0 tx=0).stop 2023-10-11T06:11:21.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.462+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 shutdown_connections 2023-10-11T06:11:21.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.462+0000 7fa6dc8ee640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa6bc06a860 0x7fa6bc06cd10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.463+0000 7fa6dc8ee640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d000dda0 0x7fa6d000e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.463+0000 7fa6dc8ee640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6d004b640 0x7fa6d00d8bc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.463+0000 7fa6dc8ee640 1 --2- 172.21.15.84:0/2919121575 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6c80047c0 0x7fa6d003a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.463+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 >> 172.21.15.84:0/2919121575 conn(0x7fa6d00388f0 msgr2=0x7fa6d0055d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.463+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 shutdown_connections 2023-10-11T06:11:21.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.463+0000 7fa6dc8ee640 1 -- 172.21.15.84:0/2919121575 wait complete. 2023-10-11T06:11:21.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f595c0057d0 con 0x7f597403e870 2023-10-11T06:11:21.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3ba406a920 msgr2=0x7f3ba406cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3ba406a920 0x7f3ba406cdd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40097f0 tx=0x7f3bb4008030 comp rx=0 tx=0).stop 2023-10-11T06:11:21.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 msgr2=0x7f3bc006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc006d1c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800cac0 tx=0x7f3bb800cf80 comp rx=0 tx=0).stop 2023-10-11T06:11:21.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 shutdown_connections 2023-10-11T06:11:21.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3ba406a920 0x7f3ba406cdd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.466+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc0160800 0x7f3bc006d1c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.467+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc00d0a50 0x7f3bc00c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.467+0000 7f3bc74d4640 1 --2- 172.21.15.84:0/557570616 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc00af4d0 0x7f3bc00c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.467+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 >> 172.21.15.84:0/557570616 conn(0x7f3bc0071cc0 msgr2=0x7f3bc00b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.467+0000 7f59637fe640 1 -- 172.21.15.84:0/2200813104 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f596c079f20 con 0x7f597403e870 2023-10-11T06:11:21.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-11T06:11:21.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.467+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 shutdown_connections 2023-10-11T06:11:21.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.467+0000 7f3bc74d4640 1 -- 172.21.15.84:0/557570616 wait complete. 2023-10-11T06:11:21.475 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.475 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-10-11T06:11:21.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-10-11T06:11:21.476 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f596806a860 msgr2=0x7f596806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f596806a860 0x7f596806cd10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f597403d1d0 tx=0x7f5970009040 comp rx=0 tx=0).stop 2023-10-11T06:11:21.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f597403e870 msgr2=0x7f59740dc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f597403e870 0x7f59740dc3e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f596c0099a0 tx=0x7f596c0046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 shutdown_connections 2023-10-11T06:11:21.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f596806a860 0x7f596806cd10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5974055670 0x7f597403a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5974055220 0x7f5974039e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 --2- 172.21.15.84:0/2200813104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f597403e870 0x7f59740dc3e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 >> 172.21.15.84:0/2200813104 conn(0x7f5974055a70 msgr2=0x7f597404f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 shutdown_connections 2023-10-11T06:11:21.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.471+0000 7f59812b4640 1 -- 172.21.15.84:0/2200813104 wait complete. 2023-10-11T06:11:21.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-10-11T06:11:21.481 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-11T06:11:21.483 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-10-11T06:11:21.491 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-10-11T06:11:21.492 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-11T06:11:21.495 DEBUG:teuthology.parallel:result is None 2023-10-11T06:11:21.495 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-11T06:11:21.496 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:11:21.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.747+0000 7f0a2b241640 1 Processor -- start 2023-10-11T06:11:21.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.747+0000 7f0a2b241640 1 -- start start 2023-10-11T06:11:21.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.748+0000 7f0a2b241640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240ce0d0 0x7f0a240c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.748+0000 7f0a2b241640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0a240adb90 0x7f0a240caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.748+0000 7f0a2b241640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a241593f0 con 0x7f0a240cd300 2023-10-11T06:11:21.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.748+0000 7f0a2b241640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2415a340 con 0x7f0a240adb90 2023-10-11T06:11:21.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.748+0000 7f0a2b241640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240ce4a0 con 0x7f0a240ce0d0 2023-10-11T06:11:21.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a28fdd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240ce0d0 0x7f0a240c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a28fdd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240ce0d0 0x7f0a240c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:53464/0 (socket says 172.21.15.84:53464) 2023-10-11T06:11:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a28fdd640 1 -- 172.21.15.84:0/236876699 learned_addr learned my addr 172.21.15.84:0/236876699 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a23fff640 1 --2- 172.21.15.84:0/236876699 >> v2:172.21.15.99:3300/0 conn(0x7f0a240adb90 0x7f0a240caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a23fff640 1 -- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240ce0d0 msgr2=0x7f0a240c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a23fff640 1 --2- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240ce0d0 0x7f0a240c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a23fff640 1 -- 172.21.15.84:0/236876699 >> v1:172.21.15.99:6789/0 conn(0x7f0a240cd300 legacy=0x7f0a240b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a23fff640 1 -- 172.21.15.84:0/236876699 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a241596c0 con 0x7f0a240adb90 2023-10-11T06:11:21.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.749+0000 7f0a28fdd640 1 --2- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240ce0d0 0x7f0a240c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:21.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.751+0000 7f0a23fff640 1 --2- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240adb90 0x7f0a240caee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a100097a0 tx=0x7f0a1002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec38db4b60b5900 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.751+0000 7f0a237fe640 1 -- 172.21.15.84:0/236876699 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1002d040 con 0x7f0a240adb90 2023-10-11T06:11:21.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.751+0000 7f0a237fe640 1 -- 172.21.15.84:0/236876699 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10031810 con 0x7f0a240adb90 2023-10-11T06:11:21.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.751+0000 7f0a237fe640 1 -- 172.21.15.84:0/236876699 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10031ba0 con 0x7f0a240adb90 2023-10-11T06:11:21.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 -- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240adb90 msgr2=0x7f0a240caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 --2- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240adb90 0x7f0a240caee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a100097a0 tx=0x7f0a1002b8b0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 -- 172.21.15.84:0/236876699 shutdown_connections 2023-10-11T06:11:21.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 --2- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240adb90 0x7f0a240caee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 --2- 172.21.15.84:0/236876699 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240ce0d0 0x7f0a240c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 -- 172.21.15.84:0/236876699 >> 172.21.15.84:0/236876699 conn(0x7f0a24069c70 msgr2=0x7f0a2406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 -- 172.21.15.84:0/236876699 shutdown_connections 2023-10-11T06:11:21.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.752+0000 7f0a2b241640 1 -- 172.21.15.84:0/236876699 wait complete. 2023-10-11T06:11:21.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.754+0000 7f0a2b241640 1 Processor -- start 2023-10-11T06:11:21.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.754+0000 7f0a2b241640 1 -- start start 2023-10-11T06:11:21.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a2b241640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a240adb90 0x7f0a2415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a2b241640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240cd300 0x7f0a240c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a2b241640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ce0d0 0x7f0a2414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a2b241640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240ae7a0 con 0x7f0a240adb90 2023-10-11T06:11:21.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a2b241640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240c81f0 con 0x7f0a240ce0d0 2023-10-11T06:11:21.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a2b241640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240c84d0 con 0x7f0a240cd300 2023-10-11T06:11:21.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a240adb90 0x7f0a2415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a240adb90 0x7f0a2415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54020/0 (socket says 172.21.15.84:54020) 2023-10-11T06:11:21.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 -- 172.21.15.84:0/3581880330 learned_addr learned my addr 172.21.15.84:0/3581880330 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:21.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a23fff640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240cd300 0x7f0a240c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 -- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240cd300 msgr2=0x7f0a240c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240cd300 0x7f0a240c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 -- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ce0d0 msgr2=0x7f0a2414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ce0d0 0x7f0a2414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a28fdd640 1 -- 172.21.15.84:0/3581880330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1800bd40 con 0x7f0a240adb90 2023-10-11T06:11:21.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.755+0000 7f0a23fff640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240cd300 0x7f0a240c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:21.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.756+0000 7f0a28fdd640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a240adb90 0x7f0a2415a910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a180103f0 tx=0x7f0a180108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.756+0000 7f0a217fa640 1 -- 172.21.15.84:0/3581880330 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18009600 con 0x7f0a240adb90 2023-10-11T06:11:21.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.756+0000 7f0a217fa640 1 -- 172.21.15.84:0/3581880330 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a18011040 con 0x7f0a240adb90 2023-10-11T06:11:21.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.756+0000 7f0a217fa640 1 -- 172.21.15.84:0/3581880330 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18009960 con 0x7f0a240adb90 2023-10-11T06:11:21.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.756+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a10009400 con 0x7f0a240adb90 2023-10-11T06:11:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.756+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a240aecd0 con 0x7f0a240adb90 2023-10-11T06:11:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.757+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a240aee60 con 0x7f0a240adb90 2023-10-11T06:11:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.758+0000 7f0a217fa640 1 -- 172.21.15.84:0/3581880330 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0a18009ac0 con 0x7f0a240adb90 2023-10-11T06:11:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.759+0000 7f0a217fa640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a0806a830 0x7f0a0806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.759+0000 7f0a217fa640 1 -- 172.21.15.84:0/3581880330 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a1808bda0 con 0x7f0a240adb90 2023-10-11T06:11:21.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.759+0000 7f0a23fff640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a0806a830 0x7f0a0806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:21.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.760+0000 7f0a23fff640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a0806a830 0x7f0a0806cce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a10036000 tx=0x7f0a1003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:21.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.764+0000 7f0a217fa640 1 -- 172.21.15.84:0/3581880330 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0a18056f80 con 0x7f0a240adb90 2023-10-11T06:11:21.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.940+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a24159c80 con 0x7f0a0806a830 2023-10-11T06:11:21.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.962+0000 7f0a217fa640 1 -- 172.21.15.84:0/3581880330 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286490 (secure 0 0 0) 0x7f0a1003a420 con 0x7f0a0806a830 2023-10-11T06:11:21.968 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:21.969 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:11:21.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.968+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a0806a830 msgr2=0x7f0a0806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.968+0000 7f0a2b241640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a0806a830 0x7f0a0806cce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a10036000 tx=0x7f0a1003c040 comp rx=0 tx=0).stop 2023-10-11T06:11:21.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.968+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a240adb90 msgr2=0x7f0a2415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:21.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.968+0000 7f0a2b241640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a240adb90 0x7f0a2415a910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a180103f0 tx=0x7f0a180108b0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 shutdown_connections 2023-10-11T06:11:21.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a0806a830 0x7f0a0806cce0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a240ce0d0 0x7f0a2414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a240cd300 0x7f0a240c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 --2- 172.21.15.84:0/3581880330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a240adb90 0x7f0a2415a910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:21.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 >> 172.21.15.84:0/3581880330 conn(0x7f0a24069c70 msgr2=0x7f0a2406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:21.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 shutdown_connections 2023-10-11T06:11:21.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:21.969+0000 7f0a2b241640 1 -- 172.21.15.84:0/3581880330 wait complete. 2023-10-11T06:11:21.988 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-11T06:11:20.773293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218824,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":3284992,"data_stored":1939620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.593672"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688857+0000","last_change":"2023-10-11T06:11:13.702417+0000","last_active":"2023-10-11T06:11:14.688857+0000","last_peered":"2023-10-11T06:11:14.688857+0000","last_clean":"2023-10-11T06:11:14.688857+0000","last_became_active":"2023-10-11T06:11:13.702128+0000","last_became_peered":"2023-10-11T06:11:13.702128+0000","last_unstale":"2023-10-11T06:11:14.688857+0000","last_undegraded":"2023-10-11T06:11:14.688857+0000","last_fullsized":"2023-10-11T06:11:14.688857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:26.874292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935985+0000","last_change":"2023-10-11T06:11:13.701166+0000","last_active":"2023-10-11T06:11:14.935985+0000","last_peered":"2023-10-11T06:11:14.935985+0000","last_clean":"2023-10-11T06:11:14.935985+0000","last_became_active":"2023-10-11T06:11:13.700706+0000","last_became_peered":"2023-10-11T06:11:13.700706+0000","last_unstale":"2023-10-11T06:11:14.935985+0000","last_undegraded":"2023-10-11T06:11:14.935985+0000","last_fullsized":"2023-10-11T06:11:14.935985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937001+0000","last_change":"2023-10-11T06:11:13.701254+0000","last_active":"2023-10-11T06:11:14.937001+0000","last_peered":"2023-10-11T06:11:14.937001+0000","last_clean":"2023-10-11T06:11:14.937001+0000","last_became_active":"2023-10-11T06:11:13.700986+0000","last_became_peered":"2023-10-11T06:11:13.700986+0000","last_unstale":"2023-10-11T06:11:14.937001+0000","last_undegraded":"2023-10-11T06:11:14.937001+0000","last_fullsized":"2023-10-11T06:11:14.937001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:08:07.089082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689491+0000","last_change":"2023-10-11T06:11:13.692093+0000","last_active":"2023-10-11T06:11:14.689491+0000","last_peered":"2023-10-11T06:11:14.689491+0000","last_clean":"2023-10-11T06:11:14.689491+0000","last_became_active":"2023-10-11T06:11:13.691749+0000","last_became_peered":"2023-10-11T06:11:13.691749+0000","last_unstale":"2023-10-11T06:11:14.689491+0000","last_undegraded":"2023-10-11T06:11:14.689491+0000","last_fullsized":"2023-10-11T06:11:14.689491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933130+0000","last_change":"2023-10-11T06:11:13.706178+0000","last_active":"2023-10-11T06:11:14.933130+0000","last_peered":"2023-10-11T06:11:14.933130+0000","last_clean":"2023-10-11T06:11:14.933130+0000","last_became_active":"2023-10-11T06:11:13.705917+0000","last_became_peered":"2023-10-11T06:11:13.705917+0000","last_unstale":"2023-10-11T06:11:14.933130+0000","last_undegraded":"2023-10-11T06:11:14.933130+0000","last_fullsized":"2023-10-11T06:11:14.933130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935149+0000","last_change":"2023-10-11T06:11:13.702491+0000","last_active":"2023-10-11T06:11:14.935149+0000","last_peered":"2023-10-11T06:11:14.935149+0000","last_clean":"2023-10-11T06:11:14.935149+0000","last_became_active":"2023-10-11T06:11:13.701716+0000","last_became_peered":"2023-10-11T06:11:13.701716+0000","last_unstale":"2023-10-11T06:11:14.935149+0000","last_undegraded":"2023-10-11T06:11:14.935149+0000","last_fullsized":"2023-10-11T06:11:14.935149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:16:18.652379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934429+0000","last_change":"2023-10-11T06:11:13.713375+0000","last_active":"2023-10-11T06:11:14.934429+0000","last_peered":"2023-10-11T06:11:14.934429+0000","last_clean":"2023-10-11T06:11:14.934429+0000","last_became_active":"2023-10-11T06:11:13.712800+0000","last_became_peered":"2023-10-11T06:11:13.712800+0000","last_unstale":"2023-10-11T06:11:14.934429+0000","last_undegraded":"2023-10-11T06:11:14.934429+0000","last_fullsized":"2023-10-11T06:11:14.934429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934085+0000","last_change":"2023-10-11T06:11:13.700021+0000","last_active":"2023-10-11T06:11:14.934085+0000","last_peered":"2023-10-11T06:11:14.934085+0000","last_clean":"2023-10-11T06:11:14.934085+0000","last_became_active":"2023-10-11T06:11:13.699710+0000","last_became_peered":"2023-10-11T06:11:13.699710+0000","last_unstale":"2023-10-11T06:11:14.934085+0000","last_undegraded":"2023-10-11T06:11:14.934085+0000","last_fullsized":"2023-10-11T06:11:14.934085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933410+0000","last_change":"2023-10-11T06:11:13.705628+0000","last_active":"2023-10-11T06:11:14.933410+0000","last_peered":"2023-10-11T06:11:14.933410+0000","last_clean":"2023-10-11T06:11:14.933410+0000","last_became_active":"2023-10-11T06:11:13.705205+0000","last_became_peered":"2023-10-11T06:11:13.705205+0000","last_unstale":"2023-10-11T06:11:14.933410+0000","last_undegraded":"2023-10-11T06:11:14.933410+0000","last_fullsized":"2023-10-11T06:11:14.933410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936060+0000","last_change":"2023-10-11T06:11:13.700979+0000","last_active":"2023-10-11T06:11:14.936060+0000","last_peered":"2023-10-11T06:11:14.936060+0000","last_clean":"2023-10-11T06:11:14.936060+0000","last_became_active":"2023-10-11T06:11:13.700344+0000","last_became_peered":"2023-10-11T06:11:13.700344+0000","last_unstale":"2023-10-11T06:11:14.936060+0000","last_undegraded":"2023-10-11T06:11:14.936060+0000","last_fullsized":"2023-10-11T06:11:14.936060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937234+0000","last_change":"2023-10-11T06:11:13.707695+0000","last_active":"2023-10-11T06:11:14.937234+0000","last_peered":"2023-10-11T06:11:14.937234+0000","last_clean":"2023-10-11T06:11:14.937234+0000","last_became_active":"2023-10-11T06:11:13.707358+0000","last_became_peered":"2023-10-11T06:11:13.707358+0000","last_unstale":"2023-10-11T06:11:14.937234+0000","last_undegraded":"2023-10-11T06:11:14.937234+0000","last_fullsized":"2023-10-11T06:11:14.937234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:38:53.832203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688144+0000","last_change":"2023-10-11T06:11:13.705483+0000","last_active":"2023-10-11T06:11:14.688144+0000","last_peered":"2023-10-11T06:11:14.688144+0000","last_clean":"2023-10-11T06:11:14.688144+0000","last_became_active":"2023-10-11T06:11:13.705228+0000","last_became_peered":"2023-10-11T06:11:13.705228+0000","last_unstale":"2023-10-11T06:11:14.688144+0000","last_undegraded":"2023-10-11T06:11:14.688144+0000","last_fullsized":"2023-10-11T06:11:14.688144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690422+0000","last_change":"2023-10-11T06:11:13.701448+0000","last_active":"2023-10-11T06:11:14.690422+0000","last_peered":"2023-10-11T06:11:14.690422+0000","last_clean":"2023-10-11T06:11:14.690422+0000","last_became_active":"2023-10-11T06:11:13.701186+0000","last_became_peered":"2023-10-11T06:11:13.701186+0000","last_unstale":"2023-10-11T06:11:14.690422+0000","last_undegraded":"2023-10-11T06:11:14.690422+0000","last_fullsized":"2023-10-11T06:11:14.690422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:01.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935482+0000","last_change":"2023-10-11T06:11:13.703675+0000","last_active":"2023-10-11T06:11:14.935482+0000","last_peered":"2023-10-11T06:11:14.935482+0000","last_clean":"2023-10-11T06:11:14.935482+0000","last_became_active":"2023-10-11T06:11:13.703365+0000","last_became_peered":"2023-10-11T06:11:13.703365+0000","last_unstale":"2023-10-11T06:11:14.935482+0000","last_undegraded":"2023-10-11T06:11:14.935482+0000","last_fullsized":"2023-10-11T06:11:14.935482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934721+0000","last_change":"2023-10-11T06:11:13.713483+0000","last_active":"2023-10-11T06:11:14.934721+0000","last_peered":"2023-10-11T06:11:14.934721+0000","last_clean":"2023-10-11T06:11:14.934721+0000","last_became_active":"2023-10-11T06:11:13.712594+0000","last_became_peered":"2023-10-11T06:11:13.712594+0000","last_unstale":"2023-10-11T06:11:14.934721+0000","last_undegraded":"2023-10-11T06:11:14.934721+0000","last_fullsized":"2023-10-11T06:11:14.934721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935163+0000","last_change":"2023-10-11T06:11:13.704217+0000","last_active":"2023-10-11T06:11:14.935163+0000","last_peered":"2023-10-11T06:11:14.935163+0000","last_clean":"2023-10-11T06:11:14.935163+0000","last_became_active":"2023-10-11T06:11:13.702922+0000","last_became_peered":"2023-10-11T06:11:13.702922+0000","last_unstale":"2023-10-11T06:11:14.935163+0000","last_undegraded":"2023-10-11T06:11:14.935163+0000","last_fullsized":"2023-10-11T06:11:14.935163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:10.963337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933536+0000","last_change":"2023-10-11T06:11:13.702292+0000","last_active":"2023-10-11T06:11:14.933536+0000","last_peered":"2023-10-11T06:11:14.933536+0000","last_clean":"2023-10-11T06:11:14.933536+0000","last_became_active":"2023-10-11T06:11:13.702060+0000","last_became_peered":"2023-10-11T06:11:13.702060+0000","last_unstale":"2023-10-11T06:11:14.933536+0000","last_undegraded":"2023-10-11T06:11:14.933536+0000","last_fullsized":"2023-10-11T06:11:14.933536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936185+0000","last_change":"2023-10-11T06:11:13.694348+0000","last_active":"2023-10-11T06:11:14.936185+0000","last_peered":"2023-10-11T06:11:14.936185+0000","last_clean":"2023-10-11T06:11:14.936185+0000","last_became_active":"2023-10-11T06:11:13.694074+0000","last_became_peered":"2023-10-11T06:11:13.694074+0000","last_unstale":"2023-10-11T06:11:14.936185+0000","last_undegraded":"2023-10-11T06:11:14.936185+0000","last_fullsized":"2023-10-11T06:11:14.936185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689126+0000","last_change":"2023-10-11T06:11:13.693722+0000","last_active":"2023-10-11T06:11:14.689126+0000","last_peered":"2023-10-11T06:11:14.689126+0000","last_clean":"2023-10-11T06:11:14.689126+0000","last_became_active":"2023-10-11T06:11:13.693338+0000","last_became_peered":"2023-10-11T06:11:13.693338+0000","last_unstale":"2023-10-11T06:11:14.689126+0000","last_undegraded":"2023-10-11T06:11:14.689126+0000","last_fullsized":"2023-10-11T06:11:14.689126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936628+0000","last_change":"2023-10-11T06:11:13.692755+0000","last_active":"2023-10-11T06:11:14.936628+0000","last_peered":"2023-10-11T06:11:14.936628+0000","last_clean":"2023-10-11T06:11:14.936628+0000","last_became_active":"2023-10-11T06:11:13.692467+0000","last_became_peered":"2023-10-11T06:11:13.692467+0000","last_unstale":"2023-10-11T06:11:14.936628+0000","last_undegraded":"2023-10-11T06:11:14.936628+0000","last_fullsized":"2023-10-11T06:11:14.936628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:12.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934603+0000","last_change":"2023-10-11T06:11:13.712835+0000","last_active":"2023-10-11T06:11:14.934603+0000","last_peered":"2023-10-11T06:11:14.934603+0000","last_clean":"2023-10-11T06:11:14.934603+0000","last_became_active":"2023-10-11T06:11:13.712498+0000","last_became_peered":"2023-10-11T06:11:13.712498+0000","last_unstale":"2023-10-11T06:11:14.934603+0000","last_undegraded":"2023-10-11T06:11:14.934603+0000","last_fullsized":"2023-10-11T06:11:14.934603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932549+0000","last_change":"2023-10-11T06:11:13.700139+0000","last_active":"2023-10-11T06:11:14.932549+0000","last_peered":"2023-10-11T06:11:14.932549+0000","last_clean":"2023-10-11T06:11:14.932549+0000","last_became_active":"2023-10-11T06:11:13.699888+0000","last_became_peered":"2023-10-11T06:11:13.699888+0000","last_unstale":"2023-10-11T06:11:14.932549+0000","last_undegraded":"2023-10-11T06:11:14.932549+0000","last_fullsized":"2023-10-11T06:11:14.932549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687599+0000","last_change":"2023-10-11T06:11:13.694139+0000","last_active":"2023-10-11T06:11:14.687599+0000","last_peered":"2023-10-11T06:11:14.687599+0000","last_clean":"2023-10-11T06:11:14.687599+0000","last_became_active":"2023-10-11T06:11:13.693870+0000","last_became_peered":"2023-10-11T06:11:13.693870+0000","last_unstale":"2023-10-11T06:11:14.687599+0000","last_undegraded":"2023-10-11T06:11:14.687599+0000","last_fullsized":"2023-10-11T06:11:14.687599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934975+0000","last_change":"2023-10-11T06:11:13.713679+0000","last_active":"2023-10-11T06:11:14.934975+0000","last_peered":"2023-10-11T06:11:14.934975+0000","last_clean":"2023-10-11T06:11:14.934975+0000","last_became_active":"2023-10-11T06:11:13.712476+0000","last_became_peered":"2023-10-11T06:11:13.712476+0000","last_unstale":"2023-10-11T06:11:14.934975+0000","last_undegraded":"2023-10-11T06:11:14.934975+0000","last_fullsized":"2023-10-11T06:11:14.934975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934737+0000","last_change":"2023-10-11T06:11:13.693952+0000","last_active":"2023-10-11T06:11:14.934737+0000","last_peered":"2023-10-11T06:11:14.934737+0000","last_clean":"2023-10-11T06:11:14.934737+0000","last_became_active":"2023-10-11T06:11:13.693582+0000","last_became_peered":"2023-10-11T06:11:13.693582+0000","last_unstale":"2023-10-11T06:11:14.934737+0000","last_undegraded":"2023-10-11T06:11:14.934737+0000","last_fullsized":"2023-10-11T06:11:14.934737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934890+0000","last_change":"2023-10-11T06:11:13.694225+0000","last_active":"2023-10-11T06:11:14.934890+0000","last_peered":"2023-10-11T06:11:14.934890+0000","last_clean":"2023-10-11T06:11:14.934890+0000","last_became_active":"2023-10-11T06:11:13.693965+0000","last_became_peered":"2023-10-11T06:11:13.693965+0000","last_unstale":"2023-10-11T06:11:14.934890+0000","last_undegraded":"2023-10-11T06:11:14.934890+0000","last_fullsized":"2023-10-11T06:11:14.934890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:07.237906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935270+0000","last_change":"2023-10-11T06:11:13.710202+0000","last_active":"2023-10-11T06:11:14.935270+0000","last_peered":"2023-10-11T06:11:14.935270+0000","last_clean":"2023-10-11T06:11:14.935270+0000","last_became_active":"2023-10-11T06:11:13.709963+0000","last_became_peered":"2023-10-11T06:11:13.709963+0000","last_unstale":"2023-10-11T06:11:14.935270+0000","last_undegraded":"2023-10-11T06:11:14.935270+0000","last_fullsized":"2023-10-11T06:11:14.935270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935088+0000","last_change":"2023-10-11T06:11:13.709433+0000","last_active":"2023-10-11T06:11:14.935088+0000","last_peered":"2023-10-11T06:11:14.935088+0000","last_clean":"2023-10-11T06:11:14.935088+0000","last_became_active":"2023-10-11T06:11:13.709162+0000","last_became_peered":"2023-10-11T06:11:13.709162+0000","last_unstale":"2023-10-11T06:11:14.935088+0000","last_undegraded":"2023-10-11T06:11:14.935088+0000","last_fullsized":"2023-10-11T06:11:14.935088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688453+0000","last_change":"2023-10-11T06:11:13.709986+0000","last_active":"2023-10-11T06:11:14.688453+0000","last_peered":"2023-10-11T06:11:14.688453+0000","last_clean":"2023-10-11T06:11:14.688453+0000","last_became_active":"2023-10-11T06:11:13.709545+0000","last_became_peered":"2023-10-11T06:11:13.709545+0000","last_unstale":"2023-10-11T06:11:14.688453+0000","last_undegraded":"2023-10-11T06:11:14.688453+0000","last_fullsized":"2023-10-11T06:11:14.688453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935870+0000","last_change":"2023-10-11T06:11:13.690029+0000","last_active":"2023-10-11T06:11:14.935870+0000","last_peered":"2023-10-11T06:11:14.935870+0000","last_clean":"2023-10-11T06:11:14.935870+0000","last_became_active":"2023-10-11T06:11:13.689452+0000","last_became_peered":"2023-10-11T06:11:13.689452+0000","last_unstale":"2023-10-11T06:11:14.935870+0000","last_undegraded":"2023-10-11T06:11:14.935870+0000","last_fullsized":"2023-10-11T06:11:14.935870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:44:24.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687703+0000","last_change":"2023-10-11T06:11:13.697848+0000","last_active":"2023-10-11T06:11:14.687703+0000","last_peered":"2023-10-11T06:11:14.687703+0000","last_clean":"2023-10-11T06:11:14.687703+0000","last_became_active":"2023-10-11T06:11:13.697611+0000","last_became_peered":"2023-10-11T06:11:13.697611+0000","last_unstale":"2023-10-11T06:11:14.687703+0000","last_undegraded":"2023-10-11T06:11:14.687703+0000","last_fullsized":"2023-10-11T06:11:14.687703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689328+0000","last_change":"2023-10-11T06:11:13.696436+0000","last_active":"2023-10-11T06:11:14.689328+0000","last_peered":"2023-10-11T06:11:14.689328+0000","last_clean":"2023-10-11T06:11:14.689328+0000","last_became_active":"2023-10-11T06:11:13.696110+0000","last_became_peered":"2023-10-11T06:11:13.696110+0000","last_unstale":"2023-10-11T06:11:14.689328+0000","last_undegraded":"2023-10-11T06:11:14.689328+0000","last_fullsized":"2023-10-11T06:11:14.689328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689195+0000","last_change":"2023-10-11T06:11:13.697298+0000","last_active":"2023-10-11T06:11:14.689195+0000","last_peered":"2023-10-11T06:11:14.689195+0000","last_clean":"2023-10-11T06:11:14.689195+0000","last_became_active":"2023-10-11T06:11:13.697002+0000","last_became_peered":"2023-10-11T06:11:13.697002+0000","last_unstale":"2023-10-11T06:11:14.689195+0000","last_undegraded":"2023-10-11T06:11:14.689195+0000","last_fullsized":"2023-10-11T06:11:14.689195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937247+0000","last_change":"2023-10-11T06:11:13.221800+0000","last_active":"2023-10-11T06:11:14.937247+0000","last_peered":"2023-10-11T06:11:14.937247+0000","last_clean":"2023-10-11T06:11:14.937247+0000","last_became_active":"2023-10-11T06:11:13.221375+0000","last_became_peered":"2023-10-11T06:11:13.221375+0000","last_unstale":"2023-10-11T06:11:14.937247+0000","last_undegraded":"2023-10-11T06:11:14.937247+0000","last_fullsized":"2023-10-11T06:11:14.937247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:57.513899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935532+0000","last_change":"2023-10-11T06:11:13.706371+0000","last_active":"2023-10-11T06:11:14.935532+0000","last_peered":"2023-10-11T06:11:14.935532+0000","last_clean":"2023-10-11T06:11:14.935532+0000","last_became_active":"2023-10-11T06:11:13.706038+0000","last_became_peered":"2023-10-11T06:11:13.706038+0000","last_unstale":"2023-10-11T06:11:14.935532+0000","last_undegraded":"2023-10-11T06:11:14.935532+0000","last_fullsized":"2023-10-11T06:11:14.935532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932620+0000","last_change":"2023-10-11T06:11:13.217589+0000","last_active":"2023-10-11T06:11:14.932620+0000","last_peered":"2023-10-11T06:11:14.932620+0000","last_clean":"2023-10-11T06:11:14.932620+0000","last_became_active":"2023-10-11T06:11:13.217168+0000","last_became_peered":"2023-10-11T06:11:13.217168+0000","last_unstale":"2023-10-11T06:11:14.932620+0000","last_undegraded":"2023-10-11T06:11:14.932620+0000","last_fullsized":"2023-10-11T06:11:14.932620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933735+0000","last_change":"2023-10-11T06:11:13.703040+0000","last_active":"2023-10-11T06:11:14.933735+0000","last_peered":"2023-10-11T06:11:14.933735+0000","last_clean":"2023-10-11T06:11:14.933735+0000","last_became_active":"2023-10-11T06:11:13.702687+0000","last_became_peered":"2023-10-11T06:11:13.702687+0000","last_unstale":"2023-10-11T06:11:14.933735+0000","last_undegraded":"2023-10-11T06:11:14.933735+0000","last_fullsized":"2023-10-11T06:11:14.933735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934835+0000","last_change":"2023-10-11T06:11:13.221450+0000","last_active":"2023-10-11T06:11:14.934835+0000","last_peered":"2023-10-11T06:11:14.934835+0000","last_clean":"2023-10-11T06:11:14.934835+0000","last_became_active":"2023-10-11T06:11:13.221110+0000","last_became_peered":"2023-10-11T06:11:13.221110+0000","last_unstale":"2023-10-11T06:11:14.934835+0000","last_undegraded":"2023-10-11T06:11:14.934835+0000","last_fullsized":"2023-10-11T06:11:14.934835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:48.142934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690013+0000","last_change":"2023-10-11T06:11:13.710650+0000","last_active":"2023-10-11T06:11:14.690013+0000","last_peered":"2023-10-11T06:11:14.690013+0000","last_clean":"2023-10-11T06:11:14.690013+0000","last_became_active":"2023-10-11T06:11:13.710414+0000","last_became_peered":"2023-10-11T06:11:13.710414+0000","last_unstale":"2023-10-11T06:11:14.690013+0000","last_undegraded":"2023-10-11T06:11:14.690013+0000","last_fullsized":"2023-10-11T06:11:14.690013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689695+0000","last_change":"2023-10-11T06:11:13.209784+0000","last_active":"2023-10-11T06:11:14.689695+0000","last_peered":"2023-10-11T06:11:14.689695+0000","last_clean":"2023-10-11T06:11:14.689695+0000","last_became_active":"2023-10-11T06:11:13.209455+0000","last_became_peered":"2023-10-11T06:11:13.209455+0000","last_unstale":"2023-10-11T06:11:14.689695+0000","last_undegraded":"2023-10-11T06:11:14.689695+0000","last_fullsized":"2023-10-11T06:11:14.689695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937957+0000","last_change":"2023-10-11T06:11:13.221554+0000","last_active":"2023-10-11T06:11:14.937957+0000","last_peered":"2023-10-11T06:11:14.937957+0000","last_clean":"2023-10-11T06:11:14.937957+0000","last_became_active":"2023-10-11T06:11:13.221257+0000","last_became_peered":"2023-10-11T06:11:13.221257+0000","last_unstale":"2023-10-11T06:11:14.937957+0000","last_undegraded":"2023-10-11T06:11:14.937957+0000","last_fullsized":"2023-10-11T06:11:14.937957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:43.560921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934456+0000","last_change":"2023-10-11T06:11:13.706727+0000","last_active":"2023-10-11T06:11:14.934456+0000","last_peered":"2023-10-11T06:11:14.934456+0000","last_clean":"2023-10-11T06:11:14.934456+0000","last_became_active":"2023-10-11T06:11:13.706016+0000","last_became_peered":"2023-10-11T06:11:13.706016+0000","last_unstale":"2023-10-11T06:11:14.934456+0000","last_undegraded":"2023-10-11T06:11:14.934456+0000","last_fullsized":"2023-10-11T06:11:14.934456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937828+0000","last_change":"2023-10-11T06:11:13.222531+0000","last_active":"2023-10-11T06:11:14.937828+0000","last_peered":"2023-10-11T06:11:14.937828+0000","last_clean":"2023-10-11T06:11:14.937828+0000","last_became_active":"2023-10-11T06:11:13.222126+0000","last_became_peered":"2023-10-11T06:11:13.222126+0000","last_unstale":"2023-10-11T06:11:14.937828+0000","last_undegraded":"2023-10-11T06:11:14.937828+0000","last_fullsized":"2023-10-11T06:11:14.937828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:03.840067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935577+0000","last_change":"2023-10-11T06:11:13.700622+0000","last_active":"2023-10-11T06:11:14.935577+0000","last_peered":"2023-10-11T06:11:14.935577+0000","last_clean":"2023-10-11T06:11:14.935577+0000","last_became_active":"2023-10-11T06:11:13.699582+0000","last_became_peered":"2023-10-11T06:11:13.699582+0000","last_unstale":"2023-10-11T06:11:14.935577+0000","last_undegraded":"2023-10-11T06:11:14.935577+0000","last_fullsized":"2023-10-11T06:11:14.935577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937834+0000","last_change":"2023-10-11T06:11:13.220125+0000","last_active":"2023-10-11T06:11:14.937834+0000","last_peered":"2023-10-11T06:11:14.937834+0000","last_clean":"2023-10-11T06:11:14.937834+0000","last_became_active":"2023-10-11T06:11:13.219575+0000","last_became_peered":"2023-10-11T06:11:13.219575+0000","last_unstale":"2023-10-11T06:11:14.937834+0000","last_undegraded":"2023-10-11T06:11:14.937834+0000","last_fullsized":"2023-10-11T06:11:14.937834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:06.223280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689340+0000","last_change":"2023-10-11T06:11:13.697891+0000","last_active":"2023-10-11T06:11:14.689340+0000","last_peered":"2023-10-11T06:11:14.689340+0000","last_clean":"2023-10-11T06:11:14.689340+0000","last_became_active":"2023-10-11T06:11:13.697630+0000","last_became_peered":"2023-10-11T06:11:13.697630+0000","last_unstale":"2023-10-11T06:11:14.689340+0000","last_undegraded":"2023-10-11T06:11:14.689340+0000","last_fullsized":"2023-10-11T06:11:14.689340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935004+0000","last_change":"2023-10-11T06:11:13.207215+0000","last_active":"2023-10-11T06:11:14.935004+0000","last_peered":"2023-10-11T06:11:14.935004+0000","last_clean":"2023-10-11T06:11:14.935004+0000","last_became_active":"2023-10-11T06:11:13.206865+0000","last_became_peered":"2023-10-11T06:11:13.206865+0000","last_unstale":"2023-10-11T06:11:14.935004+0000","last_undegraded":"2023-10-11T06:11:14.935004+0000","last_fullsized":"2023-10-11T06:11:14.935004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932758+0000","last_change":"2023-10-11T06:11:13.699849+0000","last_active":"2023-10-11T06:11:14.932758+0000","last_peered":"2023-10-11T06:11:14.932758+0000","last_clean":"2023-10-11T06:11:14.932758+0000","last_became_active":"2023-10-11T06:11:13.699499+0000","last_became_peered":"2023-10-11T06:11:13.699499+0000","last_unstale":"2023-10-11T06:11:14.932758+0000","last_undegraded":"2023-10-11T06:11:14.932758+0000","last_fullsized":"2023-10-11T06:11:14.932758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688855+0000","last_change":"2023-10-11T06:11:13.209002+0000","last_active":"2023-10-11T06:11:14.688855+0000","last_peered":"2023-10-11T06:11:14.688855+0000","last_clean":"2023-10-11T06:11:14.688855+0000","last_became_active":"2023-10-11T06:11:13.208587+0000","last_became_peered":"2023-10-11T06:11:13.208587+0000","last_unstale":"2023-10-11T06:11:14.688855+0000","last_undegraded":"2023-10-11T06:11:14.688855+0000","last_fullsized":"2023-10-11T06:11:14.688855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937054+0000","last_change":"2023-10-11T06:11:13.698914+0000","last_active":"2023-10-11T06:11:14.937054+0000","last_peered":"2023-10-11T06:11:14.937054+0000","last_clean":"2023-10-11T06:11:14.937054+0000","last_became_active":"2023-10-11T06:11:13.697742+0000","last_became_peered":"2023-10-11T06:11:13.697742+0000","last_unstale":"2023-10-11T06:11:14.937054+0000","last_undegraded":"2023-10-11T06:11:14.937054+0000","last_fullsized":"2023-10-11T06:11:14.937054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689617+0000","last_change":"2023-10-11T06:11:13.197306+0000","last_active":"2023-10-11T06:11:14.689617+0000","last_peered":"2023-10-11T06:11:14.689617+0000","last_clean":"2023-10-11T06:11:14.689617+0000","last_became_active":"2023-10-11T06:11:13.196154+0000","last_became_peered":"2023-10-11T06:11:13.196154+0000","last_unstale":"2023-10-11T06:11:14.689617+0000","last_undegraded":"2023-10-11T06:11:14.689617+0000","last_fullsized":"2023-10-11T06:11:14.689617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937482+0000","last_change":"2023-10-11T06:11:13.694445+0000","last_active":"2023-10-11T06:11:14.937482+0000","last_peered":"2023-10-11T06:11:14.937482+0000","last_clean":"2023-10-11T06:11:14.937482+0000","last_became_active":"2023-10-11T06:11:13.694063+0000","last_became_peered":"2023-10-11T06:11:13.694063+0000","last_unstale":"2023-10-11T06:11:14.937482+0000","last_undegraded":"2023-10-11T06:11:14.937482+0000","last_fullsized":"2023-10-11T06:11:14.937482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936267+0000","last_change":"2023-10-11T06:11:13.212919+0000","last_active":"2023-10-11T06:11:14.936267+0000","last_peered":"2023-10-11T06:11:14.936267+0000","last_clean":"2023-10-11T06:11:14.936267+0000","last_became_active":"2023-10-11T06:11:13.212064+0000","last_became_peered":"2023-10-11T06:11:13.212064+0000","last_unstale":"2023-10-11T06:11:14.936267+0000","last_undegraded":"2023-10-11T06:11:14.936267+0000","last_fullsized":"2023-10-11T06:11:14.936267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936642+0000","last_change":"2023-10-11T06:11:13.692017+0000","last_active":"2023-10-11T06:11:14.936642+0000","last_peered":"2023-10-11T06:11:14.936642+0000","last_clean":"2023-10-11T06:11:14.936642+0000","last_became_active":"2023-10-11T06:11:13.690806+0000","last_became_peered":"2023-10-11T06:11:13.690806+0000","last_unstale":"2023-10-11T06:11:14.936642+0000","last_undegraded":"2023-10-11T06:11:14.936642+0000","last_fullsized":"2023-10-11T06:11:14.936642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688241+0000","last_change":"2023-10-11T06:11:13.213340+0000","last_active":"2023-10-11T06:11:14.688241+0000","last_peered":"2023-10-11T06:11:14.688241+0000","last_clean":"2023-10-11T06:11:14.688241+0000","last_became_active":"2023-10-11T06:11:13.213031+0000","last_became_peered":"2023-10-11T06:11:13.213031+0000","last_unstale":"2023-10-11T06:11:14.688241+0000","last_undegraded":"2023-10-11T06:11:14.688241+0000","last_fullsized":"2023-10-11T06:11:14.688241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934847+0000","last_change":"2023-10-11T06:11:13.694560+0000","last_active":"2023-10-11T06:11:14.934847+0000","last_peered":"2023-10-11T06:11:14.934847+0000","last_clean":"2023-10-11T06:11:14.934847+0000","last_became_active":"2023-10-11T06:11:13.694226+0000","last_became_peered":"2023-10-11T06:11:13.694226+0000","last_unstale":"2023-10-11T06:11:14.934847+0000","last_undegraded":"2023-10-11T06:11:14.934847+0000","last_fullsized":"2023-10-11T06:11:14.934847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697572+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:11:13.697572+0000","last_peered":"2023-10-11T06:11:13.697572+0000","last_clean":"2023-10-11T06:11:13.697572+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:11:13.697572+0000","last_undegraded":"2023-10-11T06:11:13.697572+0000","last_fullsized":"2023-10-11T06:11:13.697572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.686018+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:11:13.686018+0000","last_peered":"2023-10-11T06:11:13.686018+0000","last_clean":"2023-10-11T06:11:13.686018+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:11:13.686018+0000","last_undegraded":"2023-10-11T06:11:13.686018+0000","last_fullsized":"2023-10-11T06:11:13.686018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688223+0000","last_change":"2023-10-11T06:11:13.208661+0000","last_active":"2023-10-11T06:11:14.688223+0000","last_peered":"2023-10-11T06:11:14.688223+0000","last_clean":"2023-10-11T06:11:14.688223+0000","last_became_active":"2023-10-11T06:11:13.208190+0000","last_became_peered":"2023-10-11T06:11:13.208190+0000","last_unstale":"2023-10-11T06:11:14.688223+0000","last_undegraded":"2023-10-11T06:11:14.688223+0000","last_fullsized":"2023-10-11T06:11:14.688223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689821+0000","last_change":"2023-10-11T06:11:13.695542+0000","last_active":"2023-10-11T06:11:14.689821+0000","last_peered":"2023-10-11T06:11:14.689821+0000","last_clean":"2023-10-11T06:11:14.689821+0000","last_became_active":"2023-10-11T06:11:13.695257+0000","last_became_peered":"2023-10-11T06:11:13.695257+0000","last_unstale":"2023-10-11T06:11:14.689821+0000","last_undegraded":"2023-10-11T06:11:14.689821+0000","last_fullsized":"2023-10-11T06:11:14.689821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:04.854693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697461+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:11:13.697461+0000","last_peered":"2023-10-11T06:11:13.697461+0000","last_clean":"2023-10-11T06:11:13.697461+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:11:13.697461+0000","last_undegraded":"2023-10-11T06:11:13.697461+0000","last_fullsized":"2023-10-11T06:11:13.697461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932509+0000","last_change":"2023-10-11T06:11:13.215225+0000","last_active":"2023-10-11T06:11:14.932509+0000","last_peered":"2023-10-11T06:11:14.932509+0000","last_clean":"2023-10-11T06:11:14.932509+0000","last_became_active":"2023-10-11T06:11:13.214888+0000","last_became_peered":"2023-10-11T06:11:13.214888+0000","last_unstale":"2023-10-11T06:11:14.932509+0000","last_undegraded":"2023-10-11T06:11:14.932509+0000","last_fullsized":"2023-10-11T06:11:14.932509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933557+0000","last_change":"2023-10-11T06:11:13.704745+0000","last_active":"2023-10-11T06:11:14.933557+0000","last_peered":"2023-10-11T06:11:14.933557+0000","last_clean":"2023-10-11T06:11:14.933557+0000","last_became_active":"2023-10-11T06:11:13.704504+0000","last_became_peered":"2023-10-11T06:11:13.704504+0000","last_unstale":"2023-10-11T06:11:14.933557+0000","last_undegraded":"2023-10-11T06:11:14.933557+0000","last_fullsized":"2023-10-11T06:11:14.933557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.694273+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:11:13.694273+0000","last_peered":"2023-10-11T06:11:13.694273+0000","last_clean":"2023-10-11T06:11:13.694273+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:11:13.694273+0000","last_undegraded":"2023-10-11T06:11:13.694273+0000","last_fullsized":"2023-10-11T06:11:13.694273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934122+0000","last_change":"2023-10-11T06:11:13.212259+0000","last_active":"2023-10-11T06:11:14.934122+0000","last_peered":"2023-10-11T06:11:14.934122+0000","last_clean":"2023-10-11T06:11:14.934122+0000","last_became_active":"2023-10-11T06:11:13.211994+0000","last_became_peered":"2023-10-11T06:11:13.211994+0000","last_unstale":"2023-10-11T06:11:14.934122+0000","last_undegraded":"2023-10-11T06:11:14.934122+0000","last_fullsized":"2023-10-11T06:11:14.934122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934946+0000","last_change":"2023-10-11T06:11:13.700897+0000","last_active":"2023-10-11T06:11:14.934946+0000","last_peered":"2023-10-11T06:11:14.934946+0000","last_clean":"2023-10-11T06:11:14.934946+0000","last_became_active":"2023-10-11T06:11:13.700631+0000","last_became_peered":"2023-10-11T06:11:13.700631+0000","last_unstale":"2023-10-11T06:11:14.934946+0000","last_undegraded":"2023-10-11T06:11:14.934946+0000","last_fullsized":"2023-10-11T06:11:14.934946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.688833+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:11:13.688833+0000","last_peered":"2023-10-11T06:11:13.688833+0000","last_clean":"2023-10-11T06:11:13.688833+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:11:13.688833+0000","last_undegraded":"2023-10-11T06:11:13.688833+0000","last_fullsized":"2023-10-11T06:11:13.688833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936462+0000","last_change":"2023-10-11T06:11:13.218760+0000","last_active":"2023-10-11T06:11:14.936462+0000","last_peered":"2023-10-11T06:11:14.936462+0000","last_clean":"2023-10-11T06:11:14.936462+0000","last_became_active":"2023-10-11T06:11:13.218429+0000","last_became_peered":"2023-10-11T06:11:13.218429+0000","last_unstale":"2023-10-11T06:11:14.936462+0000","last_undegraded":"2023-10-11T06:11:14.936462+0000","last_fullsized":"2023-10-11T06:11:14.936462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934619+0000","last_change":"2023-10-11T06:11:13.703570+0000","last_active":"2023-10-11T06:11:14.934619+0000","last_peered":"2023-10-11T06:11:14.934619+0000","last_clean":"2023-10-11T06:11:14.934619+0000","last_became_active":"2023-10-11T06:11:13.703305+0000","last_became_peered":"2023-10-11T06:11:13.703305+0000","last_unstale":"2023-10-11T06:11:14.934619+0000","last_undegraded":"2023-10-11T06:11:14.934619+0000","last_fullsized":"2023-10-11T06:11:14.934619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936650+0000","last_change":"2023-10-11T06:11:13.208687+0000","last_active":"2023-10-11T06:11:14.936650+0000","last_peered":"2023-10-11T06:11:14.936650+0000","last_clean":"2023-10-11T06:11:14.936650+0000","last_became_active":"2023-10-11T06:11:13.207444+0000","last_became_peered":"2023-10-11T06:11:13.207444+0000","last_unstale":"2023-10-11T06:11:14.936650+0000","last_undegraded":"2023-10-11T06:11:14.936650+0000","last_fullsized":"2023-10-11T06:11:14.936650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.677406+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:11:13.677406+0000","last_peered":"2023-10-11T06:11:13.677406+0000","last_clean":"2023-10-11T06:11:13.677406+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:11:13.677406+0000","last_undegraded":"2023-10-11T06:11:13.677406+0000","last_fullsized":"2023-10-11T06:11:13.677406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934834+0000","last_change":"2023-10-11T06:11:13.700806+0000","last_active":"2023-10-11T06:11:14.934834+0000","last_peered":"2023-10-11T06:11:14.934834+0000","last_clean":"2023-10-11T06:11:14.934834+0000","last_became_active":"2023-10-11T06:11:13.700595+0000","last_became_peered":"2023-10-11T06:11:13.700595+0000","last_unstale":"2023-10-11T06:11:14.934834+0000","last_undegraded":"2023-10-11T06:11:14.934834+0000","last_fullsized":"2023-10-11T06:11:14.934834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934850+0000","last_change":"2023-10-11T06:11:13.207586+0000","last_active":"2023-10-11T06:11:14.934850+0000","last_peered":"2023-10-11T06:11:14.934850+0000","last_clean":"2023-10-11T06:11:14.934850+0000","last_became_active":"2023-10-11T06:11:13.207303+0000","last_became_peered":"2023-10-11T06:11:13.207303+0000","last_unstale":"2023-10-11T06:11:14.934850+0000","last_undegraded":"2023-10-11T06:11:14.934850+0000","last_fullsized":"2023-10-11T06:11:14.934850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.696997+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:11:13.696997+0000","last_peered":"2023-10-11T06:11:13.696997+0000","last_clean":"2023-10-11T06:11:13.696997+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:11:13.696997+0000","last_undegraded":"2023-10-11T06:11:13.696997+0000","last_fullsized":"2023-10-11T06:11:13.696997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932868+0000","last_change":"2023-10-11T06:11:13.700313+0000","last_active":"2023-10-11T06:11:14.932868+0000","last_peered":"2023-10-11T06:11:14.932868+0000","last_clean":"2023-10-11T06:11:14.932868+0000","last_became_active":"2023-10-11T06:11:13.700001+0000","last_became_peered":"2023-10-11T06:11:13.700001+0000","last_unstale":"2023-10-11T06:11:14.932868+0000","last_undegraded":"2023-10-11T06:11:14.932868+0000","last_fullsized":"2023-10-11T06:11:14.932868+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690266+0000","last_change":"2023-10-11T06:11:13.214171+0000","last_active":"2023-10-11T06:11:14.690266+0000","last_peered":"2023-10-11T06:11:14.690266+0000","last_clean":"2023-10-11T06:11:14.690266+0000","last_became_active":"2023-10-11T06:11:13.213902+0000","last_became_peered":"2023-10-11T06:11:13.213902+0000","last_unstale":"2023-10-11T06:11:14.690266+0000","last_undegraded":"2023-10-11T06:11:14.690266+0000","last_fullsized":"2023-10-11T06:11:14.690266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.678254+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:11:13.678254+0000","last_peered":"2023-10-11T06:11:13.678254+0000","last_clean":"2023-10-11T06:11:13.678254+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:11:13.678254+0000","last_undegraded":"2023-10-11T06:11:13.678254+0000","last_fullsized":"2023-10-11T06:11:13.678254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687734+0000","last_change":"2023-10-11T06:11:13.709945+0000","last_active":"2023-10-11T06:11:14.687734+0000","last_peered":"2023-10-11T06:11:14.687734+0000","last_clean":"2023-10-11T06:11:14.687734+0000","last_became_active":"2023-10-11T06:11:13.709607+0000","last_became_peered":"2023-10-11T06:11:13.709607+0000","last_unstale":"2023-10-11T06:11:14.687734+0000","last_undegraded":"2023-10-11T06:11:14.687734+0000","last_fullsized":"2023-10-11T06:11:14.687734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:15.133490+0000","last_change":"2023-10-11T06:11:13.213811+0000","last_active":"2023-10-11T06:11:15.133490+0000","last_peered":"2023-10-11T06:11:15.133490+0000","last_clean":"2023-10-11T06:11:15.133490+0000","last_became_active":"2023-10-11T06:11:13.213434+0000","last_became_peered":"2023-10-11T06:11:13.213434+0000","last_unstale":"2023-10-11T06:11:15.133490+0000","last_undegraded":"2023-10-11T06:11:15.133490+0000","last_fullsized":"2023-10-11T06:11:15.133490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.680148+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:11:13.680148+0000","last_peered":"2023-10-11T06:11:13.680148+0000","last_clean":"2023-10-11T06:11:13.680148+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:11:13.680148+0000","last_undegraded":"2023-10-11T06:11:13.680148+0000","last_fullsized":"2023-10-11T06:11:13.680148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936024+0000","last_change":"2023-10-11T06:11:13.700733+0000","last_active":"2023-10-11T06:11:14.936024+0000","last_peered":"2023-10-11T06:11:14.936024+0000","last_clean":"2023-10-11T06:11:14.936024+0000","last_became_active":"2023-10-11T06:11:13.700219+0000","last_became_peered":"2023-10-11T06:11:13.700219+0000","last_unstale":"2023-10-11T06:11:14.936024+0000","last_undegraded":"2023-10-11T06:11:14.936024+0000","last_fullsized":"2023-10-11T06:11:14.936024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936212+0000","last_change":"2023-10-11T06:11:13.691032+0000","last_active":"2023-10-11T06:11:14.936212+0000","last_peered":"2023-10-11T06:11:14.936212+0000","last_clean":"2023-10-11T06:11:14.936212+0000","last_became_active":"2023-10-11T06:11:13.690737+0000","last_became_peered":"2023-10-11T06:11:13.690737+0000","last_unstale":"2023-10-11T06:11:14.936212+0000","last_undegraded":"2023-10-11T06:11:14.936212+0000","last_fullsized":"2023-10-11T06:11:14.936212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.938254+0000","last_change":"2023-10-11T06:11:13.217477+0000","last_active":"2023-10-11T06:11:14.938254+0000","last_peered":"2023-10-11T06:11:14.938254+0000","last_clean":"2023-10-11T06:11:14.938254+0000","last_became_active":"2023-10-11T06:11:13.216485+0000","last_became_peered":"2023-10-11T06:11:13.216485+0000","last_unstale":"2023-10-11T06:11:14.938254+0000","last_undegraded":"2023-10-11T06:11:14.938254+0000","last_fullsized":"2023-10-11T06:11:14.938254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934689+0000","last_change":"2023-10-11T06:11:13.691615+0000","last_active":"2023-10-11T06:11:14.934689+0000","last_peered":"2023-10-11T06:11:14.934689+0000","last_clean":"2023-10-11T06:11:14.934689+0000","last_became_active":"2023-10-11T06:11:13.690979+0000","last_became_peered":"2023-10-11T06:11:13.690979+0000","last_unstale":"2023-10-11T06:11:14.934689+0000","last_undegraded":"2023-10-11T06:11:14.934689+0000","last_fullsized":"2023-10-11T06:11:14.934689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937582+0000","last_change":"2023-10-11T06:11:13.220058+0000","last_active":"2023-10-11T06:11:14.937582+0000","last_peered":"2023-10-11T06:11:14.937582+0000","last_clean":"2023-10-11T06:11:14.937582+0000","last_became_active":"2023-10-11T06:11:13.219554+0000","last_became_peered":"2023-10-11T06:11:13.219554+0000","last_unstale":"2023-10-11T06:11:14.937582+0000","last_undegraded":"2023-10-11T06:11:14.937582+0000","last_fullsized":"2023-10-11T06:11:14.937582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:48.717818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936220+0000","last_change":"2023-10-11T06:11:13.217619+0000","last_active":"2023-10-11T06:11:14.936220+0000","last_peered":"2023-10-11T06:11:14.936220+0000","last_clean":"2023-10-11T06:11:14.936220+0000","last_became_active":"2023-10-11T06:11:13.217261+0000","last_became_peered":"2023-10-11T06:11:13.217261+0000","last_unstale":"2023-10-11T06:11:14.936220+0000","last_undegraded":"2023-10-11T06:11:14.936220+0000","last_fullsized":"2023-10-11T06:11:14.936220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:34.256594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688528+0000","last_change":"2023-10-11T06:11:13.709235+0000","last_active":"2023-10-11T06:11:14.688528+0000","last_peered":"2023-10-11T06:11:14.688528+0000","last_clean":"2023-10-11T06:11:14.688528+0000","last_became_active":"2023-10-11T06:11:13.708979+0000","last_became_peered":"2023-10-11T06:11:13.708979+0000","last_unstale":"2023-10-11T06:11:14.688528+0000","last_undegraded":"2023-10-11T06:11:14.688528+0000","last_fullsized":"2023-10-11T06:11:14.688528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:24:24.143432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933598+0000","last_change":"2023-10-11T06:11:13.213756+0000","last_active":"2023-10-11T06:11:14.933598+0000","last_peered":"2023-10-11T06:11:14.933598+0000","last_clean":"2023-10-11T06:11:14.933598+0000","last_became_active":"2023-10-11T06:11:13.213456+0000","last_became_peered":"2023-10-11T06:11:13.213456+0000","last_unstale":"2023-10-11T06:11:14.933598+0000","last_undegraded":"2023-10-11T06:11:14.933598+0000","last_fullsized":"2023-10-11T06:11:14.933598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688932+0000","last_change":"2023-10-11T06:11:13.698101+0000","last_active":"2023-10-11T06:11:14.688932+0000","last_peered":"2023-10-11T06:11:14.688932+0000","last_clean":"2023-10-11T06:11:14.688932+0000","last_became_active":"2023-10-11T06:11:13.696880+0000","last_became_peered":"2023-10-11T06:11:13.696880+0000","last_unstale":"2023-10-11T06:11:14.688932+0000","last_undegraded":"2023-10-11T06:11:14.688932+0000","last_fullsized":"2023-10-11T06:11:14.688932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690335+0000","last_change":"2023-10-11T06:11:13.701258+0000","last_active":"2023-10-11T06:11:14.690335+0000","last_peered":"2023-10-11T06:11:14.690335+0000","last_clean":"2023-10-11T06:11:14.690335+0000","last_became_active":"2023-10-11T06:11:13.700995+0000","last_became_peered":"2023-10-11T06:11:13.700995+0000","last_unstale":"2023-10-11T06:11:14.690335+0000","last_undegraded":"2023-10-11T06:11:14.690335+0000","last_fullsized":"2023-10-11T06:11:14.690335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:44.575547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687997+0000","last_change":"2023-10-11T06:11:13.210038+0000","last_active":"2023-10-11T06:11:14.687997+0000","last_peered":"2023-10-11T06:11:14.687997+0000","last_clean":"2023-10-11T06:11:14.687997+0000","last_became_active":"2023-10-11T06:11:13.209702+0000","last_became_peered":"2023-10-11T06:11:13.209702+0000","last_unstale":"2023-10-11T06:11:14.687997+0000","last_undegraded":"2023-10-11T06:11:14.687997+0000","last_fullsized":"2023-10-11T06:11:14.687997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933487+0000","last_change":"2023-10-11T06:11:13.707123+0000","last_active":"2023-10-11T06:11:14.933487+0000","last_peered":"2023-10-11T06:11:14.933487+0000","last_clean":"2023-10-11T06:11:14.933487+0000","last_became_active":"2023-10-11T06:11:13.706856+0000","last_became_peered":"2023-10-11T06:11:13.706856+0000","last_unstale":"2023-10-11T06:11:14.933487+0000","last_undegraded":"2023-10-11T06:11:14.933487+0000","last_fullsized":"2023-10-11T06:11:14.933487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934110+0000","last_change":"2023-10-11T06:11:13.206009+0000","last_active":"2023-10-11T06:11:14.934110+0000","last_peered":"2023-10-11T06:11:14.934110+0000","last_clean":"2023-10-11T06:11:14.934110+0000","last_became_active":"2023-10-11T06:11:13.205671+0000","last_became_peered":"2023-10-11T06:11:13.205671+0000","last_unstale":"2023-10-11T06:11:14.934110+0000","last_undegraded":"2023-10-11T06:11:14.934110+0000","last_fullsized":"2023-10-11T06:11:14.934110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688523+0000","last_change":"2023-10-11T06:11:13.697631+0000","last_active":"2023-10-11T06:11:14.688523+0000","last_peered":"2023-10-11T06:11:14.688523+0000","last_clean":"2023-10-11T06:11:14.688523+0000","last_became_active":"2023-10-11T06:11:13.696974+0000","last_became_peered":"2023-10-11T06:11:13.696974+0000","last_unstale":"2023-10-11T06:11:14.688523+0000","last_undegraded":"2023-10-11T06:11:14.688523+0000","last_fullsized":"2023-10-11T06:11:14.688523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936125+0000","last_change":"2023-10-11T06:11:13.221421+0000","last_active":"2023-10-11T06:11:14.936125+0000","last_peered":"2023-10-11T06:11:14.936125+0000","last_clean":"2023-10-11T06:11:14.936125+0000","last_became_active":"2023-10-11T06:11:13.221114+0000","last_became_peered":"2023-10-11T06:11:13.221114+0000","last_unstale":"2023-10-11T06:11:14.936125+0000","last_undegraded":"2023-10-11T06:11:14.936125+0000","last_fullsized":"2023-10-11T06:11:14.936125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688237+0000","last_change":"2023-10-11T06:11:13.699277+0000","last_active":"2023-10-11T06:11:14.688237+0000","last_peered":"2023-10-11T06:11:14.688237+0000","last_clean":"2023-10-11T06:11:14.688237+0000","last_became_active":"2023-10-11T06:11:13.699042+0000","last_became_peered":"2023-10-11T06:11:13.699042+0000","last_unstale":"2023-10-11T06:11:14.688237+0000","last_undegraded":"2023-10-11T06:11:14.688237+0000","last_fullsized":"2023-10-11T06:11:14.688237+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933247+0000","last_change":"2023-10-11T06:11:13.218429+0000","last_active":"2023-10-11T06:11:14.933247+0000","last_peered":"2023-10-11T06:11:14.933247+0000","last_clean":"2023-10-11T06:11:14.933247+0000","last_became_active":"2023-10-11T06:11:13.218150+0000","last_became_peered":"2023-10-11T06:11:13.218150+0000","last_unstale":"2023-10-11T06:11:14.933247+0000","last_undegraded":"2023-10-11T06:11:14.933247+0000","last_fullsized":"2023-10-11T06:11:14.933247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935952+0000","last_change":"2023-10-11T06:11:13.700494+0000","last_active":"2023-10-11T06:11:14.935952+0000","last_peered":"2023-10-11T06:11:14.935952+0000","last_clean":"2023-10-11T06:11:14.935952+0000","last_became_active":"2023-10-11T06:11:13.700182+0000","last_became_peered":"2023-10-11T06:11:13.700182+0000","last_unstale":"2023-10-11T06:11:14.935952+0000","last_undegraded":"2023-10-11T06:11:14.935952+0000","last_fullsized":"2023-10-11T06:11:14.935952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934742+0000","last_change":"2023-10-11T06:11:13.221465+0000","last_active":"2023-10-11T06:11:14.934742+0000","last_peered":"2023-10-11T06:11:14.934742+0000","last_clean":"2023-10-11T06:11:14.934742+0000","last_became_active":"2023-10-11T06:11:13.221160+0000","last_became_peered":"2023-10-11T06:11:13.221160+0000","last_unstale":"2023-10-11T06:11:14.934742+0000","last_undegraded":"2023-10-11T06:11:14.934742+0000","last_fullsized":"2023-10-11T06:11:14.934742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935425+0000","last_change":"2023-10-11T06:11:13.690241+0000","last_active":"2023-10-11T06:11:14.935425+0000","last_peered":"2023-10-11T06:11:14.935425+0000","last_clean":"2023-10-11T06:11:14.935425+0000","last_became_active":"2023-10-11T06:11:13.689766+0000","last_became_peered":"2023-10-11T06:11:13.689766+0000","last_unstale":"2023-10-11T06:11:14.935425+0000","last_undegraded":"2023-10-11T06:11:14.935425+0000","last_fullsized":"2023-10-11T06:11:14.935425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:46:46.784397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936532+0000","last_change":"2023-10-11T06:11:13.212684+0000","last_active":"2023-10-11T06:11:14.936532+0000","last_peered":"2023-10-11T06:11:14.936532+0000","last_clean":"2023-10-11T06:11:14.936532+0000","last_became_active":"2023-10-11T06:11:13.212366+0000","last_became_peered":"2023-10-11T06:11:13.212366+0000","last_unstale":"2023-10-11T06:11:14.936532+0000","last_undegraded":"2023-10-11T06:11:14.936532+0000","last_fullsized":"2023-10-11T06:11:14.936532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688632+0000","last_change":"2023-10-11T06:11:13.706859+0000","last_active":"2023-10-11T06:11:14.688632+0000","last_peered":"2023-10-11T06:11:14.688632+0000","last_clean":"2023-10-11T06:11:14.688632+0000","last_became_active":"2023-10-11T06:11:13.706584+0000","last_became_peered":"2023-10-11T06:11:13.706584+0000","last_unstale":"2023-10-11T06:11:14.688632+0000","last_undegraded":"2023-10-11T06:11:14.688632+0000","last_fullsized":"2023-10-11T06:11:14.688632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688832+0000","last_change":"2023-10-11T06:11:13.200955+0000","last_active":"2023-10-11T06:11:14.688832+0000","last_peered":"2023-10-11T06:11:14.688832+0000","last_clean":"2023-10-11T06:11:14.688832+0000","last_became_active":"2023-10-11T06:11:13.200617+0000","last_became_peered":"2023-10-11T06:11:13.200617+0000","last_unstale":"2023-10-11T06:11:14.688832+0000","last_undegraded":"2023-10-11T06:11:14.688832+0000","last_fullsized":"2023-10-11T06:11:14.688832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935118+0000","last_change":"2023-10-11T06:11:13.689555+0000","last_active":"2023-10-11T06:11:14.935118+0000","last_peered":"2023-10-11T06:11:14.935118+0000","last_clean":"2023-10-11T06:11:14.935118+0000","last_became_active":"2023-10-11T06:11:13.689252+0000","last_became_peered":"2023-10-11T06:11:13.689252+0000","last_unstale":"2023-10-11T06:11:14.935118+0000","last_undegraded":"2023-10-11T06:11:14.935118+0000","last_fullsized":"2023-10-11T06:11:14.935118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:49.157560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933504+0000","last_change":"2023-10-11T06:11:13.213633+0000","last_active":"2023-10-11T06:11:14.933504+0000","last_peered":"2023-10-11T06:11:14.933504+0000","last_clean":"2023-10-11T06:11:14.933504+0000","last_became_active":"2023-10-11T06:11:13.213082+0000","last_became_peered":"2023-10-11T06:11:13.213082+0000","last_unstale":"2023-10-11T06:11:14.933504+0000","last_undegraded":"2023-10-11T06:11:14.933504+0000","last_fullsized":"2023-10-11T06:11:14.933504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":686418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":94922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":95890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":94360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":851968,"data_stored":685220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:11:21.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:11:22.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.238+0000 7f51c1499640 1 Processor -- start 2023-10-11T06:11:22.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.238+0000 7f51c1499640 1 -- start start 2023-10-11T06:11:22.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51c1499640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f51bc160500 0x7f51bc0c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51c1499640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51bc0ccbe0 0x7f51bc0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51c1499640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc0cd6d0 con 0x7f51bc1608d0 2023-10-11T06:11:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51c1499640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0cd110 con 0x7f51bc0ccbe0 2023-10-11T06:11:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51c1499640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0cd3f0 con 0x7f51bc160500 2023-10-11T06:11:22.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51b3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51bc0ccbe0 0x7f51bc0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51b3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51bc0ccbe0 0x7f51bc0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54026/0 (socket says 172.21.15.84:54026) 2023-10-11T06:11:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51b3fff640 1 -- 172.21.15.84:0/478329715 learned_addr learned my addr 172.21.15.84:0/478329715 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51baffd640 1 --2- 172.21.15.84:0/478329715 >> v2:172.21.15.99:3300/0 conn(0x7f51bc160500 0x7f51bc0c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51b3fff640 1 -- 172.21.15.84:0/478329715 >> v2:172.21.15.99:3300/0 conn(0x7f51bc160500 msgr2=0x7f51bc0c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51b3fff640 1 --2- 172.21.15.84:0/478329715 >> v2:172.21.15.99:3300/0 conn(0x7f51bc160500 0x7f51bc0c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51b3fff640 1 -- 172.21.15.84:0/478329715 >> v1:172.21.15.84:6789/0 conn(0x7f51bc1608d0 legacy=0x7f51bc1642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51b3fff640 1 -- 172.21.15.84:0/478329715 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0cdbc0 con 0x7f51bc0ccbe0 2023-10-11T06:11:22.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.240+0000 7f51baffd640 1 --2- 172.21.15.84:0/478329715 >> v2:172.21.15.99:3300/0 conn(0x7f51bc160500 0x7f51bc0c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:22.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.241+0000 7f51b3fff640 1 --2- 172.21.15.84:0/478329715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc0ccbe0 0x7f51bc0cafe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0b0810 tx=0x7f51a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6f4730bd9c1bd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:22.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.242+0000 7f51ba7fc640 1 -- 172.21.15.84:0/478329715 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4031870 con 0x7f51bc0ccbe0 2023-10-11T06:11:22.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.242+0000 7f51ba7fc640 1 -- 172.21.15.84:0/478329715 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a40319d0 con 0x7f51bc0ccbe0 2023-10-11T06:11:22.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.242+0000 7f51ba7fc640 1 -- 172.21.15.84:0/478329715 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4031d60 con 0x7f51bc0ccbe0 2023-10-11T06:11:22.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.242+0000 7f51c1499640 1 -- 172.21.15.84:0/478329715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc0ccbe0 msgr2=0x7f51bc0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.242+0000 7f51c1499640 1 --2- 172.21.15.84:0/478329715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc0ccbe0 0x7f51bc0cafe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0b0810 tx=0x7f51a402b910 comp rx=0 tx=0).stop 2023-10-11T06:11:22.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.243+0000 7f51c1499640 1 -- 172.21.15.84:0/478329715 shutdown_connections 2023-10-11T06:11:22.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.243+0000 7f51c1499640 1 --2- 172.21.15.84:0/478329715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc0ccbe0 0x7f51bc0cafe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.243+0000 7f51c1499640 1 --2- 172.21.15.84:0/478329715 >> v2:172.21.15.99:3300/0 conn(0x7f51bc160500 0x7f51bc0c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.243+0000 7f51c1499640 1 -- 172.21.15.84:0/478329715 >> 172.21.15.84:0/478329715 conn(0x7f51bc069c70 msgr2=0x7f51bc06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:22.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.243+0000 7f51c1499640 1 -- 172.21.15.84:0/478329715 shutdown_connections 2023-10-11T06:11:22.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.243+0000 7f51c1499640 1 -- 172.21.15.84:0/478329715 wait complete. 2023-10-11T06:11:22.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.244+0000 7f51c1499640 1 Processor -- start 2023-10-11T06:11:22.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.244+0000 7f51c1499640 1 -- start start 2023-10-11T06:11:22.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.245+0000 7f51c1499640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51bc0ccbe0 0x7f51bc1f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.245+0000 7f51c1499640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc160500 0x7f51bc0c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.245+0000 7f51c1499640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51bc1608d0 0x7f51bc0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.245+0000 7f51c1499640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1623d0 con 0x7f51bc160500 2023-10-11T06:11:22.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51c1499640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1f5f50 con 0x7f51bc1608d0 2023-10-11T06:11:22.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51c1499640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1617c0 con 0x7f51bc0ccbe0 2023-10-11T06:11:22.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc160500 0x7f51bc0c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc160500 0x7f51bc0c60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54032/0 (socket says 172.21.15.84:54032) 2023-10-11T06:11:22.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 -- 172.21.15.84:0/3570037161 learned_addr learned my addr 172.21.15.84:0/3570037161 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:22.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51bb7fe640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51bc1608d0 0x7f51bc0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 -- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51bc0ccbe0 msgr2=0x7f51bc1f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51bc0ccbe0 0x7f51bc1f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 -- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51bc1608d0 msgr2=0x7f51bc0ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51bc1608d0 0x7f51bc0ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 -- 172.21.15.84:0/3570037161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40094d0 con 0x7f51bc160500 2023-10-11T06:11:22.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51bb7fe640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51bc1608d0 0x7f51bc0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:22.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.246+0000 7f51b3fff640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc160500 0x7f51bc0c60e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f51a402b950 tx=0x7f51a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:22.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.247+0000 7f51b37fe640 1 -- 172.21.15.84:0/3570037161 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4031860 con 0x7f51bc160500 2023-10-11T06:11:22.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.247+0000 7f51b37fe640 1 -- 172.21.15.84:0/3570037161 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a40443f0 con 0x7f51bc160500 2023-10-11T06:11:22.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.247+0000 7f51b37fe640 1 -- 172.21.15.84:0/3570037161 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a40447a0 con 0x7f51bc160500 2023-10-11T06:11:22.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.247+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc161aa0 con 0x7f51bc160500 2023-10-11T06:11:22.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.247+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc162060 con 0x7f51bc160500 2023-10-11T06:11:22.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.248+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc073ed0 con 0x7f51bc160500 2023-10-11T06:11:22.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.249+0000 7f51b37fe640 1 -- 172.21.15.84:0/3570037161 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f51a404d450 con 0x7f51bc160500 2023-10-11T06:11:22.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.250+0000 7f51b37fe640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f519c06a830 0x7f519c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.250+0000 7f51b37fe640 1 -- 172.21.15.84:0/3570037161 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f51a402d040 con 0x7f51bc160500 2023-10-11T06:11:22.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.250+0000 7f51baffd640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f519c06a830 0x7f519c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.251+0000 7f51baffd640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f519c06a830 0x7f519c06cce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00a8b0 tx=0x7f51ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:22.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.256+0000 7f51b37fe640 1 -- 172.21.15.84:0/3570037161 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f51a4076a40 con 0x7f51bc160500 2023-10-11T06:11:22.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.427+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51bc0cf3d0 con 0x7f519c06a830 2023-10-11T06:11:22.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.438+0000 7f51b37fe640 1 -- 172.21.15.84:0/3570037161 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286490 (secure 0 0 0) 0x7f51ac00a4a0 con 0x7f519c06a830 2023-10-11T06:11:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:22.444 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:11:22.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f519c06a830 msgr2=0x7f519c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f519c06a830 0x7f519c06cce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00a8b0 tx=0x7f51ac008040 comp rx=0 tx=0).stop 2023-10-11T06:11:22.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc160500 msgr2=0x7f51bc0c60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc160500 0x7f51bc0c60e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f51a402b950 tx=0x7f51a40046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 shutdown_connections 2023-10-11T06:11:22.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f519c06a830 0x7f519c06cce0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51bc1608d0 0x7f51bc0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51bc160500 0x7f51bc0c60e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 --2- 172.21.15.84:0/3570037161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51bc0ccbe0 0x7f51bc1f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 >> 172.21.15.84:0/3570037161 conn(0x7f51bc069c70 msgr2=0x7f51bc0cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:22.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 shutdown_connections 2023-10-11T06:11:22.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.444+0000 7f51c1499640 1 -- 172.21.15.84:0/3570037161 wait complete. 2023-10-11T06:11:22.462 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-11T06:11:20.773293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218824,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":3284992,"data_stored":1939620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.593672"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688857+0000","last_change":"2023-10-11T06:11:13.702417+0000","last_active":"2023-10-11T06:11:14.688857+0000","last_peered":"2023-10-11T06:11:14.688857+0000","last_clean":"2023-10-11T06:11:14.688857+0000","last_became_active":"2023-10-11T06:11:13.702128+0000","last_became_peered":"2023-10-11T06:11:13.702128+0000","last_unstale":"2023-10-11T06:11:14.688857+0000","last_undegraded":"2023-10-11T06:11:14.688857+0000","last_fullsized":"2023-10-11T06:11:14.688857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:26.874292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935985+0000","last_change":"2023-10-11T06:11:13.701166+0000","last_active":"2023-10-11T06:11:14.935985+0000","last_peered":"2023-10-11T06:11:14.935985+0000","last_clean":"2023-10-11T06:11:14.935985+0000","last_became_active":"2023-10-11T06:11:13.700706+0000","last_became_peered":"2023-10-11T06:11:13.700706+0000","last_unstale":"2023-10-11T06:11:14.935985+0000","last_undegraded":"2023-10-11T06:11:14.935985+0000","last_fullsized":"2023-10-11T06:11:14.935985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937001+0000","last_change":"2023-10-11T06:11:13.701254+0000","last_active":"2023-10-11T06:11:14.937001+0000","last_peered":"2023-10-11T06:11:14.937001+0000","last_clean":"2023-10-11T06:11:14.937001+0000","last_became_active":"2023-10-11T06:11:13.700986+0000","last_became_peered":"2023-10-11T06:11:13.700986+0000","last_unstale":"2023-10-11T06:11:14.937001+0000","last_undegraded":"2023-10-11T06:11:14.937001+0000","last_fullsized":"2023-10-11T06:11:14.937001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:08:07.089082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689491+0000","last_change":"2023-10-11T06:11:13.692093+0000","last_active":"2023-10-11T06:11:14.689491+0000","last_peered":"2023-10-11T06:11:14.689491+0000","last_clean":"2023-10-11T06:11:14.689491+0000","last_became_active":"2023-10-11T06:11:13.691749+0000","last_became_peered":"2023-10-11T06:11:13.691749+0000","last_unstale":"2023-10-11T06:11:14.689491+0000","last_undegraded":"2023-10-11T06:11:14.689491+0000","last_fullsized":"2023-10-11T06:11:14.689491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933130+0000","last_change":"2023-10-11T06:11:13.706178+0000","last_active":"2023-10-11T06:11:14.933130+0000","last_peered":"2023-10-11T06:11:14.933130+0000","last_clean":"2023-10-11T06:11:14.933130+0000","last_became_active":"2023-10-11T06:11:13.705917+0000","last_became_peered":"2023-10-11T06:11:13.705917+0000","last_unstale":"2023-10-11T06:11:14.933130+0000","last_undegraded":"2023-10-11T06:11:14.933130+0000","last_fullsized":"2023-10-11T06:11:14.933130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935149+0000","last_change":"2023-10-11T06:11:13.702491+0000","last_active":"2023-10-11T06:11:14.935149+0000","last_peered":"2023-10-11T06:11:14.935149+0000","last_clean":"2023-10-11T06:11:14.935149+0000","last_became_active":"2023-10-11T06:11:13.701716+0000","last_became_peered":"2023-10-11T06:11:13.701716+0000","last_unstale":"2023-10-11T06:11:14.935149+0000","last_undegraded":"2023-10-11T06:11:14.935149+0000","last_fullsized":"2023-10-11T06:11:14.935149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:16:18.652379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934429+0000","last_change":"2023-10-11T06:11:13.713375+0000","last_active":"2023-10-11T06:11:14.934429+0000","last_peered":"2023-10-11T06:11:14.934429+0000","last_clean":"2023-10-11T06:11:14.934429+0000","last_became_active":"2023-10-11T06:11:13.712800+0000","last_became_peered":"2023-10-11T06:11:13.712800+0000","last_unstale":"2023-10-11T06:11:14.934429+0000","last_undegraded":"2023-10-11T06:11:14.934429+0000","last_fullsized":"2023-10-11T06:11:14.934429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934085+0000","last_change":"2023-10-11T06:11:13.700021+0000","last_active":"2023-10-11T06:11:14.934085+0000","last_peered":"2023-10-11T06:11:14.934085+0000","last_clean":"2023-10-11T06:11:14.934085+0000","last_became_active":"2023-10-11T06:11:13.699710+0000","last_became_peered":"2023-10-11T06:11:13.699710+0000","last_unstale":"2023-10-11T06:11:14.934085+0000","last_undegraded":"2023-10-11T06:11:14.934085+0000","last_fullsized":"2023-10-11T06:11:14.934085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933410+0000","last_change":"2023-10-11T06:11:13.705628+0000","last_active":"2023-10-11T06:11:14.933410+0000","last_peered":"2023-10-11T06:11:14.933410+0000","last_clean":"2023-10-11T06:11:14.933410+0000","last_became_active":"2023-10-11T06:11:13.705205+0000","last_became_peered":"2023-10-11T06:11:13.705205+0000","last_unstale":"2023-10-11T06:11:14.933410+0000","last_undegraded":"2023-10-11T06:11:14.933410+0000","last_fullsized":"2023-10-11T06:11:14.933410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936060+0000","last_change":"2023-10-11T06:11:13.700979+0000","last_active":"2023-10-11T06:11:14.936060+0000","last_peered":"2023-10-11T06:11:14.936060+0000","last_clean":"2023-10-11T06:11:14.936060+0000","last_became_active":"2023-10-11T06:11:13.700344+0000","last_became_peered":"2023-10-11T06:11:13.700344+0000","last_unstale":"2023-10-11T06:11:14.936060+0000","last_undegraded":"2023-10-11T06:11:14.936060+0000","last_fullsized":"2023-10-11T06:11:14.936060+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937234+0000","last_change":"2023-10-11T06:11:13.707695+0000","last_active":"2023-10-11T06:11:14.937234+0000","last_peered":"2023-10-11T06:11:14.937234+0000","last_clean":"2023-10-11T06:11:14.937234+0000","last_became_active":"2023-10-11T06:11:13.707358+0000","last_became_peered":"2023-10-11T06:11:13.707358+0000","last_unstale":"2023-10-11T06:11:14.937234+0000","last_undegraded":"2023-10-11T06:11:14.937234+0000","last_fullsized":"2023-10-11T06:11:14.937234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:38:53.832203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688144+0000","last_change":"2023-10-11T06:11:13.705483+0000","last_active":"2023-10-11T06:11:14.688144+0000","last_peered":"2023-10-11T06:11:14.688144+0000","last_clean":"2023-10-11T06:11:14.688144+0000","last_became_active":"2023-10-11T06:11:13.705228+0000","last_became_peered":"2023-10-11T06:11:13.705228+0000","last_unstale":"2023-10-11T06:11:14.688144+0000","last_undegraded":"2023-10-11T06:11:14.688144+0000","last_fullsized":"2023-10-11T06:11:14.688144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690422+0000","last_change":"2023-10-11T06:11:13.701448+0000","last_active":"2023-10-11T06:11:14.690422+0000","last_peered":"2023-10-11T06:11:14.690422+0000","last_clean":"2023-10-11T06:11:14.690422+0000","last_became_active":"2023-10-11T06:11:13.701186+0000","last_became_peered":"2023-10-11T06:11:13.701186+0000","last_unstale":"2023-10-11T06:11:14.690422+0000","last_undegraded":"2023-10-11T06:11:14.690422+0000","last_fullsized":"2023-10-11T06:11:14.690422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:01.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935482+0000","last_change":"2023-10-11T06:11:13.703675+0000","last_active":"2023-10-11T06:11:14.935482+0000","last_peered":"2023-10-11T06:11:14.935482+0000","last_clean":"2023-10-11T06:11:14.935482+0000","last_became_active":"2023-10-11T06:11:13.703365+0000","last_became_peered":"2023-10-11T06:11:13.703365+0000","last_unstale":"2023-10-11T06:11:14.935482+0000","last_undegraded":"2023-10-11T06:11:14.935482+0000","last_fullsized":"2023-10-11T06:11:14.935482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934721+0000","last_change":"2023-10-11T06:11:13.713483+0000","last_active":"2023-10-11T06:11:14.934721+0000","last_peered":"2023-10-11T06:11:14.934721+0000","last_clean":"2023-10-11T06:11:14.934721+0000","last_became_active":"2023-10-11T06:11:13.712594+0000","last_became_peered":"2023-10-11T06:11:13.712594+0000","last_unstale":"2023-10-11T06:11:14.934721+0000","last_undegraded":"2023-10-11T06:11:14.934721+0000","last_fullsized":"2023-10-11T06:11:14.934721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935163+0000","last_change":"2023-10-11T06:11:13.704217+0000","last_active":"2023-10-11T06:11:14.935163+0000","last_peered":"2023-10-11T06:11:14.935163+0000","last_clean":"2023-10-11T06:11:14.935163+0000","last_became_active":"2023-10-11T06:11:13.702922+0000","last_became_peered":"2023-10-11T06:11:13.702922+0000","last_unstale":"2023-10-11T06:11:14.935163+0000","last_undegraded":"2023-10-11T06:11:14.935163+0000","last_fullsized":"2023-10-11T06:11:14.935163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:53:10.963337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933536+0000","last_change":"2023-10-11T06:11:13.702292+0000","last_active":"2023-10-11T06:11:14.933536+0000","last_peered":"2023-10-11T06:11:14.933536+0000","last_clean":"2023-10-11T06:11:14.933536+0000","last_became_active":"2023-10-11T06:11:13.702060+0000","last_became_peered":"2023-10-11T06:11:13.702060+0000","last_unstale":"2023-10-11T06:11:14.933536+0000","last_undegraded":"2023-10-11T06:11:14.933536+0000","last_fullsized":"2023-10-11T06:11:14.933536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936185+0000","last_change":"2023-10-11T06:11:13.694348+0000","last_active":"2023-10-11T06:11:14.936185+0000","last_peered":"2023-10-11T06:11:14.936185+0000","last_clean":"2023-10-11T06:11:14.936185+0000","last_became_active":"2023-10-11T06:11:13.694074+0000","last_became_peered":"2023-10-11T06:11:13.694074+0000","last_unstale":"2023-10-11T06:11:14.936185+0000","last_undegraded":"2023-10-11T06:11:14.936185+0000","last_fullsized":"2023-10-11T06:11:14.936185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689126+0000","last_change":"2023-10-11T06:11:13.693722+0000","last_active":"2023-10-11T06:11:14.689126+0000","last_peered":"2023-10-11T06:11:14.689126+0000","last_clean":"2023-10-11T06:11:14.689126+0000","last_became_active":"2023-10-11T06:11:13.693338+0000","last_became_peered":"2023-10-11T06:11:13.693338+0000","last_unstale":"2023-10-11T06:11:14.689126+0000","last_undegraded":"2023-10-11T06:11:14.689126+0000","last_fullsized":"2023-10-11T06:11:14.689126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:35.271220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936628+0000","last_change":"2023-10-11T06:11:13.692755+0000","last_active":"2023-10-11T06:11:14.936628+0000","last_peered":"2023-10-11T06:11:14.936628+0000","last_clean":"2023-10-11T06:11:14.936628+0000","last_became_active":"2023-10-11T06:11:13.692467+0000","last_became_peered":"2023-10-11T06:11:13.692467+0000","last_unstale":"2023-10-11T06:11:14.936628+0000","last_undegraded":"2023-10-11T06:11:14.936628+0000","last_fullsized":"2023-10-11T06:11:14.936628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:06:12.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934603+0000","last_change":"2023-10-11T06:11:13.712835+0000","last_active":"2023-10-11T06:11:14.934603+0000","last_peered":"2023-10-11T06:11:14.934603+0000","last_clean":"2023-10-11T06:11:14.934603+0000","last_became_active":"2023-10-11T06:11:13.712498+0000","last_became_peered":"2023-10-11T06:11:13.712498+0000","last_unstale":"2023-10-11T06:11:14.934603+0000","last_undegraded":"2023-10-11T06:11:14.934603+0000","last_fullsized":"2023-10-11T06:11:14.934603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932549+0000","last_change":"2023-10-11T06:11:13.700139+0000","last_active":"2023-10-11T06:11:14.932549+0000","last_peered":"2023-10-11T06:11:14.932549+0000","last_clean":"2023-10-11T06:11:14.932549+0000","last_became_active":"2023-10-11T06:11:13.699888+0000","last_became_peered":"2023-10-11T06:11:13.699888+0000","last_unstale":"2023-10-11T06:11:14.932549+0000","last_undegraded":"2023-10-11T06:11:14.932549+0000","last_fullsized":"2023-10-11T06:11:14.932549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687599+0000","last_change":"2023-10-11T06:11:13.694139+0000","last_active":"2023-10-11T06:11:14.687599+0000","last_peered":"2023-10-11T06:11:14.687599+0000","last_clean":"2023-10-11T06:11:14.687599+0000","last_became_active":"2023-10-11T06:11:13.693870+0000","last_became_peered":"2023-10-11T06:11:13.693870+0000","last_unstale":"2023-10-11T06:11:14.687599+0000","last_undegraded":"2023-10-11T06:11:14.687599+0000","last_fullsized":"2023-10-11T06:11:14.687599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:32:18.757622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934975+0000","last_change":"2023-10-11T06:11:13.713679+0000","last_active":"2023-10-11T06:11:14.934975+0000","last_peered":"2023-10-11T06:11:14.934975+0000","last_clean":"2023-10-11T06:11:14.934975+0000","last_became_active":"2023-10-11T06:11:13.712476+0000","last_became_peered":"2023-10-11T06:11:13.712476+0000","last_unstale":"2023-10-11T06:11:14.934975+0000","last_undegraded":"2023-10-11T06:11:14.934975+0000","last_fullsized":"2023-10-11T06:11:14.934975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934737+0000","last_change":"2023-10-11T06:11:13.693952+0000","last_active":"2023-10-11T06:11:14.934737+0000","last_peered":"2023-10-11T06:11:14.934737+0000","last_clean":"2023-10-11T06:11:14.934737+0000","last_became_active":"2023-10-11T06:11:13.693582+0000","last_became_peered":"2023-10-11T06:11:13.693582+0000","last_unstale":"2023-10-11T06:11:14.934737+0000","last_undegraded":"2023-10-11T06:11:14.934737+0000","last_fullsized":"2023-10-11T06:11:14.934737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934890+0000","last_change":"2023-10-11T06:11:13.694225+0000","last_active":"2023-10-11T06:11:14.934890+0000","last_peered":"2023-10-11T06:11:14.934890+0000","last_clean":"2023-10-11T06:11:14.934890+0000","last_became_active":"2023-10-11T06:11:13.693965+0000","last_became_peered":"2023-10-11T06:11:13.693965+0000","last_unstale":"2023-10-11T06:11:14.934890+0000","last_undegraded":"2023-10-11T06:11:14.934890+0000","last_fullsized":"2023-10-11T06:11:14.934890+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:07.237906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935270+0000","last_change":"2023-10-11T06:11:13.710202+0000","last_active":"2023-10-11T06:11:14.935270+0000","last_peered":"2023-10-11T06:11:14.935270+0000","last_clean":"2023-10-11T06:11:14.935270+0000","last_became_active":"2023-10-11T06:11:13.709963+0000","last_became_peered":"2023-10-11T06:11:13.709963+0000","last_unstale":"2023-10-11T06:11:14.935270+0000","last_undegraded":"2023-10-11T06:11:14.935270+0000","last_fullsized":"2023-10-11T06:11:14.935270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935088+0000","last_change":"2023-10-11T06:11:13.709433+0000","last_active":"2023-10-11T06:11:14.935088+0000","last_peered":"2023-10-11T06:11:14.935088+0000","last_clean":"2023-10-11T06:11:14.935088+0000","last_became_active":"2023-10-11T06:11:13.709162+0000","last_became_peered":"2023-10-11T06:11:13.709162+0000","last_unstale":"2023-10-11T06:11:14.935088+0000","last_undegraded":"2023-10-11T06:11:14.935088+0000","last_fullsized":"2023-10-11T06:11:14.935088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688453+0000","last_change":"2023-10-11T06:11:13.709986+0000","last_active":"2023-10-11T06:11:14.688453+0000","last_peered":"2023-10-11T06:11:14.688453+0000","last_clean":"2023-10-11T06:11:14.688453+0000","last_became_active":"2023-10-11T06:11:13.709545+0000","last_became_peered":"2023-10-11T06:11:13.709545+0000","last_unstale":"2023-10-11T06:11:14.688453+0000","last_undegraded":"2023-10-11T06:11:14.688453+0000","last_fullsized":"2023-10-11T06:11:14.688453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935870+0000","last_change":"2023-10-11T06:11:13.690029+0000","last_active":"2023-10-11T06:11:14.935870+0000","last_peered":"2023-10-11T06:11:14.935870+0000","last_clean":"2023-10-11T06:11:14.935870+0000","last_became_active":"2023-10-11T06:11:13.689452+0000","last_became_peered":"2023-10-11T06:11:13.689452+0000","last_unstale":"2023-10-11T06:11:14.935870+0000","last_undegraded":"2023-10-11T06:11:14.935870+0000","last_fullsized":"2023-10-11T06:11:14.935870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:44:24.814427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687703+0000","last_change":"2023-10-11T06:11:13.697848+0000","last_active":"2023-10-11T06:11:14.687703+0000","last_peered":"2023-10-11T06:11:14.687703+0000","last_clean":"2023-10-11T06:11:14.687703+0000","last_became_active":"2023-10-11T06:11:13.697611+0000","last_became_peered":"2023-10-11T06:11:13.697611+0000","last_unstale":"2023-10-11T06:11:14.687703+0000","last_undegraded":"2023-10-11T06:11:14.687703+0000","last_fullsized":"2023-10-11T06:11:14.687703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:35:33.019995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689328+0000","last_change":"2023-10-11T06:11:13.696436+0000","last_active":"2023-10-11T06:11:14.689328+0000","last_peered":"2023-10-11T06:11:14.689328+0000","last_clean":"2023-10-11T06:11:14.689328+0000","last_became_active":"2023-10-11T06:11:13.696110+0000","last_became_peered":"2023-10-11T06:11:13.696110+0000","last_unstale":"2023-10-11T06:11:14.689328+0000","last_undegraded":"2023-10-11T06:11:14.689328+0000","last_fullsized":"2023-10-11T06:11:14.689328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:58.528525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689195+0000","last_change":"2023-10-11T06:11:13.697298+0000","last_active":"2023-10-11T06:11:14.689195+0000","last_peered":"2023-10-11T06:11:14.689195+0000","last_clean":"2023-10-11T06:11:14.689195+0000","last_became_active":"2023-10-11T06:11:13.697002+0000","last_became_peered":"2023-10-11T06:11:13.697002+0000","last_unstale":"2023-10-11T06:11:14.689195+0000","last_undegraded":"2023-10-11T06:11:14.689195+0000","last_fullsized":"2023-10-11T06:11:14.689195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937247+0000","last_change":"2023-10-11T06:11:13.221800+0000","last_active":"2023-10-11T06:11:14.937247+0000","last_peered":"2023-10-11T06:11:14.937247+0000","last_clean":"2023-10-11T06:11:14.937247+0000","last_became_active":"2023-10-11T06:11:13.221375+0000","last_became_peered":"2023-10-11T06:11:13.221375+0000","last_unstale":"2023-10-11T06:11:14.937247+0000","last_undegraded":"2023-10-11T06:11:14.937247+0000","last_fullsized":"2023-10-11T06:11:14.937247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:59:57.513899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935532+0000","last_change":"2023-10-11T06:11:13.706371+0000","last_active":"2023-10-11T06:11:14.935532+0000","last_peered":"2023-10-11T06:11:14.935532+0000","last_clean":"2023-10-11T06:11:14.935532+0000","last_became_active":"2023-10-11T06:11:13.706038+0000","last_became_peered":"2023-10-11T06:11:13.706038+0000","last_unstale":"2023-10-11T06:11:14.935532+0000","last_undegraded":"2023-10-11T06:11:14.935532+0000","last_fullsized":"2023-10-11T06:11:14.935532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932620+0000","last_change":"2023-10-11T06:11:13.217589+0000","last_active":"2023-10-11T06:11:14.932620+0000","last_peered":"2023-10-11T06:11:14.932620+0000","last_clean":"2023-10-11T06:11:14.932620+0000","last_became_active":"2023-10-11T06:11:13.217168+0000","last_became_peered":"2023-10-11T06:11:13.217168+0000","last_unstale":"2023-10-11T06:11:14.932620+0000","last_undegraded":"2023-10-11T06:11:14.932620+0000","last_fullsized":"2023-10-11T06:11:14.932620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933735+0000","last_change":"2023-10-11T06:11:13.703040+0000","last_active":"2023-10-11T06:11:14.933735+0000","last_peered":"2023-10-11T06:11:14.933735+0000","last_clean":"2023-10-11T06:11:14.933735+0000","last_became_active":"2023-10-11T06:11:13.702687+0000","last_became_peered":"2023-10-11T06:11:13.702687+0000","last_unstale":"2023-10-11T06:11:14.933735+0000","last_undegraded":"2023-10-11T06:11:14.933735+0000","last_fullsized":"2023-10-11T06:11:14.933735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934835+0000","last_change":"2023-10-11T06:11:13.221450+0000","last_active":"2023-10-11T06:11:14.934835+0000","last_peered":"2023-10-11T06:11:14.934835+0000","last_clean":"2023-10-11T06:11:14.934835+0000","last_became_active":"2023-10-11T06:11:13.221110+0000","last_became_peered":"2023-10-11T06:11:13.221110+0000","last_unstale":"2023-10-11T06:11:14.934835+0000","last_undegraded":"2023-10-11T06:11:14.934835+0000","last_fullsized":"2023-10-11T06:11:14.934835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:48.142934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690013+0000","last_change":"2023-10-11T06:11:13.710650+0000","last_active":"2023-10-11T06:11:14.690013+0000","last_peered":"2023-10-11T06:11:14.690013+0000","last_clean":"2023-10-11T06:11:14.690013+0000","last_became_active":"2023-10-11T06:11:13.710414+0000","last_became_peered":"2023-10-11T06:11:13.710414+0000","last_unstale":"2023-10-11T06:11:14.690013+0000","last_undegraded":"2023-10-11T06:11:14.690013+0000","last_fullsized":"2023-10-11T06:11:14.690013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:20:50.845484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689695+0000","last_change":"2023-10-11T06:11:13.209784+0000","last_active":"2023-10-11T06:11:14.689695+0000","last_peered":"2023-10-11T06:11:14.689695+0000","last_clean":"2023-10-11T06:11:14.689695+0000","last_became_active":"2023-10-11T06:11:13.209455+0000","last_became_peered":"2023-10-11T06:11:13.209455+0000","last_unstale":"2023-10-11T06:11:14.689695+0000","last_undegraded":"2023-10-11T06:11:14.689695+0000","last_fullsized":"2023-10-11T06:11:14.689695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937957+0000","last_change":"2023-10-11T06:11:13.221554+0000","last_active":"2023-10-11T06:11:14.937957+0000","last_peered":"2023-10-11T06:11:14.937957+0000","last_clean":"2023-10-11T06:11:14.937957+0000","last_became_active":"2023-10-11T06:11:13.221257+0000","last_became_peered":"2023-10-11T06:11:13.221257+0000","last_unstale":"2023-10-11T06:11:14.937957+0000","last_undegraded":"2023-10-11T06:11:14.937957+0000","last_fullsized":"2023-10-11T06:11:14.937957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:43.560921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934456+0000","last_change":"2023-10-11T06:11:13.706727+0000","last_active":"2023-10-11T06:11:14.934456+0000","last_peered":"2023-10-11T06:11:14.934456+0000","last_clean":"2023-10-11T06:11:14.934456+0000","last_became_active":"2023-10-11T06:11:13.706016+0000","last_became_peered":"2023-10-11T06:11:13.706016+0000","last_unstale":"2023-10-11T06:11:14.934456+0000","last_undegraded":"2023-10-11T06:11:14.934456+0000","last_fullsized":"2023-10-11T06:11:14.934456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937828+0000","last_change":"2023-10-11T06:11:13.222531+0000","last_active":"2023-10-11T06:11:14.937828+0000","last_peered":"2023-10-11T06:11:14.937828+0000","last_clean":"2023-10-11T06:11:14.937828+0000","last_became_active":"2023-10-11T06:11:13.222126+0000","last_became_peered":"2023-10-11T06:11:13.222126+0000","last_unstale":"2023-10-11T06:11:14.937828+0000","last_undegraded":"2023-10-11T06:11:14.937828+0000","last_fullsized":"2023-10-11T06:11:14.937828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:03.840067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935577+0000","last_change":"2023-10-11T06:11:13.700622+0000","last_active":"2023-10-11T06:11:14.935577+0000","last_peered":"2023-10-11T06:11:14.935577+0000","last_clean":"2023-10-11T06:11:14.935577+0000","last_became_active":"2023-10-11T06:11:13.699582+0000","last_became_peered":"2023-10-11T06:11:13.699582+0000","last_unstale":"2023-10-11T06:11:14.935577+0000","last_undegraded":"2023-10-11T06:11:14.935577+0000","last_fullsized":"2023-10-11T06:11:14.935577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937834+0000","last_change":"2023-10-11T06:11:13.220125+0000","last_active":"2023-10-11T06:11:14.937834+0000","last_peered":"2023-10-11T06:11:14.937834+0000","last_clean":"2023-10-11T06:11:14.937834+0000","last_became_active":"2023-10-11T06:11:13.219575+0000","last_became_peered":"2023-10-11T06:11:13.219575+0000","last_unstale":"2023-10-11T06:11:14.937834+0000","last_undegraded":"2023-10-11T06:11:14.937834+0000","last_fullsized":"2023-10-11T06:11:14.937834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:53:06.223280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689340+0000","last_change":"2023-10-11T06:11:13.697891+0000","last_active":"2023-10-11T06:11:14.689340+0000","last_peered":"2023-10-11T06:11:14.689340+0000","last_clean":"2023-10-11T06:11:14.689340+0000","last_became_active":"2023-10-11T06:11:13.697630+0000","last_became_peered":"2023-10-11T06:11:13.697630+0000","last_unstale":"2023-10-11T06:11:14.689340+0000","last_undegraded":"2023-10-11T06:11:14.689340+0000","last_fullsized":"2023-10-11T06:11:14.689340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935004+0000","last_change":"2023-10-11T06:11:13.207215+0000","last_active":"2023-10-11T06:11:14.935004+0000","last_peered":"2023-10-11T06:11:14.935004+0000","last_clean":"2023-10-11T06:11:14.935004+0000","last_became_active":"2023-10-11T06:11:13.206865+0000","last_became_peered":"2023-10-11T06:11:13.206865+0000","last_unstale":"2023-10-11T06:11:14.935004+0000","last_undegraded":"2023-10-11T06:11:14.935004+0000","last_fullsized":"2023-10-11T06:11:14.935004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932758+0000","last_change":"2023-10-11T06:11:13.699849+0000","last_active":"2023-10-11T06:11:14.932758+0000","last_peered":"2023-10-11T06:11:14.932758+0000","last_clean":"2023-10-11T06:11:14.932758+0000","last_became_active":"2023-10-11T06:11:13.699499+0000","last_became_peered":"2023-10-11T06:11:13.699499+0000","last_unstale":"2023-10-11T06:11:14.932758+0000","last_undegraded":"2023-10-11T06:11:14.932758+0000","last_fullsized":"2023-10-11T06:11:14.932758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688855+0000","last_change":"2023-10-11T06:11:13.209002+0000","last_active":"2023-10-11T06:11:14.688855+0000","last_peered":"2023-10-11T06:11:14.688855+0000","last_clean":"2023-10-11T06:11:14.688855+0000","last_became_active":"2023-10-11T06:11:13.208587+0000","last_became_peered":"2023-10-11T06:11:13.208587+0000","last_unstale":"2023-10-11T06:11:14.688855+0000","last_undegraded":"2023-10-11T06:11:14.688855+0000","last_fullsized":"2023-10-11T06:11:14.688855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937054+0000","last_change":"2023-10-11T06:11:13.698914+0000","last_active":"2023-10-11T06:11:14.937054+0000","last_peered":"2023-10-11T06:11:14.937054+0000","last_clean":"2023-10-11T06:11:14.937054+0000","last_became_active":"2023-10-11T06:11:13.697742+0000","last_became_peered":"2023-10-11T06:11:13.697742+0000","last_unstale":"2023-10-11T06:11:14.937054+0000","last_undegraded":"2023-10-11T06:11:14.937054+0000","last_fullsized":"2023-10-11T06:11:14.937054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:03:27.516427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689617+0000","last_change":"2023-10-11T06:11:13.197306+0000","last_active":"2023-10-11T06:11:14.689617+0000","last_peered":"2023-10-11T06:11:14.689617+0000","last_clean":"2023-10-11T06:11:14.689617+0000","last_became_active":"2023-10-11T06:11:13.196154+0000","last_became_peered":"2023-10-11T06:11:13.196154+0000","last_unstale":"2023-10-11T06:11:14.689617+0000","last_undegraded":"2023-10-11T06:11:14.689617+0000","last_fullsized":"2023-10-11T06:11:14.689617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937482+0000","last_change":"2023-10-11T06:11:13.694445+0000","last_active":"2023-10-11T06:11:14.937482+0000","last_peered":"2023-10-11T06:11:14.937482+0000","last_clean":"2023-10-11T06:11:14.937482+0000","last_became_active":"2023-10-11T06:11:13.694063+0000","last_became_peered":"2023-10-11T06:11:13.694063+0000","last_unstale":"2023-10-11T06:11:14.937482+0000","last_undegraded":"2023-10-11T06:11:14.937482+0000","last_fullsized":"2023-10-11T06:11:14.937482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936267+0000","last_change":"2023-10-11T06:11:13.212919+0000","last_active":"2023-10-11T06:11:14.936267+0000","last_peered":"2023-10-11T06:11:14.936267+0000","last_clean":"2023-10-11T06:11:14.936267+0000","last_became_active":"2023-10-11T06:11:13.212064+0000","last_became_peered":"2023-10-11T06:11:13.212064+0000","last_unstale":"2023-10-11T06:11:14.936267+0000","last_undegraded":"2023-10-11T06:11:14.936267+0000","last_fullsized":"2023-10-11T06:11:14.936267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936642+0000","last_change":"2023-10-11T06:11:13.692017+0000","last_active":"2023-10-11T06:11:14.936642+0000","last_peered":"2023-10-11T06:11:14.936642+0000","last_clean":"2023-10-11T06:11:14.936642+0000","last_became_active":"2023-10-11T06:11:13.690806+0000","last_became_peered":"2023-10-11T06:11:13.690806+0000","last_unstale":"2023-10-11T06:11:14.936642+0000","last_undegraded":"2023-10-11T06:11:14.936642+0000","last_fullsized":"2023-10-11T06:11:14.936642+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688241+0000","last_change":"2023-10-11T06:11:13.213340+0000","last_active":"2023-10-11T06:11:14.688241+0000","last_peered":"2023-10-11T06:11:14.688241+0000","last_clean":"2023-10-11T06:11:14.688241+0000","last_became_active":"2023-10-11T06:11:13.213031+0000","last_became_peered":"2023-10-11T06:11:13.213031+0000","last_unstale":"2023-10-11T06:11:14.688241+0000","last_undegraded":"2023-10-11T06:11:14.688241+0000","last_fullsized":"2023-10-11T06:11:14.688241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934847+0000","last_change":"2023-10-11T06:11:13.694560+0000","last_active":"2023-10-11T06:11:14.934847+0000","last_peered":"2023-10-11T06:11:14.934847+0000","last_clean":"2023-10-11T06:11:14.934847+0000","last_became_active":"2023-10-11T06:11:13.694226+0000","last_became_peered":"2023-10-11T06:11:13.694226+0000","last_unstale":"2023-10-11T06:11:14.934847+0000","last_undegraded":"2023-10-11T06:11:14.934847+0000","last_fullsized":"2023-10-11T06:11:14.934847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:57:03.301130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697572+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:11:13.697572+0000","last_peered":"2023-10-11T06:11:13.697572+0000","last_clean":"2023-10-11T06:11:13.697572+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:11:13.697572+0000","last_undegraded":"2023-10-11T06:11:13.697572+0000","last_fullsized":"2023-10-11T06:11:13.697572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.686018+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:11:13.686018+0000","last_peered":"2023-10-11T06:11:13.686018+0000","last_clean":"2023-10-11T06:11:13.686018+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:11:13.686018+0000","last_undegraded":"2023-10-11T06:11:13.686018+0000","last_fullsized":"2023-10-11T06:11:13.686018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688223+0000","last_change":"2023-10-11T06:11:13.208661+0000","last_active":"2023-10-11T06:11:14.688223+0000","last_peered":"2023-10-11T06:11:14.688223+0000","last_clean":"2023-10-11T06:11:14.688223+0000","last_became_active":"2023-10-11T06:11:13.208190+0000","last_became_peered":"2023-10-11T06:11:13.208190+0000","last_unstale":"2023-10-11T06:11:14.688223+0000","last_undegraded":"2023-10-11T06:11:14.688223+0000","last_fullsized":"2023-10-11T06:11:14.688223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.689821+0000","last_change":"2023-10-11T06:11:13.695542+0000","last_active":"2023-10-11T06:11:14.689821+0000","last_peered":"2023-10-11T06:11:14.689821+0000","last_clean":"2023-10-11T06:11:14.689821+0000","last_became_active":"2023-10-11T06:11:13.695257+0000","last_became_peered":"2023-10-11T06:11:13.695257+0000","last_unstale":"2023-10-11T06:11:14.689821+0000","last_undegraded":"2023-10-11T06:11:14.689821+0000","last_fullsized":"2023-10-11T06:11:14.689821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:41:04.854693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.697461+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:11:13.697461+0000","last_peered":"2023-10-11T06:11:13.697461+0000","last_clean":"2023-10-11T06:11:13.697461+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:11:13.697461+0000","last_undegraded":"2023-10-11T06:11:13.697461+0000","last_fullsized":"2023-10-11T06:11:13.697461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932509+0000","last_change":"2023-10-11T06:11:13.215225+0000","last_active":"2023-10-11T06:11:14.932509+0000","last_peered":"2023-10-11T06:11:14.932509+0000","last_clean":"2023-10-11T06:11:14.932509+0000","last_became_active":"2023-10-11T06:11:13.214888+0000","last_became_peered":"2023-10-11T06:11:13.214888+0000","last_unstale":"2023-10-11T06:11:14.932509+0000","last_undegraded":"2023-10-11T06:11:14.932509+0000","last_fullsized":"2023-10-11T06:11:14.932509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933557+0000","last_change":"2023-10-11T06:11:13.704745+0000","last_active":"2023-10-11T06:11:14.933557+0000","last_peered":"2023-10-11T06:11:14.933557+0000","last_clean":"2023-10-11T06:11:14.933557+0000","last_became_active":"2023-10-11T06:11:13.704504+0000","last_became_peered":"2023-10-11T06:11:13.704504+0000","last_unstale":"2023-10-11T06:11:14.933557+0000","last_undegraded":"2023-10-11T06:11:14.933557+0000","last_fullsized":"2023-10-11T06:11:14.933557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.694273+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:11:13.694273+0000","last_peered":"2023-10-11T06:11:13.694273+0000","last_clean":"2023-10-11T06:11:13.694273+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:11:13.694273+0000","last_undegraded":"2023-10-11T06:11:13.694273+0000","last_fullsized":"2023-10-11T06:11:13.694273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934122+0000","last_change":"2023-10-11T06:11:13.212259+0000","last_active":"2023-10-11T06:11:14.934122+0000","last_peered":"2023-10-11T06:11:14.934122+0000","last_clean":"2023-10-11T06:11:14.934122+0000","last_became_active":"2023-10-11T06:11:13.211994+0000","last_became_peered":"2023-10-11T06:11:13.211994+0000","last_unstale":"2023-10-11T06:11:14.934122+0000","last_undegraded":"2023-10-11T06:11:14.934122+0000","last_fullsized":"2023-10-11T06:11:14.934122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934946+0000","last_change":"2023-10-11T06:11:13.700897+0000","last_active":"2023-10-11T06:11:14.934946+0000","last_peered":"2023-10-11T06:11:14.934946+0000","last_clean":"2023-10-11T06:11:14.934946+0000","last_became_active":"2023-10-11T06:11:13.700631+0000","last_became_peered":"2023-10-11T06:11:13.700631+0000","last_unstale":"2023-10-11T06:11:14.934946+0000","last_undegraded":"2023-10-11T06:11:14.934946+0000","last_fullsized":"2023-10-11T06:11:14.934946+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.688833+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:11:13.688833+0000","last_peered":"2023-10-11T06:11:13.688833+0000","last_clean":"2023-10-11T06:11:13.688833+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:11:13.688833+0000","last_undegraded":"2023-10-11T06:11:13.688833+0000","last_fullsized":"2023-10-11T06:11:13.688833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936462+0000","last_change":"2023-10-11T06:11:13.218760+0000","last_active":"2023-10-11T06:11:14.936462+0000","last_peered":"2023-10-11T06:11:14.936462+0000","last_clean":"2023-10-11T06:11:14.936462+0000","last_became_active":"2023-10-11T06:11:13.218429+0000","last_became_peered":"2023-10-11T06:11:13.218429+0000","last_unstale":"2023-10-11T06:11:14.936462+0000","last_undegraded":"2023-10-11T06:11:14.936462+0000","last_fullsized":"2023-10-11T06:11:14.936462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934619+0000","last_change":"2023-10-11T06:11:13.703570+0000","last_active":"2023-10-11T06:11:14.934619+0000","last_peered":"2023-10-11T06:11:14.934619+0000","last_clean":"2023-10-11T06:11:14.934619+0000","last_became_active":"2023-10-11T06:11:13.703305+0000","last_became_peered":"2023-10-11T06:11:13.703305+0000","last_unstale":"2023-10-11T06:11:14.934619+0000","last_undegraded":"2023-10-11T06:11:14.934619+0000","last_fullsized":"2023-10-11T06:11:14.934619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936650+0000","last_change":"2023-10-11T06:11:13.208687+0000","last_active":"2023-10-11T06:11:14.936650+0000","last_peered":"2023-10-11T06:11:14.936650+0000","last_clean":"2023-10-11T06:11:14.936650+0000","last_became_active":"2023-10-11T06:11:13.207444+0000","last_became_peered":"2023-10-11T06:11:13.207444+0000","last_unstale":"2023-10-11T06:11:14.936650+0000","last_undegraded":"2023-10-11T06:11:14.936650+0000","last_fullsized":"2023-10-11T06:11:14.936650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.677406+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:11:13.677406+0000","last_peered":"2023-10-11T06:11:13.677406+0000","last_clean":"2023-10-11T06:11:13.677406+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:11:13.677406+0000","last_undegraded":"2023-10-11T06:11:13.677406+0000","last_fullsized":"2023-10-11T06:11:13.677406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934834+0000","last_change":"2023-10-11T06:11:13.700806+0000","last_active":"2023-10-11T06:11:14.934834+0000","last_peered":"2023-10-11T06:11:14.934834+0000","last_clean":"2023-10-11T06:11:14.934834+0000","last_became_active":"2023-10-11T06:11:13.700595+0000","last_became_peered":"2023-10-11T06:11:13.700595+0000","last_unstale":"2023-10-11T06:11:14.934834+0000","last_undegraded":"2023-10-11T06:11:14.934834+0000","last_fullsized":"2023-10-11T06:11:14.934834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934850+0000","last_change":"2023-10-11T06:11:13.207586+0000","last_active":"2023-10-11T06:11:14.934850+0000","last_peered":"2023-10-11T06:11:14.934850+0000","last_clean":"2023-10-11T06:11:14.934850+0000","last_became_active":"2023-10-11T06:11:13.207303+0000","last_became_peered":"2023-10-11T06:11:13.207303+0000","last_unstale":"2023-10-11T06:11:14.934850+0000","last_undegraded":"2023-10-11T06:11:14.934850+0000","last_fullsized":"2023-10-11T06:11:14.934850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.696997+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:11:13.696997+0000","last_peered":"2023-10-11T06:11:13.696997+0000","last_clean":"2023-10-11T06:11:13.696997+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:11:13.696997+0000","last_undegraded":"2023-10-11T06:11:13.696997+0000","last_fullsized":"2023-10-11T06:11:13.696997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.932868+0000","last_change":"2023-10-11T06:11:13.700313+0000","last_active":"2023-10-11T06:11:14.932868+0000","last_peered":"2023-10-11T06:11:14.932868+0000","last_clean":"2023-10-11T06:11:14.932868+0000","last_became_active":"2023-10-11T06:11:13.700001+0000","last_became_peered":"2023-10-11T06:11:13.700001+0000","last_unstale":"2023-10-11T06:11:14.932868+0000","last_undegraded":"2023-10-11T06:11:14.932868+0000","last_fullsized":"2023-10-11T06:11:14.932868+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:39:57.000582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690266+0000","last_change":"2023-10-11T06:11:13.214171+0000","last_active":"2023-10-11T06:11:14.690266+0000","last_peered":"2023-10-11T06:11:14.690266+0000","last_clean":"2023-10-11T06:11:14.690266+0000","last_became_active":"2023-10-11T06:11:13.213902+0000","last_became_peered":"2023-10-11T06:11:13.213902+0000","last_unstale":"2023-10-11T06:11:14.690266+0000","last_undegraded":"2023-10-11T06:11:14.690266+0000","last_fullsized":"2023-10-11T06:11:14.690266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:41.834221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.678254+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:11:13.678254+0000","last_peered":"2023-10-11T06:11:13.678254+0000","last_clean":"2023-10-11T06:11:13.678254+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:11:13.678254+0000","last_undegraded":"2023-10-11T06:11:13.678254+0000","last_fullsized":"2023-10-11T06:11:13.678254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687734+0000","last_change":"2023-10-11T06:11:13.709945+0000","last_active":"2023-10-11T06:11:14.687734+0000","last_peered":"2023-10-11T06:11:14.687734+0000","last_clean":"2023-10-11T06:11:14.687734+0000","last_became_active":"2023-10-11T06:11:13.709607+0000","last_became_peered":"2023-10-11T06:11:13.709607+0000","last_unstale":"2023-10-11T06:11:14.687734+0000","last_undegraded":"2023-10-11T06:11:14.687734+0000","last_fullsized":"2023-10-11T06:11:14.687734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:15.133490+0000","last_change":"2023-10-11T06:11:13.213811+0000","last_active":"2023-10-11T06:11:15.133490+0000","last_peered":"2023-10-11T06:11:15.133490+0000","last_clean":"2023-10-11T06:11:15.133490+0000","last_became_active":"2023-10-11T06:11:13.213434+0000","last_became_peered":"2023-10-11T06:11:13.213434+0000","last_unstale":"2023-10-11T06:11:15.133490+0000","last_undegraded":"2023-10-11T06:11:15.133490+0000","last_fullsized":"2023-10-11T06:11:15.133490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-11T06:11:13.680148+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:11:13.680148+0000","last_peered":"2023-10-11T06:11:13.680148+0000","last_clean":"2023-10-11T06:11:13.680148+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:11:13.680148+0000","last_undegraded":"2023-10-11T06:11:13.680148+0000","last_fullsized":"2023-10-11T06:11:13.680148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936024+0000","last_change":"2023-10-11T06:11:13.700733+0000","last_active":"2023-10-11T06:11:14.936024+0000","last_peered":"2023-10-11T06:11:14.936024+0000","last_clean":"2023-10-11T06:11:14.936024+0000","last_became_active":"2023-10-11T06:11:13.700219+0000","last_became_peered":"2023-10-11T06:11:13.700219+0000","last_unstale":"2023-10-11T06:11:14.936024+0000","last_undegraded":"2023-10-11T06:11:14.936024+0000","last_fullsized":"2023-10-11T06:11:14.936024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936212+0000","last_change":"2023-10-11T06:11:13.691032+0000","last_active":"2023-10-11T06:11:14.936212+0000","last_peered":"2023-10-11T06:11:14.936212+0000","last_clean":"2023-10-11T06:11:14.936212+0000","last_became_active":"2023-10-11T06:11:13.690737+0000","last_became_peered":"2023-10-11T06:11:13.690737+0000","last_unstale":"2023-10-11T06:11:14.936212+0000","last_undegraded":"2023-10-11T06:11:14.936212+0000","last_fullsized":"2023-10-11T06:11:14.936212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:03:40.641426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.938254+0000","last_change":"2023-10-11T06:11:13.217477+0000","last_active":"2023-10-11T06:11:14.938254+0000","last_peered":"2023-10-11T06:11:14.938254+0000","last_clean":"2023-10-11T06:11:14.938254+0000","last_became_active":"2023-10-11T06:11:13.216485+0000","last_became_peered":"2023-10-11T06:11:13.216485+0000","last_unstale":"2023-10-11T06:11:14.938254+0000","last_undegraded":"2023-10-11T06:11:14.938254+0000","last_fullsized":"2023-10-11T06:11:14.938254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:16:43.132190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934689+0000","last_change":"2023-10-11T06:11:13.691615+0000","last_active":"2023-10-11T06:11:14.934689+0000","last_peered":"2023-10-11T06:11:14.934689+0000","last_clean":"2023-10-11T06:11:14.934689+0000","last_became_active":"2023-10-11T06:11:13.690979+0000","last_became_peered":"2023-10-11T06:11:13.690979+0000","last_unstale":"2023-10-11T06:11:14.934689+0000","last_undegraded":"2023-10-11T06:11:14.934689+0000","last_fullsized":"2023-10-11T06:11:14.934689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:37:59.629623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.937582+0000","last_change":"2023-10-11T06:11:13.220058+0000","last_active":"2023-10-11T06:11:14.937582+0000","last_peered":"2023-10-11T06:11:14.937582+0000","last_clean":"2023-10-11T06:11:14.937582+0000","last_became_active":"2023-10-11T06:11:13.219554+0000","last_became_peered":"2023-10-11T06:11:13.219554+0000","last_unstale":"2023-10-11T06:11:14.937582+0000","last_undegraded":"2023-10-11T06:11:14.937582+0000","last_fullsized":"2023-10-11T06:11:14.937582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:48.717818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936220+0000","last_change":"2023-10-11T06:11:13.217619+0000","last_active":"2023-10-11T06:11:14.936220+0000","last_peered":"2023-10-11T06:11:14.936220+0000","last_clean":"2023-10-11T06:11:14.936220+0000","last_became_active":"2023-10-11T06:11:13.217261+0000","last_became_peered":"2023-10-11T06:11:13.217261+0000","last_unstale":"2023-10-11T06:11:14.936220+0000","last_undegraded":"2023-10-11T06:11:14.936220+0000","last_fullsized":"2023-10-11T06:11:14.936220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:16:34.256594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688528+0000","last_change":"2023-10-11T06:11:13.709235+0000","last_active":"2023-10-11T06:11:14.688528+0000","last_peered":"2023-10-11T06:11:14.688528+0000","last_clean":"2023-10-11T06:11:14.688528+0000","last_became_active":"2023-10-11T06:11:13.708979+0000","last_became_peered":"2023-10-11T06:11:13.708979+0000","last_unstale":"2023-10-11T06:11:14.688528+0000","last_undegraded":"2023-10-11T06:11:14.688528+0000","last_fullsized":"2023-10-11T06:11:14.688528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:24:24.143432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933598+0000","last_change":"2023-10-11T06:11:13.213756+0000","last_active":"2023-10-11T06:11:14.933598+0000","last_peered":"2023-10-11T06:11:14.933598+0000","last_clean":"2023-10-11T06:11:14.933598+0000","last_became_active":"2023-10-11T06:11:13.213456+0000","last_became_peered":"2023-10-11T06:11:13.213456+0000","last_unstale":"2023-10-11T06:11:14.933598+0000","last_undegraded":"2023-10-11T06:11:14.933598+0000","last_fullsized":"2023-10-11T06:11:14.933598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:30:37.873240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688932+0000","last_change":"2023-10-11T06:11:13.698101+0000","last_active":"2023-10-11T06:11:14.688932+0000","last_peered":"2023-10-11T06:11:14.688932+0000","last_clean":"2023-10-11T06:11:14.688932+0000","last_became_active":"2023-10-11T06:11:13.696880+0000","last_became_peered":"2023-10-11T06:11:13.696880+0000","last_unstale":"2023-10-11T06:11:14.688932+0000","last_undegraded":"2023-10-11T06:11:14.688932+0000","last_fullsized":"2023-10-11T06:11:14.688932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:20:13.241492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.690335+0000","last_change":"2023-10-11T06:11:13.701258+0000","last_active":"2023-10-11T06:11:14.690335+0000","last_peered":"2023-10-11T06:11:14.690335+0000","last_clean":"2023-10-11T06:11:14.690335+0000","last_became_active":"2023-10-11T06:11:13.700995+0000","last_became_peered":"2023-10-11T06:11:13.700995+0000","last_unstale":"2023-10-11T06:11:14.690335+0000","last_undegraded":"2023-10-11T06:11:14.690335+0000","last_fullsized":"2023-10-11T06:11:14.690335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:44.575547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.687997+0000","last_change":"2023-10-11T06:11:13.210038+0000","last_active":"2023-10-11T06:11:14.687997+0000","last_peered":"2023-10-11T06:11:14.687997+0000","last_clean":"2023-10-11T06:11:14.687997+0000","last_became_active":"2023-10-11T06:11:13.209702+0000","last_became_peered":"2023-10-11T06:11:13.209702+0000","last_unstale":"2023-10-11T06:11:14.687997+0000","last_undegraded":"2023-10-11T06:11:14.687997+0000","last_fullsized":"2023-10-11T06:11:14.687997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:56:25.859666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933487+0000","last_change":"2023-10-11T06:11:13.707123+0000","last_active":"2023-10-11T06:11:14.933487+0000","last_peered":"2023-10-11T06:11:14.933487+0000","last_clean":"2023-10-11T06:11:14.933487+0000","last_became_active":"2023-10-11T06:11:13.706856+0000","last_became_peered":"2023-10-11T06:11:13.706856+0000","last_unstale":"2023-10-11T06:11:14.933487+0000","last_undegraded":"2023-10-11T06:11:14.933487+0000","last_fullsized":"2023-10-11T06:11:14.933487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:59:53.455291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934110+0000","last_change":"2023-10-11T06:11:13.206009+0000","last_active":"2023-10-11T06:11:14.934110+0000","last_peered":"2023-10-11T06:11:14.934110+0000","last_clean":"2023-10-11T06:11:14.934110+0000","last_became_active":"2023-10-11T06:11:13.205671+0000","last_became_peered":"2023-10-11T06:11:13.205671+0000","last_unstale":"2023-10-11T06:11:14.934110+0000","last_undegraded":"2023-10-11T06:11:14.934110+0000","last_fullsized":"2023-10-11T06:11:14.934110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688523+0000","last_change":"2023-10-11T06:11:13.697631+0000","last_active":"2023-10-11T06:11:14.688523+0000","last_peered":"2023-10-11T06:11:14.688523+0000","last_clean":"2023-10-11T06:11:14.688523+0000","last_became_active":"2023-10-11T06:11:13.696974+0000","last_became_peered":"2023-10-11T06:11:13.696974+0000","last_unstale":"2023-10-11T06:11:14.688523+0000","last_undegraded":"2023-10-11T06:11:14.688523+0000","last_fullsized":"2023-10-11T06:11:14.688523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:50.013454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936125+0000","last_change":"2023-10-11T06:11:13.221421+0000","last_active":"2023-10-11T06:11:14.936125+0000","last_peered":"2023-10-11T06:11:14.936125+0000","last_clean":"2023-10-11T06:11:14.936125+0000","last_became_active":"2023-10-11T06:11:13.221114+0000","last_became_peered":"2023-10-11T06:11:13.221114+0000","last_unstale":"2023-10-11T06:11:14.936125+0000","last_undegraded":"2023-10-11T06:11:14.936125+0000","last_fullsized":"2023-10-11T06:11:14.936125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:33:34.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688237+0000","last_change":"2023-10-11T06:11:13.699277+0000","last_active":"2023-10-11T06:11:14.688237+0000","last_peered":"2023-10-11T06:11:14.688237+0000","last_clean":"2023-10-11T06:11:14.688237+0000","last_became_active":"2023-10-11T06:11:13.699042+0000","last_became_peered":"2023-10-11T06:11:13.699042+0000","last_unstale":"2023-10-11T06:11:14.688237+0000","last_undegraded":"2023-10-11T06:11:14.688237+0000","last_fullsized":"2023-10-11T06:11:14.688237+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:47:42.848847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933247+0000","last_change":"2023-10-11T06:11:13.218429+0000","last_active":"2023-10-11T06:11:14.933247+0000","last_peered":"2023-10-11T06:11:14.933247+0000","last_clean":"2023-10-11T06:11:14.933247+0000","last_became_active":"2023-10-11T06:11:13.218150+0000","last_became_peered":"2023-10-11T06:11:13.218150+0000","last_unstale":"2023-10-11T06:11:14.933247+0000","last_undegraded":"2023-10-11T06:11:14.933247+0000","last_fullsized":"2023-10-11T06:11:14.933247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:34:19.838988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935952+0000","last_change":"2023-10-11T06:11:13.700494+0000","last_active":"2023-10-11T06:11:14.935952+0000","last_peered":"2023-10-11T06:11:14.935952+0000","last_clean":"2023-10-11T06:11:14.935952+0000","last_became_active":"2023-10-11T06:11:13.700182+0000","last_became_peered":"2023-10-11T06:11:13.700182+0000","last_unstale":"2023-10-11T06:11:14.935952+0000","last_undegraded":"2023-10-11T06:11:14.935952+0000","last_fullsized":"2023-10-11T06:11:14.935952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:51:29.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.934742+0000","last_change":"2023-10-11T06:11:13.221465+0000","last_active":"2023-10-11T06:11:14.934742+0000","last_peered":"2023-10-11T06:11:14.934742+0000","last_clean":"2023-10-11T06:11:14.934742+0000","last_became_active":"2023-10-11T06:11:13.221160+0000","last_became_peered":"2023-10-11T06:11:13.221160+0000","last_unstale":"2023-10-11T06:11:14.934742+0000","last_undegraded":"2023-10-11T06:11:14.934742+0000","last_fullsized":"2023-10-11T06:11:14.934742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935425+0000","last_change":"2023-10-11T06:11:13.690241+0000","last_active":"2023-10-11T06:11:14.935425+0000","last_peered":"2023-10-11T06:11:14.935425+0000","last_clean":"2023-10-11T06:11:14.935425+0000","last_became_active":"2023-10-11T06:11:13.689766+0000","last_became_peered":"2023-10-11T06:11:13.689766+0000","last_unstale":"2023-10-11T06:11:14.935425+0000","last_undegraded":"2023-10-11T06:11:14.935425+0000","last_fullsized":"2023-10-11T06:11:14.935425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:46:46.784397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.936532+0000","last_change":"2023-10-11T06:11:13.212684+0000","last_active":"2023-10-11T06:11:14.936532+0000","last_peered":"2023-10-11T06:11:14.936532+0000","last_clean":"2023-10-11T06:11:14.936532+0000","last_became_active":"2023-10-11T06:11:13.212366+0000","last_became_peered":"2023-10-11T06:11:13.212366+0000","last_unstale":"2023-10-11T06:11:14.936532+0000","last_undegraded":"2023-10-11T06:11:14.936532+0000","last_fullsized":"2023-10-11T06:11:14.936532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:16:43.774215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688632+0000","last_change":"2023-10-11T06:11:13.706859+0000","last_active":"2023-10-11T06:11:14.688632+0000","last_peered":"2023-10-11T06:11:14.688632+0000","last_clean":"2023-10-11T06:11:14.688632+0000","last_became_active":"2023-10-11T06:11:13.706584+0000","last_became_peered":"2023-10-11T06:11:13.706584+0000","last_unstale":"2023-10-11T06:11:14.688632+0000","last_undegraded":"2023-10-11T06:11:14.688632+0000","last_fullsized":"2023-10-11T06:11:14.688632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:42:08.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.688832+0000","last_change":"2023-10-11T06:11:13.200955+0000","last_active":"2023-10-11T06:11:14.688832+0000","last_peered":"2023-10-11T06:11:14.688832+0000","last_clean":"2023-10-11T06:11:14.688832+0000","last_became_active":"2023-10-11T06:11:13.200617+0000","last_became_peered":"2023-10-11T06:11:13.200617+0000","last_unstale":"2023-10-11T06:11:14.688832+0000","last_undegraded":"2023-10-11T06:11:14.688832+0000","last_fullsized":"2023-10-11T06:11:14.688832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:46:38.465742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.935118+0000","last_change":"2023-10-11T06:11:13.689555+0000","last_active":"2023-10-11T06:11:14.935118+0000","last_peered":"2023-10-11T06:11:14.935118+0000","last_clean":"2023-10-11T06:11:14.935118+0000","last_became_active":"2023-10-11T06:11:13.689252+0000","last_became_peered":"2023-10-11T06:11:13.689252+0000","last_unstale":"2023-10-11T06:11:14.935118+0000","last_undegraded":"2023-10-11T06:11:14.935118+0000","last_fullsized":"2023-10-11T06:11:14.935118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:13.164572+0000","last_clean_scrub_stamp":"2023-10-11T06:11:13.164572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:40:49.157560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-11T06:11:14.933504+0000","last_change":"2023-10-11T06:11:13.213633+0000","last_active":"2023-10-11T06:11:14.933504+0000","last_peered":"2023-10-11T06:11:14.933504+0000","last_clean":"2023-10-11T06:11:14.933504+0000","last_became_active":"2023-10-11T06:11:13.213082+0000","last_became_peered":"2023-10-11T06:11:13.213082+0000","last_unstale":"2023-10-11T06:11:14.933504+0000","last_undegraded":"2023-10-11T06:11:14.933504+0000","last_fullsized":"2023-10-11T06:11:14.933504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:12.149946+0000","last_clean_scrub_stamp":"2023-10-11T06:11:12.149946+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:03:00.586669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":686418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":94922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":95890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":94360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":851968,"data_stored":685220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:11:22.468 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-11T06:11:22.468 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-11T06:11:22.468 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-11T06:11:22.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-11T06:11:22.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.710+0000 7fc50f1c9640 1 Processor -- start 2023-10-11T06:11:22.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.710+0000 7fc50f1c9640 1 -- start start 2023-10-11T06:11:22.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50f1c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc5080cb540 0x7fc508164f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50f1c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc508071a90 con 0x7fc5080ca300 2023-10-11T06:11:22.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50f1c9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5080cb970 con 0x7fc50815d4f0 2023-10-11T06:11:22.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50f1c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc508072310 con 0x7fc5080cb540 2023-10-11T06:11:22.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50d766640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc5080cb540 0x7fc508164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc507fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc5080ca300 0x7fc5080c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54838/0 (socket says 172.21.15.84:54838) 2023-10-11T06:11:22.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50d766640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc5080cb540 0x7fc508164f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54044/0 (socket says 172.21.15.84:54044) 2023-10-11T06:11:22.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc507fff640 1 -- 172.21.15.84:0/3319459917 learned_addr learned my addr 172.21.15.84:0/3319459917 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:22.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50d766640 1 -- 172.21.15.84:0/3319459917 >> v1:172.21.15.99:6789/0 conn(0x7fc50815d4f0 legacy=0x7fc5080c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50d766640 1 -- 172.21.15.84:0/3319459917 >> v1:172.21.15.84:6789/0 conn(0x7fc5080ca300 legacy=0x7fc5080c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc50d766640 1 -- 172.21.15.84:0/3319459917 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc508070280 con 0x7fc5080cb540 2023-10-11T06:11:22.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.712+0000 7fc5077fe640 1 -- 172.21.15.84:0/3319459917 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178006544 0 0) 0x7fc508071a90 con 0x7fc5080ca300 2023-10-11T06:11:22.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.713+0000 7fc50d766640 1 --2- 172.21.15.84:0/3319459917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc508164f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc500009d90 tx=0x7fc50002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798f57b5fd53418b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:22.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc5077fe640 1 -- 172.21.15.84:0/3319459917 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50002cc60 con 0x7fc5080cb540 2023-10-11T06:11:22.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc5077fe640 1 -- 172.21.15.84:0/3319459917 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50002cdc0 con 0x7fc5080cb540 2023-10-11T06:11:22.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc5077fe640 1 -- 172.21.15.84:0/3319459917 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500034660 con 0x7fc5080cb540 2023-10-11T06:11:22.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc50f1c9640 1 -- 172.21.15.84:0/3319459917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 msgr2=0x7fc508164f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/3319459917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc508164f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc500009d90 tx=0x7fc50002beb0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc50f1c9640 1 -- 172.21.15.84:0/3319459917 shutdown_connections 2023-10-11T06:11:22.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/3319459917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc508164f50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.714+0000 7fc50f1c9640 1 -- 172.21.15.84:0/3319459917 >> 172.21.15.84:0/3319459917 conn(0x7fc508069c70 msgr2=0x7fc50806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:22.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.715+0000 7fc50f1c9640 1 -- 172.21.15.84:0/3319459917 shutdown_connections 2023-10-11T06:11:22.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.715+0000 7fc50f1c9640 1 -- 172.21.15.84:0/3319459917 wait complete. 2023-10-11T06:11:22.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.716+0000 7fc50f1c9640 1 Processor -- start 2023-10-11T06:11:22.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.716+0000 7fc50f1c9640 1 -- start start 2023-10-11T06:11:22.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc50f1c9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5080ca300 0x7fc5080c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc50f1c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc50815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc50f1c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc50815d4f0 0x7fc508154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc50f1c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080c7390 con 0x7fc5080cb540 2023-10-11T06:11:22.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc50f1c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080c9520 con 0x7fc50815d4f0 2023-10-11T06:11:22.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc50f1c9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080c9800 con 0x7fc5080ca300 2023-10-11T06:11:22.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc507fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc50815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc507fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc50815ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54046/0 (socket says 172.21.15.84:54046) 2023-10-11T06:11:22.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.717+0000 7fc507fff640 1 -- 172.21.15.84:0/1785562410 learned_addr learned my addr 172.21.15.84:0/1785562410 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:22.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc507fff640 1 -- 172.21.15.84:0/1785562410 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5080ca300 msgr2=0x7fc5080c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc507fff640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5080ca300 0x7fc5080c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc507fff640 1 -- 172.21.15.84:0/1785562410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc50815d4f0 msgr2=0x7fc508154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc507fff640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc50815d4f0 0x7fc508154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc507fff640 1 -- 172.21.15.84:0/1785562410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500009a70 con 0x7fc5080cb540 2023-10-11T06:11:22.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc507fff640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc50815ae20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c3c0 tx=0x7fc4f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:22.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc5057fa640 1 -- 172.21.15.84:0/1785562410 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f40074e0 con 0x7fc5080cb540 2023-10-11T06:11:22.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc5057fa640 1 -- 172.21.15.84:0/1785562410 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f4007640 con 0x7fc5080cb540 2023-10-11T06:11:22.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc5057fa640 1 -- 172.21.15.84:0/1785562410 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4004a20 con 0x7fc5080cb540 2023-10-11T06:11:22.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5080c6d40 con 0x7fc5080cb540 2023-10-11T06:11:22.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.718+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5080c7d60 con 0x7fc5080cb540 2023-10-11T06:11:22.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.719+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc508073ed0 con 0x7fc5080cb540 2023-10-11T06:11:22.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.720+0000 7fc5057fa640 1 -- 172.21.15.84:0/1785562410 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc4f4004b80 con 0x7fc5080cb540 2023-10-11T06:11:22.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.721+0000 7fc5057fa640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc4ec06a830 0x7fc4ec06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:22.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.721+0000 7fc5057fa640 1 -- 172.21.15.84:0/1785562410 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4f4088e20 con 0x7fc5080cb540 2023-10-11T06:11:22.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.721+0000 7fc50cf65640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc4ec06a830 0x7fc4ec06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:22.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.722+0000 7fc50cf65640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc4ec06a830 0x7fc4ec06cce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc007fd0 tx=0x7fc4fc0079a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:22.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.725+0000 7fc5057fa640 1 -- 172.21.15.84:0/1785562410 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc4f4054000 con 0x7fc5080cb540 2023-10-11T06:11:22.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.941+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc5080b09b0 con 0x7fc5080cb540 2023-10-11T06:11:22.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.941+0000 7fc5057fa640 1 -- 172.21.15.84:0/1785562410 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc4f4057620 con 0x7fc5080cb540 2023-10-11T06:11:22.943 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:22.943 INFO:teuthology.orchestra.run.smithi084.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-11T06:11:22.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc4ec06a830 msgr2=0x7fc4ec06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc4ec06a830 0x7fc4ec06cce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc007fd0 tx=0x7fc4fc0079a0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 msgr2=0x7fc50815ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:22.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc50815ae20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c3c0 tx=0x7fc4f400c880 comp rx=0 tx=0).stop 2023-10-11T06:11:22.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 shutdown_connections 2023-10-11T06:11:22.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc4ec06a830 0x7fc4ec06cce0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc50815d4f0 0x7fc508154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5080cb540 0x7fc50815ae20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 --2- 172.21.15.84:0/1785562410 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5080ca300 0x7fc5080c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:22.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 >> 172.21.15.84:0/1785562410 conn(0x7fc508069c70 msgr2=0x7fc508157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:22.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 shutdown_connections 2023-10-11T06:11:22.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:22.944+0000 7fc50f1c9640 1 -- 172.21.15.84:0/1785562410 wait complete. 2023-10-11T06:11:22.963 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-11T06:11:22.964 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:23.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.162+0000 7fa1ee3fd640 1 Processor -- start 2023-10-11T06:11:23.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.162+0000 7fa1ee3fd640 1 -- start start 2023-10-11T06:11:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.163+0000 7fa1ee3fd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1e815d4f0 0x7fa1e80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.163+0000 7fa1ee3fd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80c9360 0x7fa1e80ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.163+0000 7fa1ee3fd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e8071a90 con 0x7fa1e80ca300 2023-10-11T06:11:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.163+0000 7fa1ee3fd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e8072310 con 0x7fa1e815d4f0 2023-10-11T06:11:23.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.163+0000 7fa1ee3fd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8070280 con 0x7fa1e80c9360 2023-10-11T06:11:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.164+0000 7fa1e7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1e815d4f0 0x7fa1e80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.164+0000 7fa1e7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1e815d4f0 0x7fa1e80cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54054/0 (socket says 172.21.15.84:54054) 2023-10-11T06:11:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.164+0000 7fa1e7fff640 1 -- 172.21.15.84:0/3524269279 learned_addr learned my addr 172.21.15.84:0/3524269279 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.164+0000 7fa1e7fff640 1 -- 172.21.15.84:0/3524269279 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80c9360 msgr2=0x7fa1e80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.164+0000 7fa1e7fff640 1 --2- 172.21.15.84:0/3524269279 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80c9360 0x7fa1e80ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.164+0000 7fa1e7fff640 1 -- 172.21.15.84:0/3524269279 >> v1:172.21.15.99:6789/0 conn(0x7fa1e80ca300 legacy=0x7fa1e80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.164+0000 7fa1e7fff640 1 -- 172.21.15.84:0/3524269279 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e8071cc0 con 0x7fa1e815d4f0 2023-10-11T06:11:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.165+0000 7fa1e7fff640 1 --2- 172.21.15.84:0/3524269279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e80cb540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80cf6a0 tx=0x7fa1d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8d4d894d88250a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.165+0000 7fa1e77fe640 1 -- 172.21.15.84:0/3524269279 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004230 con 0x7fa1e815d4f0 2023-10-11T06:11:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.165+0000 7fa1e77fe640 1 -- 172.21.15.84:0/3524269279 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8004390 con 0x7fa1e815d4f0 2023-10-11T06:11:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1e77fe640 1 -- 172.21.15.84:0/3524269279 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8031830 con 0x7fa1e815d4f0 2023-10-11T06:11:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/3524269279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 msgr2=0x7fa1e80cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/3524269279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e80cb540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80cf6a0 tx=0x7fa1d802b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/3524269279 shutdown_connections 2023-10-11T06:11:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/3524269279 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80c9360 0x7fa1e80ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/3524269279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e80cb540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/3524269279 >> 172.21.15.84:0/3524269279 conn(0x7fa1e8069c70 msgr2=0x7fa1e806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:23.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/3524269279 shutdown_connections 2023-10-11T06:11:23.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.166+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/3524269279 wait complete. 2023-10-11T06:11:23.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.168+0000 7fa1ee3fd640 1 Processor -- start 2023-10-11T06:11:23.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.168+0000 7fa1ee3fd640 1 -- start start 2023-10-11T06:11:23.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ee3fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e80c9360 0x7fa1e80b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ee3fd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80ca300 0x7fa1e80cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ee3fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e8150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ee3fd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80ccbe0 con 0x7fa1e815d4f0 2023-10-11T06:11:23.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ee3fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80b58a0 con 0x7fa1e80c9360 2023-10-11T06:11:23.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ee3fd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80cce30 con 0x7fa1e80ca300 2023-10-11T06:11:23.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e8150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e8150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54066/0 (socket says 172.21.15.84:54066) 2023-10-11T06:11:23.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 -- 172.21.15.84:0/1608881664 learned_addr learned my addr 172.21.15.84:0/1608881664 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:23.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1e7fff640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e80c9360 0x7fa1e80b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1df7fe640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80ca300 0x7fa1e80cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 -- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80ca300 msgr2=0x7fa1e80cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80ca300 0x7fa1e80cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 -- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e80c9360 msgr2=0x7fa1e80b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e80c9360 0x7fa1e80b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.169+0000 7fa1ec99a640 1 -- 172.21.15.84:0/1608881664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8009460 con 0x7fa1e815d4f0 2023-10-11T06:11:23.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.170+0000 7fa1ec99a640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e8150a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000b410 tx=0x7fa1e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:23.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.170+0000 7fa1e57fa640 1 -- 172.21.15.84:0/1608881664 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0011830 con 0x7fa1e815d4f0 2023-10-11T06:11:23.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.170+0000 7fa1e57fa640 1 -- 172.21.15.84:0/1608881664 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e00119b0 con 0x7fa1e815d4f0 2023-10-11T06:11:23.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.170+0000 7fa1e57fa640 1 -- 172.21.15.84:0/1608881664 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0011d60 con 0x7fa1e815d4f0 2023-10-11T06:11:23.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.170+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e80cd110 con 0x7fa1e815d4f0 2023-10-11T06:11:23.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.170+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e8150f70 con 0x7fa1e815d4f0 2023-10-11T06:11:23.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.171+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e80cd630 con 0x7fa1e815d4f0 2023-10-11T06:11:23.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.172+0000 7fa1e57fa640 1 -- 172.21.15.84:0/1608881664 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa1e001a400 con 0x7fa1e815d4f0 2023-10-11T06:11:23.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.173+0000 7fa1e57fa640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1c806a830 0x7fa1c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.173+0000 7fa1e57fa640 1 -- 172.21.15.84:0/1608881664 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa1e00891d0 con 0x7fa1e815d4f0 2023-10-11T06:11:23.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.173+0000 7fa1e7fff640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1c806a830 0x7fa1c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.174+0000 7fa1e7fff640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1c806a830 0x7fa1c806cce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8035000 tx=0x7fa1d8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:23.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.177+0000 7fa1e57fa640 1 -- 172.21.15.84:0/1608881664 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa1e80cd630 con 0x7fa1e815d4f0 2023-10-11T06:11:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.352+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1e80cd630 con 0x7fa1e815d4f0 2023-10-11T06:11:23.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.353+0000 7fa1e57fa640 1 -- 172.21.15.84:0/1608881664 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17763 (secure 0 0 0) 0x7fa1e80cd630 con 0x7fa1e815d4f0 2023-10-11T06:11:23.355 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:23.355 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:14.671090+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:23.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1c806a830 msgr2=0x7fa1c806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1c806a830 0x7fa1c806cce0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8035000 tx=0x7fa1d8008030 comp rx=0 tx=0).stop 2023-10-11T06:11:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 msgr2=0x7fa1e8150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e8150a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000b410 tx=0x7fa1e000b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 shutdown_connections 2023-10-11T06:11:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa1c806a830 0x7fa1c806cce0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e815d4f0 0x7fa1e8150a30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1e80ca300 0x7fa1e80cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 --2- 172.21.15.84:0/1608881664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e80c9360 0x7fa1e80b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.356+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 >> 172.21.15.84:0/1608881664 conn(0x7fa1e8069c70 msgr2=0x7fa1e80b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:23.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.357+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 shutdown_connections 2023-10-11T06:11:23.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.357+0000 7fa1ee3fd640 1 -- 172.21.15.84:0/1608881664 wait complete. 2023-10-11T06:11:23.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:23.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:23.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.577+0000 7ff6abfff640 1 Processor -- start 2023-10-11T06:11:23.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.577+0000 7ff6abfff640 1 -- start start 2023-10-11T06:11:23.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.578+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 0x7ff6ac0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.578+0000 7ff6abfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac15cd40 con 0x7ff6ac158e20 2023-10-11T06:11:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.578+0000 7ff6abfff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac15cfe0 con 0x7ff6ac1591f0 2023-10-11T06:11:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.578+0000 7ff6abfff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac15d2c0 con 0x7ff6ac0cc890 2023-10-11T06:11:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6aaffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff6ac158e20 0x7ff6ac0cae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54854/0 (socket says 172.21.15.84:54854) 2023-10-11T06:11:23.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6aaffd640 1 -- 172.21.15.84:0/1812336991 learned_addr learned my addr 172.21.15.84:0/1812336991 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:23.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 0x7ff6ac0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261255526 0 0) 0x7ff6ac15cd40 con 0x7ff6ac158e20 2023-10-11T06:11:23.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff698003520 con 0x7ff6ac158e20 2023-10-11T06:11:23.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143125686 0 0) 0x7ff698003520 con 0x7ff6ac158e20 2023-10-11T06:11:23.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 msgr2=0x7ff6ac0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:23.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.579+0000 7ff6aa7fc640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 0x7ff6ac0cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 >> v1:172.21.15.99:6789/0 conn(0x7ff6ac1591f0 legacy=0x7ff6ac0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac15d5a0 con 0x7ff6ac158e20 2023-10-11T06:11:23.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7ff69c001670 con 0x7ff6ac158e20 2023-10-11T06:11:23.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 >> v1:172.21.15.84:6789/0 conn(0x7ff6ac158e20 legacy=0x7ff6ac0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6980039a0 0x7ff698003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 0x7ff6980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698004d00 0x7ff698005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac15d2c0 con 0x7ff698004d00 2023-10-11T06:11:23.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69800dd20 con 0x7ff6980039a0 2023-10-11T06:11:23.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69800dee0 con 0x7ff6ac0cc890 2023-10-11T06:11:23.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6ab7fe640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698004d00 0x7ff698005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6aaffd640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6980039a0 0x7ff698003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 0x7ff6980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6ab7fe640 1 -- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 msgr2=0x7ff6980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6ab7fe640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6ac0cc890 0x7ff6980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6ab7fe640 1 -- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6980039a0 msgr2=0x7ff698003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6ab7fe640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6980039a0 0x7ff698003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.580+0000 7ff6ab7fe640 1 -- 172.21.15.84:0/1812336991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698003520 con 0x7ff698004d00 2023-10-11T06:11:23.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.581+0000 7ff6ab7fe640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698004d00 0x7ff698005170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4001e50 tx=0x7ff6980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957df4452a6a917a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4003b70 con 0x7ff698004d00 2023-10-11T06:11:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a4003cd0 con 0x7ff698004d00 2023-10-11T06:11:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6aa7fc640 1 -- 172.21.15.84:0/1812336991 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4005310 con 0x7ff698004d00 2023-10-11T06:11:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6abfff640 1 -- 172.21.15.84:0/1812336991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698004d00 msgr2=0x7ff698005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6abfff640 1 --2- 172.21.15.84:0/1812336991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff698004d00 0x7ff698005170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4001e50 tx=0x7ff6980057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6ab7fe640 1 -- 172.21.15.84:0/1812336991 reap_dead start 2023-10-11T06:11:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6abfff640 1 -- 172.21.15.84:0/1812336991 shutdown_connections 2023-10-11T06:11:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6abfff640 1 -- 172.21.15.84:0/1812336991 >> 172.21.15.84:0/1812336991 conn(0x7ff6ac06bfb0 msgr2=0x7ff6ac06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6abfff640 1 -- 172.21.15.84:0/1812336991 shutdown_connections 2023-10-11T06:11:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.582+0000 7ff6abfff640 1 -- 172.21.15.84:0/1812336991 wait complete. 2023-10-11T06:11:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.584+0000 7ff6abfff640 1 Processor -- start 2023-10-11T06:11:23.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.584+0000 7ff6abfff640 1 -- start start 2023-10-11T06:11:23.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff698004d00 0x7ff6ac0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6ac156e80 0x7ff6ac1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6abfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6ac0aea30 0x7ff6ac0aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6abfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0af3e0 con 0x7ff6ac156e80 2023-10-11T06:11:23.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6abfff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1575b0 con 0x7ff6ac0aea30 2023-10-11T06:11:23.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6abfff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac157890 con 0x7ff698004d00 2023-10-11T06:11:23.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6ac156e80 0x7ff6ac1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6ac156e80 0x7ff6ac1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54084/0 (socket says 172.21.15.84:54084) 2023-10-11T06:11:23.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.585+0000 7ff6a3fff640 1 -- 172.21.15.84:0/3744601754 learned_addr learned my addr 172.21.15.84:0/3744601754 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:23.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6aaffd640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff698004d00 0x7ff6ac0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a3fff640 1 -- 172.21.15.84:0/3744601754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff698004d00 msgr2=0x7ff6ac0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff698004d00 0x7ff6ac0af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a3fff640 1 -- 172.21.15.84:0/3744601754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6ac0aea30 msgr2=0x7ff6ac0aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6ac0aea30 0x7ff6ac0aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a3fff640 1 -- 172.21.15.84:0/3744601754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac157b70 con 0x7ff6ac156e80 2023-10-11T06:11:23.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a3fff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6ac156e80 0x7ff6ac1529c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff694011600 tx=0x7ff694011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:23.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a37fe640 1 -- 172.21.15.84:0/3744601754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694015870 con 0x7ff6ac156e80 2023-10-11T06:11:23.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a37fe640 1 -- 172.21.15.84:0/3744601754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6940159d0 con 0x7ff6ac156e80 2023-10-11T06:11:23.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6a37fe640 1 -- 172.21.15.84:0/3744601754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694015d40 con 0x7ff6ac156e80 2023-10-11T06:11:23.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.586+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac157e50 con 0x7ff6ac156e80 2023-10-11T06:11:23.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.587+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac1572d0 con 0x7ff6ac156e80 2023-10-11T06:11:23.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.588+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac04e890 con 0x7ff6ac156e80 2023-10-11T06:11:23.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.589+0000 7ff6a37fe640 1 -- 172.21.15.84:0/3744601754 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff69400f490 con 0x7ff6ac156e80 2023-10-11T06:11:23.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.589+0000 7ff6a37fe640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff68c06a860 0x7ff68c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:23.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.590+0000 7ff6a37fe640 1 -- 172.21.15.84:0/3744601754 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff6940894f0 con 0x7ff6ac156e80 2023-10-11T06:11:23.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.590+0000 7ff6aaffd640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff68c06a860 0x7ff68c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:23.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.590+0000 7ff6aaffd640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff68c06a860 0x7ff68c06cd10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff69c001bb0 tx=0x7ff69c000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:23.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.594+0000 7ff6a37fe640 1 -- 172.21.15.84:0/3744601754 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff6940546d0 con 0x7ff6ac156e80 2023-10-11T06:11:23.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.782+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6ac0b44d0 con 0x7ff6ac156e80 2023-10-11T06:11:23.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.783+0000 7ff6a37fe640 1 -- 172.21.15.84:0/3744601754 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff694057cf0 con 0x7ff6ac156e80 2023-10-11T06:11:23.785 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:23.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff68c06a860 msgr2=0x7ff68c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff68c06a860 0x7ff68c06cd10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff69c001bb0 tx=0x7ff69c000ed0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6ac156e80 msgr2=0x7ff6ac1529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:23.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6ac156e80 0x7ff6ac1529c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff694011600 tx=0x7ff694011ac0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 shutdown_connections 2023-10-11T06:11:23.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff68c06a860 0x7ff68c06cd10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6ac0aea30 0x7ff6ac0aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6ac156e80 0x7ff6ac1529c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 --2- 172.21.15.84:0/3744601754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff698004d00 0x7ff6ac0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:23.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.785+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 >> 172.21.15.84:0/3744601754 conn(0x7ff6ac06bfb0 msgr2=0x7ff6ac071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:23.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.786+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 shutdown_connections 2023-10-11T06:11:23.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:23.786+0000 7ff6abfff640 1 -- 172.21.15.84:0/3744601754 wait complete. 2023-10-11T06:11:23.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:24.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.010+0000 7f69fcb11640 1 Processor -- start 2023-10-11T06:11:24.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.010+0000 7f69fcb11640 1 -- start start 2023-10-11T06:11:24.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.011+0000 7f69fcb11640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f69f8158e20 0x7f69f80cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69fcb11640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69fcb11640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f69f81591f0 0x7f69f8069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69fcb11640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f80cb8e0 con 0x7f69f8158e20 2023-10-11T06:11:24.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69fcb11640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f815cd40 con 0x7f69f81591f0 2023-10-11T06:11:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69fcb11640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f815cfe0 con 0x7f69f80cc890 2023-10-11T06:11:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f69f8158e20 0x7f69f80cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f69f8158e20 0x7f69f80cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54086/0 (socket says 172.21.15.84:54086) 2023-10-11T06:11:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 -- 172.21.15.84:0/1076386584 learned_addr learned my addr 172.21.15.84:0/1076386584 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f7fff640 1 --2- 172.21.15.84:0/1076386584 >> v2:172.21.15.99:3300/0 conn(0x7f69f81591f0 0x7f69f8069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 -- 172.21.15.84:0/1076386584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 msgr2=0x7f69f80cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 --2- 172.21.15.84:0/1076386584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 -- 172.21.15.84:0/1076386584 >> v2:172.21.15.99:3300/0 conn(0x7f69f81591f0 msgr2=0x7f69f8069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:24.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 --2- 172.21.15.84:0/1076386584 >> v2:172.21.15.99:3300/0 conn(0x7f69f81591f0 0x7f69f8069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f77fe640 1 -- 172.21.15.84:0/1076386584 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f815d2c0 con 0x7f69f8158e20 2023-10-11T06:11:24.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.012+0000 7f69f7fff640 1 --2- 172.21.15.84:0/1076386584 >> v2:172.21.15.99:3300/0 conn(0x7f69f81591f0 0x7f69f8069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:24.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.013+0000 7f69f77fe640 1 --2- 172.21.15.84:0/1076386584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80cae60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cc800 tx=0x7f69e802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ff671d259b6fbf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69f67fc640 1 -- 172.21.15.84:0/1076386584 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e802edf0 con 0x7f69f8158e20 2023-10-11T06:11:24.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69f67fc640 1 -- 172.21.15.84:0/1076386584 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8036440 con 0x7f69f8158e20 2023-10-11T06:11:24.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69f67fc640 1 -- 172.21.15.84:0/1076386584 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80367d0 con 0x7f69f8158e20 2023-10-11T06:11:24.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 -- 172.21.15.84:0/1076386584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 msgr2=0x7f69f80cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 --2- 172.21.15.84:0/1076386584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80cae60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cc800 tx=0x7f69e802e040 comp rx=0 tx=0).stop 2023-10-11T06:11:24.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 -- 172.21.15.84:0/1076386584 shutdown_connections 2023-10-11T06:11:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 --2- 172.21.15.84:0/1076386584 >> v2:172.21.15.99:3300/0 conn(0x7f69f81591f0 0x7f69f8069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 --2- 172.21.15.84:0/1076386584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 --2- 172.21.15.84:0/1076386584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80cae60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 -- 172.21.15.84:0/1076386584 >> 172.21.15.84:0/1076386584 conn(0x7f69f806bfb0 msgr2=0x7f69f806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 -- 172.21.15.84:0/1076386584 shutdown_connections 2023-10-11T06:11:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.014+0000 7f69fcb11640 1 -- 172.21.15.84:0/1076386584 wait complete. 2023-10-11T06:11:24.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.016+0000 7f69fcb11640 1 Processor -- start 2023-10-11T06:11:24.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.017+0000 7f69fcb11640 1 -- start start 2023-10-11T06:11:24.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69fcb11640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69fcb11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69fcb11640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69f81591f0 0x7f69f81f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69fcb11640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81f14e0 con 0x7f69f8158e20 2023-10-11T06:11:24.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69fcb11640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80ac9a0 con 0x7f69f81591f0 2023-10-11T06:11:24.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69fcb11640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80ac390 con 0x7f69f80cc890 2023-10-11T06:11:24.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69f6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69f6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54090/0 (socket says 172.21.15.84:54090) 2023-10-11T06:11:24.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69f6ffd640 1 -- 172.21.15.84:0/3837256958 learned_addr learned my addr 172.21.15.84:0/3837256958 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:24.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.018+0000 7f69f77fe640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f6ffd640 1 -- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 msgr2=0x7f69f80aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f7fff640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69f81591f0 0x7f69f81f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f6ffd640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f6ffd640 1 -- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69f81591f0 msgr2=0x7f69f81f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f6ffd640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69f81591f0 0x7f69f81f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f6ffd640 1 -- 172.21.15.84:0/3837256958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e800b770 con 0x7f69f8158e20 2023-10-11T06:11:24.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f77fe640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:24.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f7fff640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69f81591f0 0x7f69f81f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:24.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69f6ffd640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80ace30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00b410 tx=0x7f69ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69dffff640 1 -- 172.21.15.84:0/3837256958 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec011830 con 0x7f69f8158e20 2023-10-11T06:11:24.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69dffff640 1 -- 172.21.15.84:0/3837256958 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec0119b0 con 0x7f69f8158e20 2023-10-11T06:11:24.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f80c2400 con 0x7f69f8158e20 2023-10-11T06:11:24.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69dffff640 1 -- 172.21.15.84:0/3837256958 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec011d60 con 0x7f69f8158e20 2023-10-11T06:11:24.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.019+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f80c2080 con 0x7f69f8158e20 2023-10-11T06:11:24.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.020+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f804e890 con 0x7f69f8158e20 2023-10-11T06:11:24.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.021+0000 7f69dffff640 1 -- 172.21.15.84:0/3837256958 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f69ec01a400 con 0x7f69f8158e20 2023-10-11T06:11:24.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.022+0000 7f69dffff640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69e406a830 0x7f69e406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.023+0000 7f69dffff640 1 -- 172.21.15.84:0/3837256958 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69ec0891e0 con 0x7f69f8158e20 2023-10-11T06:11:24.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.023+0000 7f69f77fe640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69e406a830 0x7f69e406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.023+0000 7f69f77fe640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69e406a830 0x7f69e406cce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69e8038000 tx=0x7f69e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.027+0000 7f69dffff640 1 -- 172.21.15.84:0/3837256958 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f69ec054450 con 0x7f69f8158e20 2023-10-11T06:11:24.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.201+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f69f80acae0 con 0x7f69f8158e20 2023-10-11T06:11:24.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.201+0000 7f69dffff640 1 -- 172.21.15.84:0/3837256958 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f69ec057a70 con 0x7f69f8158e20 2023-10-11T06:11:24.203 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:24.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69e406a830 msgr2=0x7f69e406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69e406a830 0x7f69e406cce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69e8038000 tx=0x7f69e8008040 comp rx=0 tx=0).stop 2023-10-11T06:11:24.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 msgr2=0x7f69f80ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80ace30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00b410 tx=0x7f69ec00b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 shutdown_connections 2023-10-11T06:11:24.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69e406a830 0x7f69e406cce0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69f81591f0 0x7f69f81f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f8158e20 0x7f69f80ace30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 --2- 172.21.15.84:0/3837256958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69f80cc890 0x7f69f80aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 >> 172.21.15.84:0/3837256958 conn(0x7f69f806bfb0 msgr2=0x7f69f81489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 shutdown_connections 2023-10-11T06:11:24.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.204+0000 7f69fcb11640 1 -- 172.21.15.84:0/3837256958 wait complete. 2023-10-11T06:11:24.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:11:24.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.439+0000 7fdf598e7640 1 Processor -- start 2023-10-11T06:11:24.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.439+0000 7fdf598e7640 1 -- start start 2023-10-11T06:11:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.441+0000 7fdf598e7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf5406c880 0x7fdf5406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.441+0000 7fdf598e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540cde80 con 0x7fdf54156be0 2023-10-11T06:11:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.441+0000 7fdf598e7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5406df80 con 0x7fdf54156810 2023-10-11T06:11:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.441+0000 7fdf598e7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf540ce140 con 0x7fdf5406c880 2023-10-11T06:11:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.441+0000 7fdf4bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf5406c880 0x7fdf5406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.441+0000 7fdf4bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf5406c880 0x7fdf5406d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54096/0 (socket says 172.21.15.84:54096) 2023-10-11T06:11:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.441+0000 7fdf4bfff640 1 -- 172.21.15.84:0/3047091559 learned_addr learned my addr 172.21.15.84:0/3047091559 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.442+0000 7fdf527fc640 1 -- 172.21.15.84:0/3047091559 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127204189 0 0) 0x7fdf540cde80 con 0x7fdf54156be0 2023-10-11T06:11:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.442+0000 7fdf527fc640 1 -- 172.21.15.84:0/3047091559 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf34003520 con 0x7fdf54156be0 2023-10-11T06:11:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.442+0000 7fdf4bfff640 1 -- 172.21.15.84:0/3047091559 >> v1:172.21.15.99:6789/0 conn(0x7fdf54156810 legacy=0x7fdf5406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.442+0000 7fdf4bfff640 1 -- 172.21.15.84:0/3047091559 >> v1:172.21.15.84:6789/0 conn(0x7fdf54156be0 legacy=0x7fdf540ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.442+0000 7fdf4bfff640 1 -- 172.21.15.84:0/3047091559 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540cc3b0 con 0x7fdf5406c880 2023-10-11T06:11:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.443+0000 7fdf4bfff640 1 --2- 172.21.15.84:0/3047091559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf5406c880 0x7fdf5406d8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84e445bb39e555f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.443+0000 7fdf527fc640 1 -- 172.21.15.84:0/3047091559 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c030860 con 0x7fdf5406c880 2023-10-11T06:11:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.443+0000 7fdf527fc640 1 -- 172.21.15.84:0/3047091559 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c0309c0 con 0x7fdf5406c880 2023-10-11T06:11:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.443+0000 7fdf527fc640 1 -- 172.21.15.84:0/3047091559 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c030d50 con 0x7fdf5406c880 2023-10-11T06:11:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.444+0000 7fdf598e7640 1 -- 172.21.15.84:0/3047091559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf5406c880 msgr2=0x7fdf5406d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.444+0000 7fdf598e7640 1 --2- 172.21.15.84:0/3047091559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf5406c880 0x7fdf5406d8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c02b9a0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.444+0000 7fdf598e7640 1 -- 172.21.15.84:0/3047091559 shutdown_connections 2023-10-11T06:11:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.444+0000 7fdf598e7640 1 --2- 172.21.15.84:0/3047091559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf5406c880 0x7fdf5406d8e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.444+0000 7fdf598e7640 1 -- 172.21.15.84:0/3047091559 >> 172.21.15.84:0/3047091559 conn(0x7fdf54069c70 msgr2=0x7fdf5406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.444+0000 7fdf598e7640 1 -- 172.21.15.84:0/3047091559 shutdown_connections 2023-10-11T06:11:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.444+0000 7fdf598e7640 1 -- 172.21.15.84:0/3047091559 wait complete. 2023-10-11T06:11:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.446+0000 7fdf598e7640 1 Processor -- start 2023-10-11T06:11:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.446+0000 7fdf598e7640 1 -- start start 2023-10-11T06:11:24.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf598e7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5406c880 0x7fdf54143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf598e7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf54156810 0x7fdf540c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf598e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf54156be0 0x7fdf540c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf598e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540c6e70 con 0x7fdf54156be0 2023-10-11T06:11:24.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf598e7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540c62f0 con 0x7fdf5406c880 2023-10-11T06:11:24.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf598e7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540c65d0 con 0x7fdf54156810 2023-10-11T06:11:24.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf54156be0 0x7fdf540c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf54156be0 0x7fdf540c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54098/0 (socket says 172.21.15.84:54098) 2023-10-11T06:11:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.447+0000 7fdf537fe640 1 -- 172.21.15.84:0/4094658282 learned_addr learned my addr 172.21.15.84:0/4094658282 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf52ffd640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5406c880 0x7fdf54143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf4bfff640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf54156810 0x7fdf540c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf537fe640 1 -- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf54156810 msgr2=0x7fdf540c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf537fe640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf54156810 0x7fdf540c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf537fe640 1 -- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5406c880 msgr2=0x7fdf54143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf537fe640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5406c880 0x7fdf54143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf537fe640 1 -- 172.21.15.84:0/4094658282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0094d0 con 0x7fdf54156be0 2023-10-11T06:11:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf4bfff640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf54156810 0x7fdf540c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:24.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf537fe640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf54156be0 0x7fdf540c5cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c6e0 tx=0x7fdf4c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf50ff9640 1 -- 172.21.15.84:0/4094658282 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c003b70 con 0x7fdf54156be0 2023-10-11T06:11:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf50ff9640 1 -- 172.21.15.84:0/4094658282 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c003cd0 con 0x7fdf54156be0 2023-10-11T06:11:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf50ff9640 1 -- 172.21.15.84:0/4094658282 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0052d0 con 0x7fdf54156be0 2023-10-11T06:11:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.448+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf540c68b0 con 0x7fdf54156be0 2023-10-11T06:11:24.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.449+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5414ac20 con 0x7fdf54156be0 2023-10-11T06:11:24.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.450+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf540adb30 con 0x7fdf54156be0 2023-10-11T06:11:24.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.451+0000 7fdf50ff9640 1 -- 172.21.15.84:0/4094658282 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdf4c005430 con 0x7fdf54156be0 2023-10-11T06:11:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.452+0000 7fdf50ff9640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdf3806a830 0x7fdf3806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.452+0000 7fdf50ff9640 1 -- 172.21.15.84:0/4094658282 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf4c088d90 con 0x7fdf54156be0 2023-10-11T06:11:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.452+0000 7fdf52ffd640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdf3806a830 0x7fdf3806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.452+0000 7fdf52ffd640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdf3806a830 0x7fdf3806cce0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf44000f60 tx=0x7fdf44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.456+0000 7fdf50ff9640 1 -- 172.21.15.84:0/4094658282 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdf540adb30 con 0x7fdf54156be0 2023-10-11T06:11:24.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.642+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf540adb30 con 0x7fdf54156be0 2023-10-11T06:11:24.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.643+0000 7fdf50ff9640 1 -- 172.21.15.84:0/4094658282 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf540adb30 con 0x7fdf54156be0 2023-10-11T06:11:24.645 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:11:24.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdf3806a830 msgr2=0x7fdf3806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdf3806a830 0x7fdf3806cce0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf44000f60 tx=0x7fdf44009040 comp rx=0 tx=0).stop 2023-10-11T06:11:24.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf54156be0 msgr2=0x7fdf540c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf54156be0 0x7fdf540c5cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c6e0 tx=0x7fdf4c00cba0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 shutdown_connections 2023-10-11T06:11:24.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdf3806a830 0x7fdf3806cce0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf54156be0 0x7fdf540c5cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf54156810 0x7fdf540c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 --2- 172.21.15.84:0/4094658282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5406c880 0x7fdf54143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 >> 172.21.15.84:0/4094658282 conn(0x7fdf54069c70 msgr2=0x7fdf5415ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:24.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 shutdown_connections 2023-10-11T06:11:24.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.646+0000 7fdf598e7640 1 -- 172.21.15.84:0/4094658282 wait complete. 2023-10-11T06:11:24.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:11:24.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.868+0000 7fc19d93c640 1 Processor -- start 2023-10-11T06:11:24.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.868+0000 7fc19d93c640 1 -- start start 2023-10-11T06:11:24.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.869+0000 7fc19d93c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc19814ff30 0x7fc1980c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.869+0000 7fc19d93c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.869+0000 7fc19d93c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1980cc7c0 0x7fc1980ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.869+0000 7fc19d93c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980c9b90 con 0x7fc1980cc7c0 2023-10-11T06:11:24.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc19d93c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19815da10 con 0x7fc19814ff30 2023-10-11T06:11:24.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc19d93c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19815ce00 con 0x7fc198150300 2023-10-11T06:11:24.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1980cc7c0 0x7fc1980ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1980cc7c0 0x7fc1980ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54106/0 (socket says 172.21.15.84:54106) 2023-10-11T06:11:24.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 -- 172.21.15.84:0/2988041802 learned_addr learned my addr 172.21.15.84:0/2988041802 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:24.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc196ffd640 1 --2- 172.21.15.84:0/2988041802 >> v2:172.21.15.99:3300/0 conn(0x7fc19814ff30 0x7fc1980c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc18ffff640 1 --2- 172.21.15.84:0/2988041802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 -- 172.21.15.84:0/2988041802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 msgr2=0x7fc1980cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:24.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 --2- 172.21.15.84:0/2988041802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 -- 172.21.15.84:0/2988041802 >> v2:172.21.15.99:3300/0 conn(0x7fc19814ff30 msgr2=0x7fc1980c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:24.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 --2- 172.21.15.84:0/2988041802 >> v2:172.21.15.99:3300/0 conn(0x7fc19814ff30 0x7fc1980c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc1977fe640 1 -- 172.21.15.84:0/2988041802 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19815d0e0 con 0x7fc1980cc7c0 2023-10-11T06:11:24.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.870+0000 7fc196ffd640 1 --2- 172.21.15.84:0/2988041802 >> v2:172.21.15.99:3300/0 conn(0x7fc19814ff30 0x7fc1980c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:24.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.871+0000 7fc1977fe640 1 --2- 172.21.15.84:0/2988041802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1980cc7c0 0x7fc1980ce510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc190009e50 tx=0x7fc19002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e7634c34b651d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.872+0000 7fc1967fc640 1 -- 172.21.15.84:0/2988041802 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19002c9f0 con 0x7fc1980cc7c0 2023-10-11T06:11:24.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.874+0000 7fc1967fc640 1 -- 172.21.15.84:0/2988041802 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19002cb50 con 0x7fc1980cc7c0 2023-10-11T06:11:24.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.874+0000 7fc1967fc640 1 -- 172.21.15.84:0/2988041802 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900343f0 con 0x7fc1980cc7c0 2023-10-11T06:11:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 -- 172.21.15.84:0/2988041802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1980cc7c0 msgr2=0x7fc1980ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 --2- 172.21.15.84:0/2988041802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1980cc7c0 0x7fc1980ce510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc190009e50 tx=0x7fc19002be50 comp rx=0 tx=0).stop 2023-10-11T06:11:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 -- 172.21.15.84:0/2988041802 shutdown_connections 2023-10-11T06:11:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 --2- 172.21.15.84:0/2988041802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1980cc7c0 0x7fc1980ce510 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 --2- 172.21.15.84:0/2988041802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 --2- 172.21.15.84:0/2988041802 >> v2:172.21.15.99:3300/0 conn(0x7fc19814ff30 0x7fc1980c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 -- 172.21.15.84:0/2988041802 >> 172.21.15.84:0/2988041802 conn(0x7fc198069c70 msgr2=0x7fc19806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:24.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 -- 172.21.15.84:0/2988041802 shutdown_connections 2023-10-11T06:11:24.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.875+0000 7fc19d93c640 1 -- 172.21.15.84:0/2988041802 wait complete. 2023-10-11T06:11:24.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.877+0000 7fc19d93c640 1 Processor -- start 2023-10-11T06:11:24.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.877+0000 7fc19d93c640 1 -- start start 2023-10-11T06:11:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc19d93c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc19814ff30 0x7fc19814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc19d93c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc19d93c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19806c7c0 0x7fc19806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc19d93c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980b1f60 con 0x7fc19814ff30 2023-10-11T06:11:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc19d93c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19806d180 con 0x7fc19806c7c0 2023-10-11T06:11:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc19d93c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981565d0 con 0x7fc198150300 2023-10-11T06:11:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc196ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc19814ff30 0x7fc19814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc196ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc19814ff30 0x7fc19814e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54112/0 (socket says 172.21.15.84:54112) 2023-10-11T06:11:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.878+0000 7fc196ffd640 1 -- 172.21.15.84:0/4244253311 learned_addr learned my addr 172.21.15.84:0/4244253311 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc18ffff640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc196ffd640 1 -- 172.21.15.84:0/4244253311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 msgr2=0x7fc1980b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc196ffd640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc196ffd640 1 -- 172.21.15.84:0/4244253311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19806c7c0 msgr2=0x7fc19806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:24.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc196ffd640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19806c7c0 0x7fc19806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:24.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc196ffd640 1 -- 172.21.15.84:0/4244253311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc190009b30 con 0x7fc19814ff30 2023-10-11T06:11:24.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc196ffd640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc19814ff30 0x7fc19814e4f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c7c0 tx=0x7fc18800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc194ff9640 1 -- 172.21.15.84:0/4244253311 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880079d0 con 0x7fc19814ff30 2023-10-11T06:11:24.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc194ff9640 1 -- 172.21.15.84:0/4244253311 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188007b30 con 0x7fc19814ff30 2023-10-11T06:11:24.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc194ff9640 1 -- 172.21.15.84:0/4244253311 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18800ce70 con 0x7fc19814ff30 2023-10-11T06:11:24.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1980c6400 con 0x7fc19814ff30 2023-10-11T06:11:24.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.879+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1980b18a0 con 0x7fc19814ff30 2023-10-11T06:11:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.880+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19806ab90 con 0x7fc19814ff30 2023-10-11T06:11:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.881+0000 7fc194ff9640 1 -- 172.21.15.84:0/4244253311 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc1880113f0 con 0x7fc19814ff30 2023-10-11T06:11:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.882+0000 7fc194ff9640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc17806a860 0x7fc17806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.882+0000 7fc194ff9640 1 -- 172.21.15.84:0/4244253311 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc188089060 con 0x7fc19814ff30 2023-10-11T06:11:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.883+0000 7fc18ffff640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc17806a860 0x7fc17806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.883+0000 7fc18ffff640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc17806a860 0x7fc17806cd10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc19815cb60 tx=0x7fc180008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:24.887+0000 7fc194ff9640 1 -- 172.21.15.84:0/4244253311 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc1880542e0 con 0x7fc19814ff30 2023-10-11T06:11:25.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.060+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc198073ed0 con 0x7fc19814ff30 2023-10-11T06:11:25.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.061+0000 7fc194ff9640 1 -- 172.21.15.84:0/4244253311 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc188057900 con 0x7fc19814ff30 2023-10-11T06:11:25.062 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:11:25.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc17806a860 msgr2=0x7fc17806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc17806a860 0x7fc17806cd10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc19815cb60 tx=0x7fc180008040 comp rx=0 tx=0).stop 2023-10-11T06:11:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc19814ff30 msgr2=0x7fc19814e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc19814ff30 0x7fc19814e4f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c7c0 tx=0x7fc18800cc80 comp rx=0 tx=0).stop 2023-10-11T06:11:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 shutdown_connections 2023-10-11T06:11:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc17806a860 0x7fc17806cd10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19806c7c0 0x7fc19806cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc198150300 0x7fc1980b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 --2- 172.21.15.84:0/4244253311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc19814ff30 0x7fc19814e4f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 >> 172.21.15.84:0/4244253311 conn(0x7fc198069c70 msgr2=0x7fc1980b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 shutdown_connections 2023-10-11T06:11:25.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.063+0000 7fc19d93c640 1 -- 172.21.15.84:0/4244253311 wait complete. 2023-10-11T06:11:25.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:11:25.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.286+0000 7f9a11111640 1 Processor -- start 2023-10-11T06:11:25.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.286+0000 7f9a11111640 1 -- start start 2023-10-11T06:11:25.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a11111640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9a0c0cda20 0x7f9a0c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a11111640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a0c06fd80 0x7f9a0c06bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a11111640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0ce090 con 0x7f9a0c0cd650 2023-10-11T06:11:25.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a11111640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ca670 con 0x7f9a0c06fd80 2023-10-11T06:11:25.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a11111640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ca910 con 0x7f9a0c0cda20 2023-10-11T06:11:25.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a10910640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a0c06fd80 0x7f9a0c06bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a10910640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a0c06fd80 0x7f9a0c06bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54116/0 (socket says 172.21.15.84:54116) 2023-10-11T06:11:25.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.288+0000 7f9a10910640 1 -- 172.21.15.84:0/226948807 learned_addr learned my addr 172.21.15.84:0/226948807 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.289+0000 7f9a0b7fe640 1 --2- 172.21.15.84:0/226948807 >> v2:172.21.15.99:3300/0 conn(0x7f9a0c0cda20 0x7f9a0c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.289+0000 7f9a10910640 1 -- 172.21.15.84:0/226948807 >> v2:172.21.15.99:3300/0 conn(0x7f9a0c0cda20 msgr2=0x7f9a0c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.289+0000 7f9a10910640 1 --2- 172.21.15.84:0/226948807 >> v2:172.21.15.99:3300/0 conn(0x7f9a0c0cda20 0x7f9a0c15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.289+0000 7f9a10910640 1 -- 172.21.15.84:0/226948807 >> v1:172.21.15.99:6789/0 conn(0x7f9a0c0cd650 legacy=0x7f9a0c0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.289+0000 7f9a10910640 1 -- 172.21.15.84:0/226948807 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0cabf0 con 0x7f9a0c06fd80 2023-10-11T06:11:25.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.290+0000 7f9a10910640 1 --2- 172.21.15.84:0/226948807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c06bd90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0cb650 tx=0x7f9a0402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7464fd7d5321a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:25.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a0affd640 1 -- 172.21.15.84:0/226948807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0402cb10 con 0x7f9a0c06fd80 2023-10-11T06:11:25.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a0affd640 1 -- 172.21.15.84:0/226948807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0402cc70 con 0x7f9a0c06fd80 2023-10-11T06:11:25.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a0affd640 1 -- 172.21.15.84:0/226948807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040343f0 con 0x7f9a0c06fd80 2023-10-11T06:11:25.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 -- 172.21.15.84:0/226948807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 msgr2=0x7f9a0c06bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 --2- 172.21.15.84:0/226948807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c06bd90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0cb650 tx=0x7f9a0402bf70 comp rx=0 tx=0).stop 2023-10-11T06:11:25.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 -- 172.21.15.84:0/226948807 shutdown_connections 2023-10-11T06:11:25.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 --2- 172.21.15.84:0/226948807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c06bd90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 --2- 172.21.15.84:0/226948807 >> v2:172.21.15.99:3300/0 conn(0x7f9a0c0cda20 0x7f9a0c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 -- 172.21.15.84:0/226948807 >> 172.21.15.84:0/226948807 conn(0x7f9a0c069c70 msgr2=0x7f9a0c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 -- 172.21.15.84:0/226948807 shutdown_connections 2023-10-11T06:11:25.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.291+0000 7f9a11111640 1 -- 172.21.15.84:0/226948807 wait complete. 2023-10-11T06:11:25.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.293+0000 7f9a11111640 1 Processor -- start 2023-10-11T06:11:25.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.293+0000 7f9a11111640 1 -- start start 2023-10-11T06:11:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.294+0000 7f9a11111640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c0ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a11111640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a0c0cd650 0x7f9a0c14de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a11111640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c0cda20 0x7f9a0c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a11111640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c157460 con 0x7f9a0c06fd80 2023-10-11T06:11:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a11111640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0acaa0 con 0x7f9a0c0cda20 2023-10-11T06:11:25.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a11111640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ac0d0 con 0x7f9a0c0cd650 2023-10-11T06:11:25.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c0ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c0ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54124/0 (socket says 172.21.15.84:54124) 2023-10-11T06:11:25.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 -- 172.21.15.84:0/4083185637 learned_addr learned my addr 172.21.15.84:0/4083185637 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:25.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 -- 172.21.15.84:0/4083185637 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a0c0cd650 msgr2=0x7f9a0c14de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a0c0cd650 0x7f9a0c14de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 -- 172.21.15.84:0/4083185637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c0cda20 msgr2=0x7f9a0c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c0cda20 0x7f9a0c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 -- 172.21.15.84:0/4083185637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009b30 con 0x7f9a0c06fd80 2023-10-11T06:11:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.295+0000 7f9a0bfff640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c0ac560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c4b0 tx=0x7f99fc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:25.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.296+0000 7f9a08ff9640 1 -- 172.21.15.84:0/4083185637 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0052d0 con 0x7f9a0c06fd80 2023-10-11T06:11:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.296+0000 7f9a08ff9640 1 -- 172.21.15.84:0/4083185637 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc005430 con 0x7f9a0c06fd80 2023-10-11T06:11:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.296+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c157180 con 0x7f9a0c06fd80 2023-10-11T06:11:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.296+0000 7f9a08ff9640 1 -- 172.21.15.84:0/4083185637 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc012870 con 0x7f9a0c06fd80 2023-10-11T06:11:25.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.296+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c0abd60 con 0x7f9a0c06fd80 2023-10-11T06:11:25.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.297+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c0ab8e0 con 0x7f9a0c06fd80 2023-10-11T06:11:25.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.298+0000 7f9a08ff9640 1 -- 172.21.15.84:0/4083185637 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f99fc0129d0 con 0x7f9a0c06fd80 2023-10-11T06:11:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.299+0000 7f9a08ff9640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f99f806a830 0x7f99f806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.299+0000 7f9a08ff9640 1 -- 172.21.15.84:0/4083185637 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99fc088a30 con 0x7f9a0c06fd80 2023-10-11T06:11:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.299+0000 7f9a0b7fe640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f99f806a830 0x7f99f806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.300+0000 7f9a0b7fe640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f99f806a830 0x7f99f806cce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a00006fd0 tx=0x7f9a00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:25.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.303+0000 7f9a08ff9640 1 -- 172.21.15.84:0/4083185637 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9a0c0ab8e0 con 0x7f9a0c06fd80 2023-10-11T06:11:25.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.527+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a0c0abf20 con 0x7f9a0c06fd80 2023-10-11T06:11:25.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.529+0000 7f9a08ff9640 1 -- 172.21.15.84:0/4083185637 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7f99fc053d40 con 0x7f9a0c06fd80 2023-10-11T06:11:25.530 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:25.531 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:11:25.531 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:11:25.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f99f806a830 msgr2=0x7f99f806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f99f806a830 0x7f99f806cce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a00006fd0 tx=0x7f9a00008040 comp rx=0 tx=0).stop 2023-10-11T06:11:25.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 msgr2=0x7f9a0c0ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c0ac560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c4b0 tx=0x7f99fc00c970 comp rx=0 tx=0).stop 2023-10-11T06:11:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 shutdown_connections 2023-10-11T06:11:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f99f806a830 0x7f99f806cce0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a0c0cda20 0x7f9a0c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a0c0cd650 0x7f9a0c14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 --2- 172.21.15.84:0/4083185637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a0c06fd80 0x7f9a0c0ac560 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.531+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 >> 172.21.15.84:0/4083185637 conn(0x7f9a0c069c70 msgr2=0x7f9a0c159780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:25.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.532+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 shutdown_connections 2023-10-11T06:11:25.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.532+0000 7f9a11111640 1 -- 172.21.15.84:0/4083185637 wait complete. 2023-10-11T06:11:25.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:25.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.759+0000 7fb947610640 1 Processor -- start 2023-10-11T06:11:25.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.759+0000 7fb947610640 1 -- start start 2023-10-11T06:11:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb947610640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400adb90 0x7fb9400caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb947610640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb9400cd300 0x7fb9400cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb947610640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400cb960 con 0x7fb9400ce0d0 2023-10-11T06:11:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb947610640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94015a340 con 0x7fb9400cd300 2023-10-11T06:11:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb947610640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400ce4a0 con 0x7fb9400adb90 2023-10-11T06:11:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb9453ac640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb9400ce0d0 0x7fb9400c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54870/0 (socket says 172.21.15.84:54870) 2023-10-11T06:11:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb9453ac640 1 -- 172.21.15.84:0/3072187573 learned_addr learned my addr 172.21.15.84:0/3072187573 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:25.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb945bad640 1 --2- 172.21.15.84:0/3072187573 >> v2:172.21.15.84:3300/0 conn(0x7fb9400cd300 0x7fb9400cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb944bab640 1 --2- 172.21.15.84:0/3072187573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400adb90 0x7fb9400caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb93ffff640 1 -- 172.21.15.84:0/3072187573 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84627674 0 0) 0x7fb9400cb960 con 0x7fb9400ce0d0 2023-10-11T06:11:25.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb93ffff640 1 -- 172.21.15.84:0/3072187573 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb930003520 con 0x7fb9400ce0d0 2023-10-11T06:11:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb945bad640 1 -- 172.21.15.84:0/3072187573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400adb90 msgr2=0x7fb9400caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb945bad640 1 --2- 172.21.15.84:0/3072187573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400adb90 0x7fb9400caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb945bad640 1 -- 172.21.15.84:0/3072187573 >> v1:172.21.15.84:6789/0 conn(0x7fb9400ce0d0 legacy=0x7fb9400c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb945bad640 1 -- 172.21.15.84:0/3072187573 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9401593f0 con 0x7fb9400cd300 2023-10-11T06:11:25.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.761+0000 7fb944bab640 1 --2- 172.21.15.84:0/3072187573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400adb90 0x7fb9400caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.763+0000 7fb945bad640 1 --2- 172.21.15.84:0/3072187573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400cd300 0x7fb9400cb420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097d0 tx=0x7fb93802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6f124b2481751 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.763+0000 7fb93ffff640 1 -- 172.21.15.84:0/3072187573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938031800 con 0x7fb9400cd300 2023-10-11T06:11:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.763+0000 7fb93ffff640 1 -- 172.21.15.84:0/3072187573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938031960 con 0x7fb9400cd300 2023-10-11T06:11:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.763+0000 7fb93ffff640 1 -- 172.21.15.84:0/3072187573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938031cf0 con 0x7fb9400cd300 2023-10-11T06:11:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 -- 172.21.15.84:0/3072187573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400cd300 msgr2=0x7fb9400cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 --2- 172.21.15.84:0/3072187573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400cd300 0x7fb9400cb420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097d0 tx=0x7fb93802b8a0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 -- 172.21.15.84:0/3072187573 shutdown_connections 2023-10-11T06:11:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 --2- 172.21.15.84:0/3072187573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400cd300 0x7fb9400cb420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 --2- 172.21.15.84:0/3072187573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400adb90 0x7fb9400caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 -- 172.21.15.84:0/3072187573 >> 172.21.15.84:0/3072187573 conn(0x7fb940069c70 msgr2=0x7fb94006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 -- 172.21.15.84:0/3072187573 shutdown_connections 2023-10-11T06:11:25.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.764+0000 7fb947610640 1 -- 172.21.15.84:0/3072187573 wait complete. 2023-10-11T06:11:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.765+0000 7fb947610640 1 Processor -- start 2023-10-11T06:11:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.766+0000 7fb947610640 1 -- start start 2023-10-11T06:11:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.766+0000 7fb947610640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9400adb90 0x7fb9400b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb947610640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400cd300 0x7fb94014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb947610640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400ce0d0 0x7fb94006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb947610640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400b1a80 con 0x7fb9400ce0d0 2023-10-11T06:11:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb947610640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400b0140 con 0x7fb9400adb90 2023-10-11T06:11:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb947610640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400b0420 con 0x7fb9400cd300 2023-10-11T06:11:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400ce0d0 0x7fb94006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400ce0d0 0x7fb94006bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54136/0 (socket says 172.21.15.84:54136) 2023-10-11T06:11:25.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 -- 172.21.15.84:0/1301723700 learned_addr learned my addr 172.21.15.84:0/1301723700 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:25.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 -- 172.21.15.84:0/1301723700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400cd300 msgr2=0x7fb94014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400cd300 0x7fb94014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 -- 172.21.15.84:0/1301723700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9400adb90 msgr2=0x7fb9400b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9400adb90 0x7fb9400b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 -- 172.21.15.84:0/1301723700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938009400 con 0x7fb9400ce0d0 2023-10-11T06:11:25.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.767+0000 7fb945bad640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400ce0d0 0x7fb94006bf30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb938009720 tx=0x7fb9380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:25.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.768+0000 7fb93dffb640 1 -- 172.21.15.84:0/1301723700 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93803abe0 con 0x7fb9400ce0d0 2023-10-11T06:11:25.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.768+0000 7fb93dffb640 1 -- 172.21.15.84:0/1301723700 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93803ad40 con 0x7fb9400ce0d0 2023-10-11T06:11:25.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.768+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9400b1430 con 0x7fb9400ce0d0 2023-10-11T06:11:25.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.768+0000 7fb93dffb640 1 -- 172.21.15.84:0/1301723700 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938044610 con 0x7fb9400ce0d0 2023-10-11T06:11:25.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.768+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9400b3f70 con 0x7fb9400ce0d0 2023-10-11T06:11:25.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.769+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb940066fc0 con 0x7fb9400ce0d0 2023-10-11T06:11:25.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.770+0000 7fb93dffb640 1 -- 172.21.15.84:0/1301723700 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb93804d450 con 0x7fb9400ce0d0 2023-10-11T06:11:25.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.771+0000 7fb93dffb640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb92406a830 0x7fb92406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:25.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.771+0000 7fb93dffb640 1 -- 172.21.15.84:0/1301723700 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb93802e070 con 0x7fb9400ce0d0 2023-10-11T06:11:25.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.771+0000 7fb9453ac640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb92406a830 0x7fb92406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:25.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.771+0000 7fb9453ac640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb92406a830 0x7fb92406cce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb934001360 tx=0x7fb9340077c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:25.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.775+0000 7fb93dffb640 1 -- 172.21.15.84:0/1301723700 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb9380764e0 con 0x7fb9400ce0d0 2023-10-11T06:11:25.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.968+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb94006ce70 con 0x7fb9400ce0d0 2023-10-11T06:11:25.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.970+0000 7fb93dffb640 1 -- 172.21.15.84:0/1301723700 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17763 (secure 0 0 0) 0x7fb938079b00 con 0x7fb9400ce0d0 2023-10-11T06:11:25.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:25.972 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:14.671090+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:25.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb92406a830 msgr2=0x7fb92406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb92406a830 0x7fb92406cce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb934001360 tx=0x7fb9340077c0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400ce0d0 msgr2=0x7fb94006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:25.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400ce0d0 0x7fb94006bf30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb938009720 tx=0x7fb9380046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 shutdown_connections 2023-10-11T06:11:25.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb92406a830 0x7fb92406cce0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9400ce0d0 0x7fb94006bf30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.972+0000 7fb947610640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400cd300 0x7fb94014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.973+0000 7fb947610640 1 --2- 172.21.15.84:0/1301723700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9400adb90 0x7fb9400b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.973+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 >> 172.21.15.84:0/1301723700 conn(0x7fb940069c70 msgr2=0x7fb9400c4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:25.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.973+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 shutdown_connections 2023-10-11T06:11:25.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:25.973+0000 7fb947610640 1 -- 172.21.15.84:0/1301723700 wait complete. 2023-10-11T06:11:25.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:25.993 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:26.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.204+0000 7fe2abe23640 1 Processor -- start 2023-10-11T06:11:26.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.205+0000 7fe2abe23640 1 -- start start 2023-10-11T06:11:26.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.206+0000 7fe2abe23640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2a415f7e0 0x7fe2a40cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.206+0000 7fe2abe23640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a40cd6c0 0x7fe2a40d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.206+0000 7fe2abe23640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a40d3460 con 0x7fe2a40d3000 2023-10-11T06:11:26.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.206+0000 7fe2abe23640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a40cf220 con 0x7fe2a415f7e0 2023-10-11T06:11:26.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.206+0000 7fe2abe23640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40cf500 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.206+0000 7fe2a9bbf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2a415f7e0 0x7fe2a40cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2aa3c0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe2a40d3000 0x7fe2a415fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54872/0 (socket says 172.21.15.84:54872) 2023-10-11T06:11:26.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2aa3c0640 1 -- 172.21.15.84:0/1093361983 learned_addr learned my addr 172.21.15.84:0/1093361983 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:26.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2a93be640 1 --2- 172.21.15.84:0/1093361983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a40cd6c0 0x7fe2a40d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2a9bbf640 1 -- 172.21.15.84:0/1093361983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a40cd6c0 msgr2=0x7fe2a40d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:26.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2a9bbf640 1 --2- 172.21.15.84:0/1093361983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a40cd6c0 0x7fe2a40d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2a9bbf640 1 -- 172.21.15.84:0/1093361983 >> v1:172.21.15.84:6789/0 conn(0x7fe2a40d3000 legacy=0x7fe2a415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2a9bbf640 1 -- 172.21.15.84:0/1093361983 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a40cf7e0 con 0x7fe2a415f7e0 2023-10-11T06:11:26.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.207+0000 7fe2a8bbd640 1 -- 172.21.15.84:0/1093361983 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046085752 0 0) 0x7fe2a40d3460 con 0x7fe2a40d3000 2023-10-11T06:11:26.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.208+0000 7fe2a9bbf640 1 --2- 172.21.15.84:0/1093361983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a415f7e0 0x7fe2a40cd180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe2940097a0 tx=0x7fe29402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c4d9d884637c63 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:26.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.208+0000 7fe2a8bbd640 1 -- 172.21.15.84:0/1093361983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294031830 con 0x7fe2a415f7e0 2023-10-11T06:11:26.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.208+0000 7fe2a8bbd640 1 -- 172.21.15.84:0/1093361983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294031990 con 0x7fe2a415f7e0 2023-10-11T06:11:26.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.208+0000 7fe2a8bbd640 1 -- 172.21.15.84:0/1093361983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294031d20 con 0x7fe2a415f7e0 2023-10-11T06:11:26.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 -- 172.21.15.84:0/1093361983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a415f7e0 msgr2=0x7fe2a40cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 --2- 172.21.15.84:0/1093361983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a415f7e0 0x7fe2a40cd180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe2940097a0 tx=0x7fe29402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 -- 172.21.15.84:0/1093361983 shutdown_connections 2023-10-11T06:11:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 --2- 172.21.15.84:0/1093361983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a40cd6c0 0x7fe2a40d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 --2- 172.21.15.84:0/1093361983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a415f7e0 0x7fe2a40cd180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 -- 172.21.15.84:0/1093361983 >> 172.21.15.84:0/1093361983 conn(0x7fe2a40713a0 msgr2=0x7fe2a40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:26.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 -- 172.21.15.84:0/1093361983 shutdown_connections 2023-10-11T06:11:26.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.209+0000 7fe2abe23640 1 -- 172.21.15.84:0/1093361983 wait complete. 2023-10-11T06:11:26.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.211+0000 7fe2abe23640 1 Processor -- start 2023-10-11T06:11:26.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.211+0000 7fe2abe23640 1 -- start start 2023-10-11T06:11:26.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2abe23640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a40cd6c0 0x7fe2a4154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2abe23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2a40d3000 0x7fe2a40c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2abe23640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a415f7e0 0x7fe2a4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2abe23640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c8250 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2abe23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c8530 con 0x7fe2a40d3000 2023-10-11T06:11:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2abe23640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40c8810 con 0x7fe2a415f7e0 2023-10-11T06:11:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2a9bbf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a40cd6c0 0x7fe2a4154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2a9bbf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a40cd6c0 0x7fe2a4154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54152/0 (socket says 172.21.15.84:54152) 2023-10-11T06:11:26.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.212+0000 7fe2a9bbf640 1 -- 172.21.15.84:0/674227236 learned_addr learned my addr 172.21.15.84:0/674227236 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:26.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a9bbf640 1 -- 172.21.15.84:0/674227236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a415f7e0 msgr2=0x7fe2a4155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a9bbf640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a415f7e0 0x7fe2a4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a9bbf640 1 -- 172.21.15.84:0/674227236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2a40d3000 msgr2=0x7fe2a40c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a9bbf640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2a40d3000 0x7fe2a40c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a9bbf640 1 -- 172.21.15.84:0/674227236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe294009400 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a9bbf640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a40cd6c0 0x7fe2a4154370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe2940097a0 tx=0x7fe2940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:26.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a27fc640 1 -- 172.21.15.84:0/674227236 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294004230 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a27fc640 1 -- 172.21.15.84:0/674227236 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294004390 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2a27fc640 1 -- 172.21.15.84:0/674227236 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29403a870 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.213+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a40c8af0 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.214+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a41548b0 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.215+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a40c9010 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.215+0000 7fe2a27fc640 1 -- 172.21.15.84:0/674227236 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe29403a9d0 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.216+0000 7fe2a27fc640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe29006a830 0x7fe29006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.216+0000 7fe2a27fc640 1 -- 172.21.15.84:0/674227236 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe2940ab2c0 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.217+0000 7fe2a93be640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe29006a830 0x7fe29006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.217+0000 7fe2a93be640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe29006a830 0x7fe29006cce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2a41f4d60 tx=0x7fe298008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:26.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.221+0000 7fe2a27fc640 1 -- 172.21.15.84:0/674227236 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe2a40c9010 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:11:26.283 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:11:26.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.411+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2a40c9010 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.411+0000 7fe2a27fc640 1 -- 172.21.15.84:0/674227236 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2940764a0 con 0x7fe2a40cd6c0 2023-10-11T06:11:26.414 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:26.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe29006a830 msgr2=0x7fe29006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe29006a830 0x7fe29006cce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2a41f4d60 tx=0x7fe298008040 comp rx=0 tx=0).stop 2023-10-11T06:11:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a40cd6c0 msgr2=0x7fe2a4154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a40cd6c0 0x7fe2a4154370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe2940097a0 tx=0x7fe2940046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 shutdown_connections 2023-10-11T06:11:26.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe29006a830 0x7fe29006cce0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2a415f7e0 0x7fe2a4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2a40d3000 0x7fe2a40c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 --2- 172.21.15.84:0/674227236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2a40cd6c0 0x7fe2a4154370 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 >> 172.21.15.84:0/674227236 conn(0x7fe2a40713a0 msgr2=0x7fe2a4071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:26.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 shutdown_connections 2023-10-11T06:11:26.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.414+0000 7fe2abe23640 1 -- 172.21.15.84:0/674227236 wait complete. 2023-10-11T06:11:26.434 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:26.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.698+0000 7f6cc5342640 1 Processor -- start 2023-10-11T06:11:26.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.699+0000 7f6cc5342640 1 -- start start 2023-10-11T06:11:26.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.700+0000 7f6cc5342640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00b1b10 0x7f6cc00cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.700+0000 7f6cc5342640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6cc00b1ee0 0x7f6cc00cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.700+0000 7f6cc5342640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc015cba0 con 0x7f6cc00cecb0 2023-10-11T06:11:26.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.700+0000 7f6cc5342640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc009c220 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.700+0000 7f6cc5342640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc00d1360 con 0x7f6cc00b1b10 2023-10-11T06:11:26.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.700+0000 7f6cbf7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6cc00b1ee0 0x7f6cc00cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cc4b41640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6cc00cecb0 0x7f6cc00ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54874/0 (socket says 172.21.15.84:54874) 2023-10-11T06:11:26.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cc4b41640 1 -- 172.21.15.84:0/2901777937 learned_addr learned my addr 172.21.15.84:0/2901777937 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cbffff640 1 --2- 172.21.15.84:0/2901777937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00b1b10 0x7f6cc00cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cbf7fe640 1 -- 172.21.15.84:0/2901777937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00b1b10 msgr2=0x7f6cc00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cbf7fe640 1 --2- 172.21.15.84:0/2901777937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00b1b10 0x7f6cc00cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cbf7fe640 1 -- 172.21.15.84:0/2901777937 >> v1:172.21.15.84:6789/0 conn(0x7f6cc00cecb0 legacy=0x7f6cc00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cbf7fe640 1 -- 172.21.15.84:0/2901777937 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc00d0ac0 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.701+0000 7f6cbffff640 1 --2- 172.21.15.84:0/2901777937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00b1b10 0x7f6cc00cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.702+0000 7f6cbf7fe640 1 --2- 172.21.15.84:0/2901777937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc00cf740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80097a0 tx=0x7f6ca802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c570feb76504a0e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.702+0000 7f6cbeffd640 1 -- 172.21.15.84:0/2901777937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8030800 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.702+0000 7f6cbeffd640 1 -- 172.21.15.84:0/2901777937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8030960 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.702+0000 7f6cbeffd640 1 -- 172.21.15.84:0/2901777937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8030cf0 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 -- 172.21.15.84:0/2901777937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 msgr2=0x7f6cc00cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 --2- 172.21.15.84:0/2901777937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc00cf740 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80097a0 tx=0x7f6ca802b940 comp rx=0 tx=0).stop 2023-10-11T06:11:26.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 -- 172.21.15.84:0/2901777937 shutdown_connections 2023-10-11T06:11:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 --2- 172.21.15.84:0/2901777937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc00cf740 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 --2- 172.21.15.84:0/2901777937 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00b1b10 0x7f6cc00cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 -- 172.21.15.84:0/2901777937 >> 172.21.15.84:0/2901777937 conn(0x7f6cc009bc20 msgr2=0x7f6cc00d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:26.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 -- 172.21.15.84:0/2901777937 shutdown_connections 2023-10-11T06:11:26.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.703+0000 7f6cc5342640 1 -- 172.21.15.84:0/2901777937 wait complete. 2023-10-11T06:11:26.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.705+0000 7f6cc5342640 1 Processor -- start 2023-10-11T06:11:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.705+0000 7f6cc5342640 1 -- start start 2023-10-11T06:11:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cc5342640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc00b1b10 0x7f6cc0150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cc5342640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc01591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cc5342640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00cecb0 0x7f6cc0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cc5342640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc015dd80 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cc5342640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc015d170 con 0x7f6cc00b1b10 2023-10-11T06:11:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cc5342640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc015d450 con 0x7f6cc00cecb0 2023-10-11T06:11:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cbf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc01591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cbf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc01591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54176/0 (socket says 172.21.15.84:54176) 2023-10-11T06:11:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cbf7fe640 1 -- 172.21.15.84:0/1104049663 learned_addr learned my addr 172.21.15.84:0/1104049663 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.706+0000 7f6cbffff640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc00b1b10 0x7f6cc0150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cc4b41640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00cecb0 0x7f6cc0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbf7fe640 1 -- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00cecb0 msgr2=0x7f6cc0159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbf7fe640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00cecb0 0x7f6cc0159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbf7fe640 1 -- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc00b1b10 msgr2=0x7f6cc0150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbf7fe640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc00b1b10 0x7f6cc0150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbf7fe640 1 -- 172.21.15.84:0/1104049663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009400 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbf7fe640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc01591c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ca802bf50 tx=0x7f6ca8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:26.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbcff9640 1 -- 172.21.15.84:0/1104049663 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca803e040 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbcff9640 1 -- 172.21.15.84:0/1104049663 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8039d00 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc015d730 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.707+0000 7f6cbcff9640 1 -- 172.21.15.84:0/1104049663 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8044630 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.708+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc00b6e70 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.709+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc015c400 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.710+0000 7f6cbcff9640 1 -- 172.21.15.84:0/1104049663 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6ca8043390 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.710+0000 7f6cbcff9640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ca006a830 0x7f6ca006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.711+0000 7f6cbcff9640 1 -- 172.21.15.84:0/1104049663 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ca80ab450 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.711+0000 7f6cbffff640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ca006a830 0x7f6ca006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.711+0000 7f6cbffff640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ca006a830 0x7f6ca006cce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007600 tx=0x7f6cb0007590 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.715+0000 7f6cbcff9640 1 -- 172.21.15.84:0/1104049663 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f6ca80766d0 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.906+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6cc004e890 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.906+0000 7f6cbcff9640 1 -- 172.21.15.84:0/1104049663 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6ca8079cf0 con 0x7f6cc00b1ee0 2023-10-11T06:11:26.909 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:26.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ca006a830 msgr2=0x7f6ca006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ca006a830 0x7f6ca006cce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007600 tx=0x7f6cb0007590 comp rx=0 tx=0).stop 2023-10-11T06:11:26.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 msgr2=0x7f6cc01591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:26.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc01591c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ca802bf50 tx=0x7f6ca8004730 comp rx=0 tx=0).stop 2023-10-11T06:11:26.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 shutdown_connections 2023-10-11T06:11:26.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ca006a830 0x7f6ca006cce0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cc00cecb0 0x7f6cc0159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cc00b1ee0 0x7f6cc01591c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 --2- 172.21.15.84:0/1104049663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cc00b1b10 0x7f6cc0150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:26.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 >> 172.21.15.84:0/1104049663 conn(0x7f6cc009bc20 msgr2=0x7f6cc00cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 shutdown_connections 2023-10-11T06:11:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:26.909+0000 7f6cc5342640 1 -- 172.21.15.84:0/1104049663 wait complete. 2023-10-11T06:11:26.930 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:11:27.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.143+0000 7fecf305e640 1 Processor -- start 2023-10-11T06:11:27.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.143+0000 7fecf305e640 1 -- start start 2023-10-11T06:11:27.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.144+0000 7fecf305e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 0x7fecec0affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.144+0000 7fecf305e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec0ccb60 con 0x7fecec154940 2023-10-11T06:11:27.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.144+0000 7fecf305e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec070420 con 0x7fecec154570 2023-10-11T06:11:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.144+0000 7fecf305e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec071e00 con 0x7fecec06fee0 2023-10-11T06:11:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.145+0000 7fecf185b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fecec154940 0x7fecec0cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54888/0 (socket says 172.21.15.84:54888) 2023-10-11T06:11:27.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.145+0000 7fecf185b640 1 -- 172.21.15.84:0/479469806 learned_addr learned my addr 172.21.15.84:0/479469806 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.145+0000 7fecf285d640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 0x7fecec0affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.145+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341394816 0 0) 0x7fecec0ccb60 con 0x7fecec154940 2023-10-11T06:11:27.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.145+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd8003520 con 0x7fecec154940 2023-10-11T06:11:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370923100 0 0) 0x7fecd8003520 con 0x7fecec154940 2023-10-11T06:11:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 msgr2=0x7fecec0affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:27.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 0x7fecec0affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 >> v1:172.21.15.99:6789/0 conn(0x7fecec154570 legacy=0x7fecec15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0720e0 con 0x7fecec154940 2023-10-11T06:11:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fecdc001670 con 0x7fecec154940 2023-10-11T06:11:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 >> v1:172.21.15.84:6789/0 conn(0x7fecec154940 legacy=0x7fecec0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd80039a0 0x7fecd8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 0x7fecd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd8004d00 0x7fecd8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec071e00 con 0x7fecd80039a0 2023-10-11T06:11:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd800ba60 con 0x7fecd8004d00 2023-10-11T06:11:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd800bc20 con 0x7fecec06fee0 2023-10-11T06:11:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf205c640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd80039a0 0x7fecd8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf185b640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 0x7fecd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf205c640 1 -- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 msgr2=0x7fecd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf205c640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 0x7fecd80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf205c640 1 -- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd8004d00 msgr2=0x7fecd8005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:27.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf205c640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd8004d00 0x7fecd8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf205c640 1 -- 172.21.15.84:0/479469806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8003520 con 0x7fecd80039a0 2023-10-11T06:11:27.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.146+0000 7fecf185b640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec06fee0 0x7fecd80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:27.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.147+0000 7fecf205c640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd80039a0 0x7fecd8003db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fece8001750 tx=0x7fecd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9a2b58bcc2878d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:27.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece80052d0 con 0x7fecd80039a0 2023-10-11T06:11:27.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece802e050 con 0x7fecd80039a0 2023-10-11T06:11:27.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf105a640 1 -- 172.21.15.84:0/479469806 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece80035f0 con 0x7fecd80039a0 2023-10-11T06:11:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf305e640 1 -- 172.21.15.84:0/479469806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd80039a0 msgr2=0x7fecd8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf305e640 1 --2- 172.21.15.84:0/479469806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecd80039a0 0x7fecd8003db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fece8001750 tx=0x7fecd80057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf285d640 1 -- 172.21.15.84:0/479469806 reap_dead start 2023-10-11T06:11:27.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf305e640 1 -- 172.21.15.84:0/479469806 shutdown_connections 2023-10-11T06:11:27.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf305e640 1 -- 172.21.15.84:0/479469806 >> 172.21.15.84:0/479469806 conn(0x7fecec06bfb0 msgr2=0x7fecec06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:27.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf305e640 1 -- 172.21.15.84:0/479469806 shutdown_connections 2023-10-11T06:11:27.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.148+0000 7fecf305e640 1 -- 172.21.15.84:0/479469806 wait complete. 2023-10-11T06:11:27.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.150+0000 7fecf305e640 1 Processor -- start 2023-10-11T06:11:27.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.150+0000 7fecf305e640 1 -- start start 2023-10-11T06:11:27.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf305e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd80039a0 0x7fecec1f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf305e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec159610 0x7fecec1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf305e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec0c1e90 0x7fecec0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf305e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec1f1d00 con 0x7fecec0c1e90 2023-10-11T06:11:27.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf305e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b1b00 con 0x7fecd80039a0 2023-10-11T06:11:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf305e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec159dd0 con 0x7fecec159610 2023-10-11T06:11:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf285d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec0c1e90 0x7fecec0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.151+0000 7fecf285d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec0c1e90 0x7fecec0b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54196/0 (socket says 172.21.15.84:54196) 2023-10-11T06:11:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf285d640 1 -- 172.21.15.84:0/2935324554 learned_addr learned my addr 172.21.15.84:0/2935324554 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf185b640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec159610 0x7fecec1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf205c640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd80039a0 0x7fecec1f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf285d640 1 -- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec159610 msgr2=0x7fecec1506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf285d640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec159610 0x7fecec1506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf285d640 1 -- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd80039a0 msgr2=0x7fecec1f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf285d640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd80039a0 0x7fecec1f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf285d640 1 -- 172.21.15.84:0/2935324554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec150c20 con 0x7fecec0c1e90 2023-10-11T06:11:27.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf185b640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec159610 0x7fecec1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:27.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf205c640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd80039a0 0x7fecec1f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:27.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fecf285d640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec0c1e90 0x7fecec0b32d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fecec06cf70 tx=0x7fece400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:27.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fece2ffd640 1 -- 172.21.15.84:0/2935324554 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4010040 con 0x7fecec0c1e90 2023-10-11T06:11:27.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fece2ffd640 1 -- 172.21.15.84:0/2935324554 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4009e90 con 0x7fecec0c1e90 2023-10-11T06:11:27.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.152+0000 7fece2ffd640 1 -- 172.21.15.84:0/2935324554 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4014a50 con 0x7fecec0c1e90 2023-10-11T06:11:27.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.153+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec0c4870 con 0x7fecec0c1e90 2023-10-11T06:11:27.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.153+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec0b14b0 con 0x7fecec0c1e90 2023-10-11T06:11:27.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.154+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec0b1cb0 con 0x7fecec0c1e90 2023-10-11T06:11:27.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.155+0000 7fece2ffd640 1 -- 172.21.15.84:0/2935324554 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fece4014bb0 con 0x7fecec0c1e90 2023-10-11T06:11:27.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.156+0000 7fece2ffd640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fecd006a860 0x7fecd006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.156+0000 7fece2ffd640 1 -- 172.21.15.84:0/2935324554 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fece4088d30 con 0x7fecec0c1e90 2023-10-11T06:11:27.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.156+0000 7fecf205c640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fecd006a860 0x7fecd006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.157+0000 7fecf205c640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fecd006a860 0x7fecd006cd10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fece8033000 tx=0x7fece8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:27.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.160+0000 7fece2ffd640 1 -- 172.21.15.84:0/2935324554 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fecec0b1cb0 con 0x7fecec0c1e90 2023-10-11T06:11:27.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.347+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecec0b1c70 con 0x7fecec0c1e90 2023-10-11T06:11:27.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.348+0000 7fece2ffd640 1 -- 172.21.15.84:0/2935324554 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fece4018020 con 0x7fecec0c1e90 2023-10-11T06:11:27.351 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:11:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.350+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fecd006a860 msgr2=0x7fecd006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.350+0000 7fecf305e640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fecd006a860 0x7fecd006cd10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fece8033000 tx=0x7fece8009030 comp rx=0 tx=0).stop 2023-10-11T06:11:27.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.350+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec0c1e90 msgr2=0x7fecec0b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.350+0000 7fecf305e640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec0c1e90 0x7fecec0b32d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fecec06cf70 tx=0x7fece400ee80 comp rx=0 tx=0).stop 2023-10-11T06:11:27.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 shutdown_connections 2023-10-11T06:11:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fecd006a860 0x7fecd006cd10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecec0c1e90 0x7fecec0b32d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecec159610 0x7fecec1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 --2- 172.21.15.84:0/2935324554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd80039a0 0x7fecec1f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 >> 172.21.15.84:0/2935324554 conn(0x7fecec06bfb0 msgr2=0x7fecec0adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:27.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 shutdown_connections 2023-10-11T06:11:27.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.351+0000 7fecf305e640 1 -- 172.21.15.84:0/2935324554 wait complete. 2023-10-11T06:11:27.371 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:11:27.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.589+0000 7f46b98e7640 1 Processor -- start 2023-10-11T06:11:27.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.590+0000 7f46b98e7640 1 -- start start 2023-10-11T06:11:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.591+0000 7f46b98e7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f46b4156810 0x7f46b406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.591+0000 7f46b98e7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46b4156be0 0x7f46b415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.591+0000 7f46b98e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b40cde80 con 0x7f46b406c170 2023-10-11T06:11:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46b98e7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40ce140 con 0x7f46b4156be0 2023-10-11T06:11:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46b98e7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b415da30 con 0x7f46b4156810 2023-10-11T06:11:27.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46b37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f46b406c170 0x7f46b4160020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54896/0 (socket says 172.21.15.84:54896) 2023-10-11T06:11:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46b37fe640 1 -- 172.21.15.84:0/3786325093 learned_addr learned my addr 172.21.15.84:0/3786325093 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46abfff640 1 --2- 172.21.15.84:0/3786325093 >> v2:172.21.15.84:3300/0 conn(0x7f46b4156be0 0x7f46b415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46b2ffd640 1 --2- 172.21.15.84:0/3786325093 >> v2:172.21.15.99:3300/0 conn(0x7f46b4156810 0x7f46b406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46b27fc640 1 -- 172.21.15.84:0/3786325093 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731570411 0 0) 0x7f46b40cde80 con 0x7f46b406c170 2023-10-11T06:11:27.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46b27fc640 1 -- 172.21.15.84:0/3786325093 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46a0003520 con 0x7f46b406c170 2023-10-11T06:11:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46abfff640 1 -- 172.21.15.84:0/3786325093 >> v2:172.21.15.99:3300/0 conn(0x7f46b4156810 msgr2=0x7f46b406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:27.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46abfff640 1 --2- 172.21.15.84:0/3786325093 >> v2:172.21.15.99:3300/0 conn(0x7f46b4156810 0x7f46b406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46abfff640 1 -- 172.21.15.84:0/3786325093 >> v1:172.21.15.84:6789/0 conn(0x7f46b406c170 legacy=0x7f46b4160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.592+0000 7f46abfff640 1 -- 172.21.15.84:0/3786325093 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b406df80 con 0x7f46b4156be0 2023-10-11T06:11:27.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.593+0000 7f46abfff640 1 --2- 172.21.15.84:0/3786325093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b415d4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46b406e470 tx=0x7f469c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9675b8ed3bab6308 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:27.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b27fc640 1 -- 172.21.15.84:0/3786325093 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c02e070 con 0x7f46b4156be0 2023-10-11T06:11:27.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b27fc640 1 -- 172.21.15.84:0/3786325093 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c0315a0 con 0x7f46b4156be0 2023-10-11T06:11:27.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b27fc640 1 -- 172.21.15.84:0/3786325093 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c031930 con 0x7f46b4156be0 2023-10-11T06:11:27.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 -- 172.21.15.84:0/3786325093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 msgr2=0x7f46b415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3786325093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b415d4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46b406e470 tx=0x7f469c02b840 comp rx=0 tx=0).stop 2023-10-11T06:11:27.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 -- 172.21.15.84:0/3786325093 shutdown_connections 2023-10-11T06:11:27.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3786325093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b415d4f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3786325093 >> v2:172.21.15.99:3300/0 conn(0x7f46b4156810 0x7f46b406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 -- 172.21.15.84:0/3786325093 >> 172.21.15.84:0/3786325093 conn(0x7f46b4069c70 msgr2=0x7f46b406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:27.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 -- 172.21.15.84:0/3786325093 shutdown_connections 2023-10-11T06:11:27.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.594+0000 7f46b98e7640 1 -- 172.21.15.84:0/3786325093 wait complete. 2023-10-11T06:11:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.596+0000 7f46b98e7640 1 Processor -- start 2023-10-11T06:11:27.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.596+0000 7f46b98e7640 1 -- start start 2023-10-11T06:11:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b98e7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b406c170 0x7f46b4151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b98e7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b4156810 0x7f46b40b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b98e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b40b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b98e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4153b90 con 0x7f46b4156be0 2023-10-11T06:11:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b98e7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4153e70 con 0x7f46b4156810 2023-10-11T06:11:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b98e7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4154150 con 0x7f46b406c170 2023-10-11T06:11:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b40b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b40b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54206/0 (socket says 172.21.15.84:54206) 2023-10-11T06:11:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.597+0000 7f46b37fe640 1 -- 172.21.15.84:0/3884761119 learned_addr learned my addr 172.21.15.84:0/3884761119 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b2ffd640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b406c170 0x7f46b4151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46abfff640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b4156810 0x7f46b40b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b37fe640 1 -- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b406c170 msgr2=0x7f46b4151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b37fe640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b406c170 0x7f46b4151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b37fe640 1 -- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b4156810 msgr2=0x7f46b40b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b37fe640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b4156810 0x7f46b40b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b37fe640 1 -- 172.21.15.84:0/3884761119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c009400 con 0x7f46b4156be0 2023-10-11T06:11:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b37fe640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b40b2c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00cb40 tx=0x7f46ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b0ff9640 1 -- 172.21.15.84:0/3884761119 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00aca0 con 0x7f46b4156be0 2023-10-11T06:11:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b0ff9640 1 -- 172.21.15.84:0/3884761119 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac00ae00 con 0x7f46b4156be0 2023-10-11T06:11:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b0ff9640 1 -- 172.21.15.84:0/3884761119 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac012620 con 0x7f46b4156be0 2023-10-11T06:11:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.598+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b4154430 con 0x7f46b4156be0 2023-10-11T06:11:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.599+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b4151a60 con 0x7f46b4156be0 2023-10-11T06:11:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.600+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4154950 con 0x7f46b4156be0 2023-10-11T06:11:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.600+0000 7f46b0ff9640 1 -- 172.21.15.84:0/3884761119 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f46ac007070 con 0x7f46b4156be0 2023-10-11T06:11:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.601+0000 7f46b0ff9640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f469406a830 0x7f469406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.602+0000 7f46b0ff9640 1 -- 172.21.15.84:0/3884761119 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46ac088fe0 con 0x7f46b4156be0 2023-10-11T06:11:27.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.602+0000 7f46b2ffd640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f469406a830 0x7f469406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.602+0000 7f46b2ffd640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f469406a830 0x7f469406cce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009ad0 tx=0x7f46a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:27.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.606+0000 7f46b0ff9640 1 -- 172.21.15.84:0/3884761119 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f46ac0542b0 con 0x7f46b4156be0 2023-10-11T06:11:27.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.780+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46b40cf390 con 0x7f46b4156be0 2023-10-11T06:11:27.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.780+0000 7f46b0ff9640 1 -- 172.21.15.84:0/3884761119 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46ac0170a0 con 0x7f46b4156be0 2023-10-11T06:11:27.782 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:11:27.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.782+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f469406a830 msgr2=0x7f469406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.782+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f469406a830 0x7f469406cce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009ad0 tx=0x7f46a4008040 comp rx=0 tx=0).stop 2023-10-11T06:11:27.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.782+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 msgr2=0x7f46b40b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.782+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b40b2c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00cb40 tx=0x7f46ac00a060 comp rx=0 tx=0).stop 2023-10-11T06:11:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 shutdown_connections 2023-10-11T06:11:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f469406a830 0x7f469406cce0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b4156be0 0x7f46b40b2c80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b4156810 0x7f46b40b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 --2- 172.21.15.84:0/3884761119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b406c170 0x7f46b4151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 >> 172.21.15.84:0/3884761119 conn(0x7f46b4069c70 msgr2=0x7f46b415fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 shutdown_connections 2023-10-11T06:11:27.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:27.783+0000 7f46b98e7640 1 -- 172.21.15.84:0/3884761119 wait complete. 2023-10-11T06:11:27.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:11:28.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.004+0000 7f8ed656e640 1 Processor -- start 2023-10-11T06:11:28.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.004+0000 7f8ed656e640 1 -- start start 2023-10-11T06:11:28.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.005+0000 7f8ed656e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ed015d4f0 0x7f8ed00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ed656e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed00ca300 0x7f8ed00c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ed656e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed0071a90 con 0x7f8ed00c9190 2023-10-11T06:11:28.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ed656e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed0072310 con 0x7f8ed015d4f0 2023-10-11T06:11:28.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ed656e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0070280 con 0x7f8ed00ca300 2023-10-11T06:11:28.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ecf7fe640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed00ca300 0x7f8ed00c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ecffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ed015d4f0 0x7f8ed00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ecffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ed015d4f0 0x7f8ed00cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:35718/0 (socket says 172.21.15.84:35718) 2023-10-11T06:11:28.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ecffff640 1 -- 172.21.15.84:0/1632750515 learned_addr learned my addr 172.21.15.84:0/1632750515 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:28.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.006+0000 7f8ed4b0b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8ed00c9190 0x7f8ed015dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:53360/0 (socket says 172.21.15.84:53360) 2023-10-11T06:11:28.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.007+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/1632750515 >> v2:172.21.15.99:3300/0 conn(0x7f8ed015d4f0 msgr2=0x7f8ed00cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:28.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.007+0000 7f8ecf7fe640 1 --2- 172.21.15.84:0/1632750515 >> v2:172.21.15.99:3300/0 conn(0x7f8ed015d4f0 0x7f8ed00cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.007+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/1632750515 >> v1:172.21.15.99:6789/0 conn(0x7f8ed00c9190 legacy=0x7f8ed015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.007+0000 7f8ecf7fe640 1 -- 172.21.15.84:0/1632750515 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed0071cc0 con 0x7f8ed00ca300 2023-10-11T06:11:28.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.007+0000 7f8ecffff640 1 --2- 172.21.15.84:0/1632750515 >> v2:172.21.15.99:3300/0 conn(0x7f8ed015d4f0 0x7f8ed00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:28.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.008+0000 7f8ecf7fe640 1 --2- 172.21.15.84:0/1632750515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed00ca300 0x7f8ed00c8c50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40097a0 tx=0x7f8ec402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c00cc9880ed8bf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8eceffd640 1 -- 172.21.15.84:0/1632750515 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec402d040 con 0x7f8ed00ca300 2023-10-11T06:11:28.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8eceffd640 1 -- 172.21.15.84:0/1632750515 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4031810 con 0x7f8ed00ca300 2023-10-11T06:11:28.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8eceffd640 1 -- 172.21.15.84:0/1632750515 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4031ba0 con 0x7f8ed00ca300 2023-10-11T06:11:28.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8ed656e640 1 -- 172.21.15.84:0/1632750515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed00ca300 msgr2=0x7f8ed00c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8ed656e640 1 --2- 172.21.15.84:0/1632750515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed00ca300 0x7f8ed00c8c50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40097a0 tx=0x7f8ec402b8b0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8ed656e640 1 -- 172.21.15.84:0/1632750515 shutdown_connections 2023-10-11T06:11:28.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8ed656e640 1 --2- 172.21.15.84:0/1632750515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed00ca300 0x7f8ed00c8c50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8ed656e640 1 --2- 172.21.15.84:0/1632750515 >> v2:172.21.15.99:3300/0 conn(0x7f8ed015d4f0 0x7f8ed00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.009+0000 7f8ed656e640 1 -- 172.21.15.84:0/1632750515 >> 172.21.15.84:0/1632750515 conn(0x7f8ed0069c70 msgr2=0x7f8ed006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:28.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.010+0000 7f8ed656e640 1 -- 172.21.15.84:0/1632750515 shutdown_connections 2023-10-11T06:11:28.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.010+0000 7f8ed656e640 1 -- 172.21.15.84:0/1632750515 wait complete. 2023-10-11T06:11:28.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.011+0000 7f8ed656e640 1 Processor -- start 2023-10-11T06:11:28.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.011+0000 7f8ed656e640 1 -- start start 2023-10-11T06:11:28.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ed656e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed00c9190 0x7f8ed014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ed656e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ed00ca300 0x7f8ed013c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ed656e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed015d4f0 0x7f8ed015da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ed656e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00c56d0 con 0x7f8ed00c9190 2023-10-11T06:11:28.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ed656e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed014e840 con 0x7f8ed00ca300 2023-10-11T06:11:28.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ed656e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed014eb20 con 0x7f8ed015d4f0 2023-10-11T06:11:28.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed00c9190 0x7f8ed014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ecffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed00c9190 0x7f8ed014e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54214/0 (socket says 172.21.15.84:54214) 2023-10-11T06:11:28.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.012+0000 7f8ecffff640 1 -- 172.21.15.84:0/847457639 learned_addr learned my addr 172.21.15.84:0/847457639 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:28.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8ecffff640 1 -- 172.21.15.84:0/847457639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed015d4f0 msgr2=0x7f8ed015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8ecffff640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed015d4f0 0x7f8ed015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8ecffff640 1 -- 172.21.15.84:0/847457639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ed00ca300 msgr2=0x7f8ed013c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8ecffff640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ed00ca300 0x7f8ed013c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8ecffff640 1 -- 172.21.15.84:0/847457639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec0009460 con 0x7f8ed00c9190 2023-10-11T06:11:28.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8ecffff640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed00c9190 0x7f8ed014e010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000e9b0 tx=0x7f8ec000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8eccff9640 1 -- 172.21.15.84:0/847457639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000c9a0 con 0x7f8ed00c9190 2023-10-11T06:11:28.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8eccff9640 1 -- 172.21.15.84:0/847457639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec000cb00 con 0x7f8ed00c9190 2023-10-11T06:11:28.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec4009400 con 0x7f8ed00c9190 2023-10-11T06:11:28.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.013+0000 7f8eccff9640 1 -- 172.21.15.84:0/847457639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000ce70 con 0x7f8ed00c9190 2023-10-11T06:11:28.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.014+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed014f0e0 con 0x7f8ed00c9190 2023-10-11T06:11:28.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.015+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed0073ed0 con 0x7f8ed00c9190 2023-10-11T06:11:28.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.015+0000 7f8eccff9640 1 -- 172.21.15.84:0/847457639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8ec0005990 con 0x7f8ed00c9190 2023-10-11T06:11:28.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.016+0000 7f8eccff9640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ebc06a830 0x7f8ebc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.016+0000 7f8eccff9640 1 -- 172.21.15.84:0/847457639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ec00894b0 con 0x7f8ed00c9190 2023-10-11T06:11:28.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.017+0000 7f8ecf7fe640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ebc06a830 0x7f8ebc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.017+0000 7f8ecf7fe640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ebc06a830 0x7f8ebc06cce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4036000 tx=0x7f8ec4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.021+0000 7f8eccff9640 1 -- 172.21.15.84:0/847457639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8ec0054690 con 0x7f8ed00c9190 2023-10-11T06:11:28.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.244+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ed00cc4e0 con 0x7f8ed00c9190 2023-10-11T06:11:28.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.245+0000 7f8eccff9640 1 -- 172.21.15.84:0/847457639 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7f8ec0057cb0 con 0x7f8ed00c9190 2023-10-11T06:11:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:28.247 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:11:28.247 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:11:28.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ebc06a830 msgr2=0x7f8ebc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ebc06a830 0x7f8ebc06cce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4036000 tx=0x7f8ec4030000 comp rx=0 tx=0).stop 2023-10-11T06:11:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed00c9190 msgr2=0x7f8ed014e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed00c9190 0x7f8ed014e010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000e9b0 tx=0x7f8ec000ee70 comp rx=0 tx=0).stop 2023-10-11T06:11:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 shutdown_connections 2023-10-11T06:11:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ebc06a830 0x7f8ebc06cce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ed015d4f0 0x7f8ed015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ed00ca300 0x7f8ed013c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 --2- 172.21.15.84:0/847457639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed00c9190 0x7f8ed014e010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 >> 172.21.15.84:0/847457639 conn(0x7f8ed0069c70 msgr2=0x7f8ed01534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 shutdown_connections 2023-10-11T06:11:28.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.248+0000 7f8ed656e640 1 -- 172.21.15.84:0/847457639 wait complete. 2023-10-11T06:11:28.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:28.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.477+0000 7fd3bb301640 1 Processor -- start 2023-10-11T06:11:28.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.477+0000 7fd3bb301640 1 -- start start 2023-10-11T06:11:28.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.478+0000 7fd3bb301640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 0x7fd3b40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3bb301640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b415fe40 con 0x7fd3b40c9190 2023-10-11T06:11:28.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3bb301640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b406ff30 con 0x7fd3b40cb6c0 2023-10-11T06:11:28.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3bb301640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40cad70 con 0x7fd3b415c290 2023-10-11T06:11:28.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3b889c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd3b40c9190 0x7fd3b40cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54912/0 (socket says 172.21.15.84:54912) 2023-10-11T06:11:28.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3b889c640 1 -- 172.21.15.84:0/4242743335 learned_addr learned my addr 172.21.15.84:0/4242743335 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:28.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3b909d640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 0x7fd3b40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058612936 0 0) 0x7fd3b415fe40 con 0x7fd3b40c9190 2023-10-11T06:11:28.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.479+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3a0003520 con 0x7fd3b40c9190 2023-10-11T06:11:28.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642624015 0 0) 0x7fd3a0003520 con 0x7fd3b40c9190 2023-10-11T06:11:28.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 msgr2=0x7fd3b40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:28.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 0x7fd3b40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 >> v1:172.21.15.99:6789/0 conn(0x7fd3b40cb6c0 legacy=0x7fd3b40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b41600e0 con 0x7fd3b40c9190 2023-10-11T06:11:28.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fd3a4001670 con 0x7fd3b40c9190 2023-10-11T06:11:28.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 >> v1:172.21.15.84:6789/0 conn(0x7fd3b40c9190 legacy=0x7fd3b40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 0x7fd3a00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0003f20 0x7fd3a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3a0005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40cad70 con 0x7fd3a0004d00 2023-10-11T06:11:28.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a000ba60 con 0x7fd3a0003f20 2023-10-11T06:11:28.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a000bc20 con 0x7fd3b415c290 2023-10-11T06:11:28.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3b989e640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3a0005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.480+0000 7fd3b909d640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 0x7fd3a00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.481+0000 7fd3b889c640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0003f20 0x7fd3a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.481+0000 7fd3b989e640 1 -- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 msgr2=0x7fd3a00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:28.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.481+0000 7fd3b989e640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 0x7fd3a00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.481+0000 7fd3b989e640 1 -- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0003f20 msgr2=0x7fd3a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:28.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.481+0000 7fd3b989e640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3a0003f20 0x7fd3a00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.481+0000 7fd3b989e640 1 -- 172.21.15.84:0/4242743335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a0003520 con 0x7fd3a0004d00 2023-10-11T06:11:28.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.481+0000 7fd3b909d640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b415c290 0x7fd3a00039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:28.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.482+0000 7fd3b989e640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3a0005170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac001e20 tx=0x7fd3a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921c63d4c34d3bfd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.482+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac003b70 con 0x7fd3a0004d00 2023-10-11T06:11:28.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.482+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac003cd0 con 0x7fd3a0004d00 2023-10-11T06:11:28.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.482+0000 7fd3abfff640 1 -- 172.21.15.84:0/4242743335 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac005310 con 0x7fd3a0004d00 2023-10-11T06:11:28.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.482+0000 7fd3bb301640 1 -- 172.21.15.84:0/4242743335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 msgr2=0x7fd3a0005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.482+0000 7fd3bb301640 1 --2- 172.21.15.84:0/4242743335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3a0005170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac001e20 tx=0x7fd3a00057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.482+0000 7fd3b989e640 1 -- 172.21.15.84:0/4242743335 reap_dead start 2023-10-11T06:11:28.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.483+0000 7fd3bb301640 1 -- 172.21.15.84:0/4242743335 shutdown_connections 2023-10-11T06:11:28.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.483+0000 7fd3bb301640 1 -- 172.21.15.84:0/4242743335 >> 172.21.15.84:0/4242743335 conn(0x7fd3b4069c70 msgr2=0x7fd3b406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:28.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.483+0000 7fd3bb301640 1 -- 172.21.15.84:0/4242743335 shutdown_connections 2023-10-11T06:11:28.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.483+0000 7fd3bb301640 1 -- 172.21.15.84:0/4242743335 wait complete. 2023-10-11T06:11:28.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.484+0000 7fd3bb301640 1 Processor -- start 2023-10-11T06:11:28.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.485+0000 7fd3bb301640 1 -- start start 2023-10-11T06:11:28.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.485+0000 7fd3bb301640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3b414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.485+0000 7fd3bb301640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40c56d0 0x7fd3b40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3bb301640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b40cc4a0 0x7fd3b40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3bb301640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b41f6290 con 0x7fd3a0004d00 2023-10-11T06:11:28.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3bb301640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c6080 con 0x7fd3b40c56d0 2023-10-11T06:11:28.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3bb301640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40cce50 con 0x7fd3b40cc4a0 2023-10-11T06:11:28.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3b414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3b414d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54224/0 (socket says 172.21.15.84:54224) 2023-10-11T06:11:28.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 -- 172.21.15.84:0/739308912 learned_addr learned my addr 172.21.15.84:0/739308912 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:28.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 -- 172.21.15.84:0/739308912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b40cc4a0 msgr2=0x7fd3b40cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b40cc4a0 0x7fd3b40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 -- 172.21.15.84:0/739308912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40c56d0 msgr2=0x7fd3b40c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40c56d0 0x7fd3b40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 -- 172.21.15.84:0/739308912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b41f5610 con 0x7fd3a0004d00 2023-10-11T06:11:28.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.486+0000 7fd3b909d640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3b414d640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0010930 tx=0x7fd3b0010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.487+0000 7fd3a9ffb640 1 -- 172.21.15.84:0/739308912 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0004220 con 0x7fd3a0004d00 2023-10-11T06:11:28.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.487+0000 7fd3a9ffb640 1 -- 172.21.15.84:0/739308912 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0004380 con 0x7fd3a0004d00 2023-10-11T06:11:28.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.487+0000 7fd3a9ffb640 1 -- 172.21.15.84:0/739308912 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0005bd0 con 0x7fd3a0004d00 2023-10-11T06:11:28.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.487+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b41f58f0 con 0x7fd3a0004d00 2023-10-11T06:11:28.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.487+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b41f5eb0 con 0x7fd3a0004d00 2023-10-11T06:11:28.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.488+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b4073ed0 con 0x7fd3a0004d00 2023-10-11T06:11:28.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.489+0000 7fd3a9ffb640 1 -- 172.21.15.84:0/739308912 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd3b0005d30 con 0x7fd3a0004d00 2023-10-11T06:11:28.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.490+0000 7fd3a9ffb640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd39806a860 0x7fd39806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.490+0000 7fd3a9ffb640 1 -- 172.21.15.84:0/739308912 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd3b0089320 con 0x7fd3a0004d00 2023-10-11T06:11:28.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.490+0000 7fd3b889c640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd39806a860 0x7fd39806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.490+0000 7fd3b889c640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd39806a860 0x7fd39806cd10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40c5260 tx=0x7fd3a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.494+0000 7fd3a9ffb640 1 -- 172.21.15.84:0/739308912 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd3b0054590 con 0x7fd3a0004d00 2023-10-11T06:11:28.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.685+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3b406a320 con 0x7fd3a0004d00 2023-10-11T06:11:28.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.687+0000 7fd3a9ffb640 1 -- 172.21.15.84:0/739308912 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17763 (secure 0 0 0) 0x7fd3b0057bb0 con 0x7fd3a0004d00 2023-10-11T06:11:28.689 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:28.689 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:14.671090+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:28.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd39806a860 msgr2=0x7fd39806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd39806a860 0x7fd39806cd10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40c5260 tx=0x7fd3a4009040 comp rx=0 tx=0).stop 2023-10-11T06:11:28.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 msgr2=0x7fd3b414d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3b414d640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0010930 tx=0x7fd3b0010df0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 shutdown_connections 2023-10-11T06:11:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd39806a860 0x7fd39806cd10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3b40cc4a0 0x7fd3b40cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b40c56d0 0x7fd3b40c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 --2- 172.21.15.84:0/739308912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3a0004d00 0x7fd3b414d640 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.689+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 >> 172.21.15.84:0/739308912 conn(0x7fd3b4069c70 msgr2=0x7fd3b40abda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.690+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 shutdown_connections 2023-10-11T06:11:28.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.690+0000 7fd3bb301640 1 -- 172.21.15.84:0/739308912 wait complete. 2023-10-11T06:11:28.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:11:28.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.925+0000 7f0da1eb6640 1 Processor -- start 2023-10-11T06:11:28.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.925+0000 7f0da1eb6640 1 -- start start 2023-10-11T06:11:28.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.926+0000 7f0da1eb6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d9c15c290 0x7f0d9c0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0da1eb6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d9c0cb6c0 0x7f0d9c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0da1eb6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9c15fe40 con 0x7f0d9c0c9190 2023-10-11T06:11:28.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0da1eb6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9c0cad70 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0da1eb6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9c06ff30 con 0x7f0d9c15c290 2023-10-11T06:11:28.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d9c0cb6c0 0x7f0d9c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d9c0cb6c0 0x7f0d9c0cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35322/0 (socket says 172.21.15.84:35322) 2023-10-11T06:11:28.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9bfff640 1 -- 172.21.15.84:0/1569551472 learned_addr learned my addr 172.21.15.84:0/1569551472 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:28.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d93fff640 1 --1- 172.21.15.84:0/1569551472 >> v1:172.21.15.84:6789/0 conn(0x7f0d9c0c9190 0x7f0d9c0cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43522/0 (socket says 172.21.15.84:43522) 2023-10-11T06:11:28.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9b7fe640 1 --2- 172.21.15.84:0/1569551472 >> v2:172.21.15.99:3300/0 conn(0x7f0d9c15c290 0x7f0d9c0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9bfff640 1 -- 172.21.15.84:0/1569551472 >> v2:172.21.15.99:3300/0 conn(0x7f0d9c15c290 msgr2=0x7f0d9c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:28.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9bfff640 1 --2- 172.21.15.84:0/1569551472 >> v2:172.21.15.99:3300/0 conn(0x7f0d9c15c290 0x7f0d9c0c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9bfff640 1 -- 172.21.15.84:0/1569551472 >> v1:172.21.15.84:6789/0 conn(0x7f0d9c0c9190 legacy=0x7f0d9c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.927+0000 7f0d9bfff640 1 -- 172.21.15.84:0/1569551472 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c1600e0 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.928+0000 7f0d9bfff640 1 --2- 172.21.15.84:0/1569551472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c0cc410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0b0810 tx=0x7f0d9402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a777797f0bb7d4ae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0d9affd640 1 -- 172.21.15.84:0/1569551472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9402cb10 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0d9affd640 1 -- 172.21.15.84:0/1569551472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9402cc70 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0d9affd640 1 -- 172.21.15.84:0/1569551472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d940343f0 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0da1eb6640 1 -- 172.21.15.84:0/1569551472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 msgr2=0x7f0d9c0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/1569551472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c0cc410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0b0810 tx=0x7f0d9402bf70 comp rx=0 tx=0).stop 2023-10-11T06:11:28.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0da1eb6640 1 -- 172.21.15.84:0/1569551472 shutdown_connections 2023-10-11T06:11:28.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/1569551472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c0cc410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/1569551472 >> v2:172.21.15.99:3300/0 conn(0x7f0d9c15c290 0x7f0d9c0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0da1eb6640 1 -- 172.21.15.84:0/1569551472 >> 172.21.15.84:0/1569551472 conn(0x7f0d9c069c70 msgr2=0x7f0d9c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:28.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.929+0000 7f0da1eb6640 1 -- 172.21.15.84:0/1569551472 shutdown_connections 2023-10-11T06:11:28.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.930+0000 7f0da1eb6640 1 -- 172.21.15.84:0/1569551472 wait complete. 2023-10-11T06:11:28.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.931+0000 7f0da1eb6640 1 Processor -- start 2023-10-11T06:11:28.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.931+0000 7f0da1eb6640 1 -- start start 2023-10-11T06:11:28.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.932+0000 7f0da1eb6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d9c0c9190 0x7f0d9c15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.932+0000 7f0da1eb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.932+0000 7f0da1eb6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9c15c290 0x7f0d9c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.932+0000 7f0da1eb6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c1f4ef0 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.932+0000 7f0da1eb6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c0c49b0 con 0x7f0d9c15c290 2023-10-11T06:11:28.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.932+0000 7f0da1eb6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c0c3e30 con 0x7f0d9c0c9190 2023-10-11T06:11:28.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.932+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35324/0 (socket says 172.21.15.84:35324) 2023-10-11T06:11:28.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 -- 172.21.15.84:0/2413286643 learned_addr learned my addr 172.21.15.84:0/2413286643 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:28.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 -- 172.21.15.84:0/2413286643 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d9c0c9190 msgr2=0x7f0d9c15f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d9b7fe640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d9c0c9190 0x7f0d9c15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d9c0c9190 0x7f0d9c15f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 -- 172.21.15.84:0/2413286643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9c15c290 msgr2=0x7f0d9c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:28.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9c15c290 0x7f0d9c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:28.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 -- 172.21.15.84:0/2413286643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d94009b30 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d93fff640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c15c660 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400c450 tx=0x7f0d8400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d98ff9640 1 -- 172.21.15.84:0/2413286643 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84007480 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.933+0000 7f0d98ff9640 1 -- 172.21.15.84:0/2413286643 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d840075e0 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.934+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9c0c4110 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.934+0000 7f0d98ff9640 1 -- 172.21.15.84:0/2413286643 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84004a00 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.934+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c0c46d0 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.935+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9c066fc0 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.936+0000 7f0d98ff9640 1 -- 172.21.15.84:0/2413286643 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0d84004b60 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.936+0000 7f0d98ff9640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0d7c06a830 0x7f0d7c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:28.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.937+0000 7f0d98ff9640 1 -- 172.21.15.84:0/2413286643 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d840880a0 con 0x7f0d9c0cb6c0 2023-10-11T06:11:28.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.937+0000 7f0d9b7fe640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0d7c06a830 0x7f0d7c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:28.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.937+0000 7f0d9b7fe640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0d7c06a830 0x7f0d7c06cce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0096d0 tx=0x7f0d8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:28.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:28.941+0000 7f0d98ff9640 1 -- 172.21.15.84:0/2413286643 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0d84053310 con 0x7f0d9c0cb6c0 2023-10-11T06:11:29.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.165+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d9c143780 con 0x7f0d9c0cb6c0 2023-10-11T06:11:29.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.166+0000 7f0d98ff9640 1 -- 172.21.15.84:0/2413286643 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7f0d9c143780 con 0x7f0d9c0cb6c0 2023-10-11T06:11:29.168 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:29.168 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:11:29.168 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:11:29.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0d7c06a830 msgr2=0x7f0d7c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0d7c06a830 0x7f0d7c06cce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0096d0 tx=0x7f0d8c008040 comp rx=0 tx=0).stop 2023-10-11T06:11:29.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 msgr2=0x7f0d9c15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c15c660 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400c450 tx=0x7f0d8400c910 comp rx=0 tx=0).stop 2023-10-11T06:11:29.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 shutdown_connections 2023-10-11T06:11:29.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0d7c06a830 0x7f0d7c06cce0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d9c15c290 0x7f0d9c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d9c0cb6c0 0x7f0d9c15c660 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 --2- 172.21.15.84:0/2413286643 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d9c0c9190 0x7f0d9c15f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 >> 172.21.15.84:0/2413286643 conn(0x7f0d9c069c70 msgr2=0x7f0d9c0ab9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:29.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 shutdown_connections 2023-10-11T06:11:29.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.169+0000 7f0da1eb6640 1 -- 172.21.15.84:0/2413286643 wait complete. 2023-10-11T06:11:29.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T06:11:13.667179+0000', 'modified': '2023-10-11T06:11:16.824257+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4314}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4314': {'gid': 4314, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6835/2350376542', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6833', 'nonce': 2350376542}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2350376542}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T06:11:29.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T06:11:29.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-11T06:11:29.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.814+0000 7f61f3fff640 1 Processor -- start 2023-10-11T06:11:29.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.815+0000 7f61f3fff640 1 -- start start 2023-10-11T06:11:29.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.816+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 0x7f61f415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.816+0000 7f61f3fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f415da50 con 0x7f61f40b3e70 2023-10-11T06:11:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.816+0000 7f61f3fff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40cad10 con 0x7f61f40ceae0 2023-10-11T06:11:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.816+0000 7f61f3fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415dca0 con 0x7f61f406fd80 2023-10-11T06:11:29.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.816+0000 7f61f2ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61f40b3e70 0x7f61f40ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43530/0 (socket says 172.21.15.84:43530) 2023-10-11T06:11:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.816+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1071457005 learned_addr learned my addr 172.21.15.84:0/1071457005 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134352721 0 0) 0x7f61f415da50 con 0x7f61f40b3e70 2023-10-11T06:11:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f37fe640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 0x7f61f415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e0003520 con 0x7f61f40b3e70 2023-10-11T06:11:29.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000280553 0 0) 0x7f61e0003520 con 0x7f61f40b3e70 2023-10-11T06:11:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 msgr2=0x7f61f415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 0x7f61f415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 >> v1:172.21.15.99:6789/0 conn(0x7f61f40ceae0 legacy=0x7f61f415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f415df80 con 0x7f61f40b3e70 2023-10-11T06:11:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f61e4001670 con 0x7f61f40b3e70 2023-10-11T06:11:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 >> v1:172.21.15.84:6789/0 conn(0x7f61f40b3e70 legacy=0x7f61f40ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 0x7f61e00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.817+0000 7f61f27fc640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e0003f20 0x7f61e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61f27fc640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e0004d00 0x7f61e0005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415dca0 con 0x7f61e0003f20 2023-10-11T06:11:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e000ba60 con 0x7f61e0004d00 2023-10-11T06:11:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e000bc20 con 0x7f61f406fd80 2023-10-11T06:11:29.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e0003f20 0x7f61e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 0x7f61e00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61f37fe640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e0004d00 0x7f61e0005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61ebfff640 1 -- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 msgr2=0x7f61e00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 0x7f61e00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61ebfff640 1 -- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e0004d00 msgr2=0x7f61e0005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e0004d00 0x7f61e0005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61ebfff640 1 -- 172.21.15.84:0/1071457005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0003520 con 0x7f61e0003f20 2023-10-11T06:11:29.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.818+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61f406fd80 0x7f61e00039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.819+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e0003f20 0x7f61e00047c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61f40ceed0 tx=0x7f61e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fce85a1c016a43c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.819+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0052d0 con 0x7f61e0003f20 2023-10-11T06:11:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.819+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc02e050 con 0x7f61e0003f20 2023-10-11T06:11:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.819+0000 7f61f27fc640 1 -- 172.21.15.84:0/1071457005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc003830 con 0x7f61e0003f20 2023-10-11T06:11:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.820+0000 7f61f3fff640 1 -- 172.21.15.84:0/1071457005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e0003f20 msgr2=0x7f61e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.820+0000 7f61f3fff640 1 --2- 172.21.15.84:0/1071457005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e0003f20 0x7f61e00047c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f61f40ceed0 tx=0x7f61e00057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.820+0000 7f61f37fe640 1 -- 172.21.15.84:0/1071457005 reap_dead start 2023-10-11T06:11:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.820+0000 7f61f3fff640 1 -- 172.21.15.84:0/1071457005 shutdown_connections 2023-10-11T06:11:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.820+0000 7f61f3fff640 1 -- 172.21.15.84:0/1071457005 >> 172.21.15.84:0/1071457005 conn(0x7f61f4069c70 msgr2=0x7f61f406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.820+0000 7f61f3fff640 1 -- 172.21.15.84:0/1071457005 shutdown_connections 2023-10-11T06:11:29.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.820+0000 7f61f3fff640 1 -- 172.21.15.84:0/1071457005 wait complete. 2023-10-11T06:11:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.822+0000 7f61f3fff640 1 Processor -- start 2023-10-11T06:11:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.822+0000 7f61f3fff640 1 -- start start 2023-10-11T06:11:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61e0003f20 0x7f61f41f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f40c4910 0x7f61f415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f415a780 0x7f61f41f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61f3fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415abf0 con 0x7f61f40c4910 2023-10-11T06:11:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61f3fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f41f2640 con 0x7f61f415a780 2023-10-11T06:11:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61f3fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40b1650 con 0x7f61e0003f20 2023-10-11T06:11:29.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f40c4910 0x7f61f415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f40c4910 0x7f61f415a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35342/0 (socket says 172.21.15.84:35342) 2023-10-11T06:11:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61ebfff640 1 -- 172.21.15.84:0/2034780268 learned_addr learned my addr 172.21.15.84:0/2034780268 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.823+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61e0003f20 0x7f61f41f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61ebfff640 1 -- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61e0003f20 msgr2=0x7f61f41f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61ebfff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61e0003f20 0x7f61f41f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61ebfff640 1 -- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f415a780 msgr2=0x7f61f41f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61ebfff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f415a780 0x7f61f41f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61ebfff640 1 -- 172.21.15.84:0/2034780268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40b1930 con 0x7f61f40c4910 2023-10-11T06:11:29.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61e0003f20 0x7f61f41f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:29.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61ebfff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f40c4910 0x7f61f415a240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f61dc02bbb0 tx=0x7f61dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:29.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61f0ff9640 1 -- 172.21.15.84:0/2034780268 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc007a30 con 0x7f61f40c4910 2023-10-11T06:11:29.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61f0ff9640 1 -- 172.21.15.84:0/2034780268 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc007640 con 0x7f61f40c4910 2023-10-11T06:11:29.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f40b1c10 con 0x7f61f40c4910 2023-10-11T06:11:29.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61f0ff9640 1 -- 172.21.15.84:0/2034780268 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc006c00 con 0x7f61f40c4910 2023-10-11T06:11:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.824+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f40b21d0 con 0x7f61f40c4910 2023-10-11T06:11:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.827+0000 7f61f0ff9640 1 -- 172.21.15.84:0/2034780268 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f61dc0035f0 con 0x7f61f40c4910 2023-10-11T06:11:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.828+0000 7f61f0ff9640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f61d406a860 0x7f61d406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.828+0000 7f61f0ff9640 1 -- 172.21.15.84:0/2034780268 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61dc0aad10 con 0x7f61f40c4910 2023-10-11T06:11:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.828+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f61d406a860 0x7f61d406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.829+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f61d406a860 0x7f61d406cd10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001200 tx=0x7f61e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.829+0000 7f61fc45a740 1 Processor -- start 2023-10-11T06:11:29.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.829+0000 7f61fc45a740 1 -- start start 2023-10-11T06:11:29.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e08a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e0b36a0 0x55ff6e0ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e08aef0 0x55ff6e0bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61fc45a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e066250 con 0x55ff6e0b36a0 2023-10-11T06:11:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6dffaf80 con 0x55ff6e08aef0 2023-10-11T06:11:29.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6dff8710 con 0x55ff6e0b32d0 2023-10-11T06:11:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e0b36a0 0x55ff6e0ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e0b36a0 0x55ff6e0ba840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35350/0 (socket says 172.21.15.84:35350) 2023-10-11T06:11:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 -- 172.21.15.84:0/1779295784 learned_addr learned my addr 172.21.15.84:0/1779295784 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e08a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 -- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 msgr2=0x55ff6e08a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e08a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61f37fe640 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e08aef0 0x55ff6e0bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 -- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e08aef0 msgr2=0x55ff6e0bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e08aef0 0x55ff6e0bad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.830+0000 7f61ebfff640 1 -- 172.21.15.84:0/1779295784 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6dfbb690 con 0x55ff6e0b36a0 2023-10-11T06:11:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.831+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e0b36a0 0x55ff6e0ba840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61dc076140 tx=0x7f61dc07a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e53e21c1112015 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.831+0000 7f61e97fa640 1 -- 172.21.15.84:0/1779295784 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc081c20 con 0x55ff6e0b36a0 2023-10-11T06:11:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.831+0000 7f61e97fa640 1 -- 172.21.15.84:0/1779295784 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc091170 con 0x55ff6e0b36a0 2023-10-11T06:11:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.831+0000 7f61e97fa640 1 -- 172.21.15.84:0/1779295784 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc001370 con 0x55ff6e0b36a0 2023-10-11T06:11:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.831+0000 7f61fc45a740 1 -- 172.21.15.84:0/1779295784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e0b36a0 msgr2=0x55ff6e0ba840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.831+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e0b36a0 0x55ff6e0ba840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61dc076140 tx=0x7f61dc07a340 comp rx=0 tx=0).stop 2023-10-11T06:11:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 -- 172.21.15.84:0/1779295784 shutdown_connections 2023-10-11T06:11:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e08aef0 0x55ff6e0bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e0b36a0 0x55ff6e0ba840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1779295784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e08a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 -- 172.21.15.84:0/1779295784 >> 172.21.15.84:0/1779295784 conn(0x55ff6e08ccc0 msgr2=0x55ff6e0d5840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 -- 172.21.15.84:0/1779295784 shutdown_connections 2023-10-11T06:11:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 -- 172.21.15.84:0/1779295784 wait complete. 2023-10-11T06:11:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.832+0000 7f61fc45a740 1 -- start start 2023-10-11T06:11:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.833+0000 7f61fc45a740 1 Processor -- start 2023-10-11T06:11:29.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.833+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e090d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 0x55ff6e08f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61fc45a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6dff4000 con 0x55ff6e08aef0 2023-10-11T06:11:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6df957d0 con 0x55ff6e0b36a0 2023-10-11T06:11:29.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6df95470 con 0x55ff6e0b32d0 2023-10-11T06:11:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61fc45a740 10 client.0 _ll_register_callbacks cb 0x55ff6dfe41b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T06:11:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e090d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e090d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35352/0 (socket says 172.21.15.84:35352) 2023-10-11T06:11:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 -- 172.21.15.84:0/2261573856 learned_addr learned my addr 172.21.15.84:0/2261573856 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61ebfff640 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 -- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 msgr2=0x55ff6e0d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 -- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 msgr2=0x55ff6e08f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 0x55ff6e08f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 -- 172.21.15.84:0/2261573856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f40b21d0 con 0x55ff6e08aef0 2023-10-11T06:11:29.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61ebfff640 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:29.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e090d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f61e400eac0 tx=0x7f61e4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08c8c26145b4afb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:29.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.834+0000 7f61d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-11T06:11:29.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61d27fc640 1 -- 172.21.15.84:0/2261573856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e400c070 con 0x55ff6e08aef0 2023-10-11T06:11:29.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61d27fc640 1 -- 172.21.15.84:0/2261573856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61e4021be0 con 0x55ff6e08aef0 2023-10-11T06:11:29.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff6e01e550 con 0x55ff6e08aef0 2023-10-11T06:11:29.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61d27fc640 1 -- 172.21.15.84:0/2261573856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61e4021da0 con 0x55ff6e08aef0 2023-10-11T06:11:29.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 10 client.4704 fetch_fsmap learned FSMap version 8 2023-10-11T06:11:29.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55ff6dff2680 con 0x55ff6e08aef0 2023-10-11T06:11:29.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61d27fc640 1 -- 172.21.15.84:0/2261573856 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff6e01e550 con 0x55ff6e08aef0 2023-10-11T06:11:29.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61d27fc640 1 -- 172.21.15.84:0/2261573856 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f61e4021420 con 0x55ff6e08aef0 2023-10-11T06:11:29.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 10 client.4704 fetch_fsmap finished waiting for FSMap version 8 2023-10-11T06:11:29.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 10 client.4704 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-11T06:11:29.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 20 client.4704 populate_metadata read hostname 'smithi084' 2023-10-11T06:11:29.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e0aef40 0x55ff6e0f2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 4 client.4704 mds_command: new command op to 4314 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T06:11:29.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.835+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 --> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff6dfe6b50 con 0x55ff6e0aef40 2023-10-11T06:11:29.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.836+0000 7f61ebfff640 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e0aef40 0x55ff6e0f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.837+0000 7f61ebfff640 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e0aef40 0x55ff6e0f2910 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37831c8aa3821d6c server_cookie=66329c8b70be53e6 in_seq=0 out_seq=0 2023-10-11T06:11:29.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.837+0000 7f61d27fc640 10 client.4704 ms_handle_connect on v2:172.21.15.99:6833/2350376542 2023-10-11T06:11:29.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61d27fc640 1 -- 172.21.15.84:0/2261573856 <== mds.0 v2:172.21.15.99:6833/2350376542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ff6dfe6b50 con 0x55ff6e0aef40 2023-10-11T06:11:29.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61d27fc640 10 client.4704 handle_command_reply: tid=0 2023-10-11T06:11:29.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 client.4704 shutdown 2023-10-11T06:11:29.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 msgr2=0x55ff6e090d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e090d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f61e400eac0 tx=0x7f61e4001a00 comp rx=0 tx=0).stop 2023-10-11T06:11:29.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 shutdown_connections 2023-10-11T06:11:29.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e0aef40 0x55ff6e0f2910 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 0x55ff6e08f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 --2- 172.21.15.84:0/2261573856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e090d80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61d27fc640 0 client.4704 ms_handle_reset on v2:172.21.15.99:6833/2350376542 2023-10-11T06:11:29.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 >> 172.21.15.84:0/2261573856 conn(0x55ff6e08ccc0 msgr2=0x55ff6e0d9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:29.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.839+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 shutdown_connections 2023-10-11T06:11:29.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.840+0000 7f61fc45a740 1 -- 172.21.15.84:0/2261573856 wait complete. 2023-10-11T06:11:29.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.840+0000 7f61fc45a740 20 client.4704 trim_cache size 0 max 16384 2023-10-11T06:11:29.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.856+0000 7f61fc45a740 1 Processor -- start 2023-10-11T06:11:29.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.856+0000 7f61fc45a740 1 -- start start 2023-10-11T06:11:29.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0f6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 0x55ff6e08e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61fc45a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e0b0170 con 0x55ff6e08aef0 2023-10-11T06:11:29.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6dd5bf20 con 0x55ff6e0b36a0 2023-10-11T06:11:29.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e0bae20 con 0x55ff6e0b32d0 2023-10-11T06:11:29.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e0d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35368/0 (socket says 172.21.15.84:35368) 2023-10-11T06:11:29.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.857+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1807121299 learned_addr learned my addr 172.21.15.84:0/1807121299 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:29.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1807121299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 msgr2=0x55ff6e0f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1807121299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1807121299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 msgr2=0x55ff6e08e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1807121299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 0x55ff6e08e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1807121299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6df949e0 con 0x55ff6e08aef0 2023-10-11T06:11:29.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1807121299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e0d4970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e660 tx=0x7f61e401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84552df057236aa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:29.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61e9ffb640 1 -- 172.21.15.84:0/1807121299 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40218e0 con 0x55ff6e08aef0 2023-10-11T06:11:29.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61e9ffb640 1 -- 172.21.15.84:0/1807121299 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e401b050 con 0x55ff6e08aef0 2023-10-11T06:11:29.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.858+0000 7f61e9ffb640 1 -- 172.21.15.84:0/1807121299 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4021be0 con 0x55ff6e08aef0 2023-10-11T06:11:29.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 -- 172.21.15.84:0/1807121299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 msgr2=0x55ff6e0d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1807121299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e0d4970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e660 tx=0x7f61e401e000 comp rx=0 tx=0).stop 2023-10-11T06:11:29.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 -- 172.21.15.84:0/1807121299 shutdown_connections 2023-10-11T06:11:29.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1807121299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b36a0 0x55ff6e08e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1807121299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b32d0 0x55ff6e0f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1807121299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e0d4970 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 -- 172.21.15.84:0/1807121299 >> 172.21.15.84:0/1807121299 conn(0x55ff6e08ccc0 msgr2=0x55ff6e08f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:29.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 -- 172.21.15.84:0/1807121299 shutdown_connections 2023-10-11T06:11:29.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.859+0000 7f61fc45a740 1 -- 172.21.15.84:0/1807121299 wait complete. 2023-10-11T06:11:29.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.860+0000 7f61fc45a740 1 -- start start 2023-10-11T06:11:29.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.860+0000 7f61fc45a740 1 Processor -- start 2023-10-11T06:11:29.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e13a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b32d0 0x55ff6e08e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61fc45a740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b36a0 0x55ff6e08eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61fc45a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e0d3e10 con 0x55ff6e08aef0 2023-10-11T06:11:29.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e13b000 con 0x55ff6e0b32d0 2023-10-11T06:11:29.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61fc45a740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6e0ba8c0 con 0x55ff6e0b36a0 2023-10-11T06:11:29.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61fc45a740 10 client.0 _ll_register_callbacks cb 0x7f61dc075f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T06:11:29.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e13a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e13a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35374/0 (socket says 172.21.15.84:35374) 2023-10-11T06:11:29.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1713536191 learned_addr learned my addr 172.21.15.84:0/1713536191 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:29.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b36a0 msgr2=0x55ff6e08eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b36a0 0x55ff6e08eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b32d0 msgr2=0x55ff6e08e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:29.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b32d0 0x55ff6e08e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.861+0000 7f61f2ffd640 1 -- 172.21.15.84:0/1713536191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff6dff2680 con 0x55ff6e08aef0 2023-10-11T06:11:29.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61f2ffd640 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e13a350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001910 tx=0x7f61e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5bf973973e4164 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:29.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-11T06:11:29.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61d27fc640 1 -- 172.21.15.84:0/1713536191 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e400c040 con 0x55ff6e08aef0 2023-10-11T06:11:29.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61d27fc640 1 -- 172.21.15.84:0/1713536191 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61e4040c50 con 0x55ff6e08aef0 2023-10-11T06:11:29.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61dc091170 con 0x55ff6e08aef0 2023-10-11T06:11:29.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61d27fc640 1 -- 172.21.15.84:0/1713536191 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61e4040e10 con 0x55ff6e08aef0 2023-10-11T06:11:29.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61fc45a740 10 client.4710 fetch_fsmap learned FSMap version 8 2023-10-11T06:11:29.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.862+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55ff6dfb9ef0 con 0x55ff6e08aef0 2023-10-11T06:11:29.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61d27fc640 1 -- 172.21.15.84:0/1713536191 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc091170 con 0x55ff6e08aef0 2023-10-11T06:11:29.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61d27fc640 1 -- 172.21.15.84:0/1713536191 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f61e4013480 con 0x55ff6e08aef0 2023-10-11T06:11:29.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61fc45a740 10 client.4710 fetch_fsmap finished waiting for FSMap version 8 2023-10-11T06:11:29.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61fc45a740 10 client.4710 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-11T06:11:29.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61fc45a740 20 client.4710 populate_metadata read hostname 'smithi084' 2023-10-11T06:11:29.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e13e300 0x55ff6e13e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:29.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61fc45a740 4 client.4710 mds_command: new command op to 4314 tid=0[{"prefix": "status"}] 2023-10-11T06:11:29.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 --> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ff6dfed080 con 0x55ff6e13e300 2023-10-11T06:11:29.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.863+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e13e300 0x55ff6e13e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:29.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.864+0000 7f61d27fc640 10 client.4710 ms_handle_connect on v2:172.21.15.99:6833/2350376542 2023-10-11T06:11:29.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.864+0000 7f61ebfff640 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e13e300 0x55ff6e13e6d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7574078a539573b server_cookie=f93d972490292fd8 in_seq=0 out_seq=0 2023-10-11T06:11:29.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61d27fc640 1 -- 172.21.15.84:0/1713536191 <== mds.0 v2:172.21.15.99:6833/2350376542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55ff6dfed080 con 0x55ff6e13e300 2023-10-11T06:11:29.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61d27fc640 10 client.4710 handle_command_reply: tid=0 2023-10-11T06:11:29.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 client.4710 shutdown 2023-10-11T06:11:29.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 msgr2=0x55ff6e13a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e13a350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001910 tx=0x7f61e4012000 comp rx=0 tx=0).stop 2023-10-11T06:11:29.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 shutdown_connections 2023-10-11T06:11:29.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:6833/2350376542,v1:172.21.15.99:6835/2350376542] conn(0x55ff6e13e300 0x55ff6e13e6d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ff6e0b36a0 0x55ff6e08eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff6e0b32d0 0x55ff6e08e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 --2- 172.21.15.84:0/1713536191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff6e08aef0 0x55ff6e13a350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61d27fc640 0 client.4710 ms_handle_reset on v2:172.21.15.99:6833/2350376542 2023-10-11T06:11:29.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 >> 172.21.15.84:0/1713536191 conn(0x55ff6e08ccc0 msgr2=0x55ff6e089c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:29.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 shutdown_connections 2023-10-11T06:11:29.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 1 -- 172.21.15.84:0/1713536191 wait complete. 2023-10-11T06:11:29.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.865+0000 7f61fc45a740 20 client.4710 trim_cache size 0 max 16384 2023-10-11T06:11:29.890 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:11:29.890 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c84dc637-9184-44da-bf67-6cb4224adc5a", 2023-10-11T06:11:29.890 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-11T06:11:29.890 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4314, 2023-10-11T06:11:29.891 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-11T06:11:29.891 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-11T06:11:29.891 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-11T06:11:29.891 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 16.193527193000001, 2023-10-11T06:11:29.891 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 8, 2023-10-11T06:11:29.892 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 24, 2023-10-11T06:11:29.892 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 24, 2023-10-11T06:11:29.892 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 21.528209889999999 2023-10-11T06:11:29.892 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:11:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f61d406a860 msgr2=0x7f61d406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f61d406a860 0x7f61d406cd10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001200 tx=0x7f61e4009040 comp rx=0 tx=0).stop 2023-10-11T06:11:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f40c4910 msgr2=0x7f61f415a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f40c4910 0x7f61f415a240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f61dc02bbb0 tx=0x7f61dc001120 comp rx=0 tx=0).stop 2023-10-11T06:11:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 shutdown_connections 2023-10-11T06:11:29.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f61d406a860 0x7f61d406cd10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f415a780 0x7f61f41f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61f40c4910 0x7f61f415a240 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 --2- 172.21.15.84:0/2034780268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61e0003f20 0x7f61f41f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:29.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 >> 172.21.15.84:0/2034780268 conn(0x7f61f4069c70 msgr2=0x7f61f406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 shutdown_connections 2023-10-11T06:11:29.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:29.866+0000 7f61f3fff640 1 -- 172.21.15.84:0/2034780268 wait complete. 2023-10-11T06:11:29.896 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-11T06:11:29.919 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-11T06:11:29.920 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-11T06:11:29.920 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-11T06:11:29.920 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T06:11:29.921 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:11:29.921 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-11T06:11:29.922 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T06:11:29.922 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-11T06:11:29.922 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-11T06:11:29.922 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T06:11:29.923 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-11T06:11:29.923 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-11T06:11:29.923 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-11T06:11:29.923 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-11T06:11:29.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:29.924 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-11T06:11:29.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:29.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-10-11T06:11:30.088 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2023-10-11T06:11:30.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:11:30.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:30.093 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:11:30.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:30.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-10-11T06:11:30.256 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2023-10-11T06:11:30.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:11:30.260 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-11T06:11:30.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:11:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.470+0000 7fe63b22d640 1 Processor -- start 2023-10-11T06:11:30.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.471+0000 7fe63b22d640 1 -- start start 2023-10-11T06:11:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.472+0000 7fe63b22d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6340ca300 0x7fe6340cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.472+0000 7fe63b22d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe634071a90 con 0x7fe6340c9360 2023-10-11T06:11:30.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.472+0000 7fe63b22d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe634070280 con 0x7fe63415d4f0 2023-10-11T06:11:30.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.472+0000 7fe63b22d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe634072310 con 0x7fe6340ca300 2023-10-11T06:11:30.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.472+0000 7fe6397ca640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe6340c9360 0x7fe6340ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43534/0 (socket says 172.21.15.84:43534) 2023-10-11T06:11:30.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.472+0000 7fe6397ca640 1 -- 172.21.15.84:0/1269841072 learned_addr learned my addr 172.21.15.84:0/1269841072 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe633fff640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6340ca300 0x7fe6340cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204581327 0 0) 0x7fe634071a90 con 0x7fe6340c9360 2023-10-11T06:11:30.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe624003520 con 0x7fe6340c9360 2023-10-11T06:11:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2229254529 0 0) 0x7fe624003520 con 0x7fe6340c9360 2023-10-11T06:11:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6340ca300 msgr2=0x7fe6340cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6340ca300 0x7fe6340cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 >> v1:172.21.15.99:6789/0 conn(0x7fe63415d4f0 legacy=0x7fe6340c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe634071cc0 con 0x7fe6340c9360 2023-10-11T06:11:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fe62c0019f0 con 0x7fe6340c9360 2023-10-11T06:11:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 >> v1:172.21.15.84:6789/0 conn(0x7fe6340c9360 legacy=0x7fe6340ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6240039a0 0x7fe624003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6340ca300 0x7fe6240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe624005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe634072310 con 0x7fe624004d00 2023-10-11T06:11:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62400ba60 con 0x7fe6240039a0 2023-10-11T06:11:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.473+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62400bc20 con 0x7fe6340ca300 2023-10-11T06:11:30.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe6397ca640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe624005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe638fc9640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6240039a0 0x7fe624003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe6397ca640 1 -- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6340ca300 msgr2=0x7fe6240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe6397ca640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6340ca300 0x7fe6240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe6397ca640 1 -- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6240039a0 msgr2=0x7fe624003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe6397ca640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6240039a0 0x7fe624003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe6397ca640 1 -- 172.21.15.84:0/1269841072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624003520 con 0x7fe624004d00 2023-10-11T06:11:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.474+0000 7fe638fc9640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6240039a0 0x7fe624003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.475+0000 7fe6397ca640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe624005170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00a760 tx=0x7fe6240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85f53d3f376b2e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.475+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c034630 con 0x7fe624004d00 2023-10-11T06:11:30.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.475+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c02f070 con 0x7fe624004d00 2023-10-11T06:11:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.475+0000 7fe6337fe640 1 -- 172.21.15.84:0/1269841072 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c034a10 con 0x7fe624004d00 2023-10-11T06:11:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.476+0000 7fe63b22d640 1 -- 172.21.15.84:0/1269841072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 msgr2=0x7fe624005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.476+0000 7fe63b22d640 1 --2- 172.21.15.84:0/1269841072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe624005170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00a760 tx=0x7fe6240057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.476+0000 7fe6397ca640 1 -- 172.21.15.84:0/1269841072 reap_dead start 2023-10-11T06:11:30.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.476+0000 7fe63b22d640 1 -- 172.21.15.84:0/1269841072 shutdown_connections 2023-10-11T06:11:30.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.476+0000 7fe63b22d640 1 -- 172.21.15.84:0/1269841072 >> 172.21.15.84:0/1269841072 conn(0x7fe634069c70 msgr2=0x7fe63406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:30.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.476+0000 7fe63b22d640 1 -- 172.21.15.84:0/1269841072 shutdown_connections 2023-10-11T06:11:30.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.476+0000 7fe63b22d640 1 -- 172.21.15.84:0/1269841072 wait complete. 2023-10-11T06:11:30.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.478+0000 7fe63b22d640 1 Processor -- start 2023-10-11T06:11:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.478+0000 7fe63b22d640 1 -- start start 2023-10-11T06:11:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe63b22d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe634150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe63b22d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6341577d0 0x7fe6340c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe63b22d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6340cc800 0x7fe6340ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe63b22d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340c5a00 con 0x7fe624004d00 2023-10-11T06:11:30.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe63b22d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340cd1b0 con 0x7fe6340cc800 2023-10-11T06:11:30.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe63b22d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340c6890 con 0x7fe6341577d0 2023-10-11T06:11:30.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe634150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:30.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe634150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35384/0 (socket says 172.21.15.84:35384) 2023-10-11T06:11:30.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 -- 172.21.15.84:0/167998549 learned_addr learned my addr 172.21.15.84:0/167998549 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:30.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 -- 172.21.15.84:0/167998549 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6341577d0 msgr2=0x7fe6340c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:30.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6341577d0 0x7fe6340c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 -- 172.21.15.84:0/167998549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6340cc800 msgr2=0x7fe6340ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:30.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6340cc800 0x7fe6340ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.479+0000 7fe638fc9640 1 -- 172.21.15.84:0/167998549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6340b2f50 con 0x7fe624004d00 2023-10-11T06:11:30.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.480+0000 7fe638fc9640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe634150ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe62800d7a0 tx=0x7fe62800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:30.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.480+0000 7fe6317fa640 1 -- 172.21.15.84:0/167998549 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628005270 con 0x7fe624004d00 2023-10-11T06:11:30.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.480+0000 7fe6317fa640 1 -- 172.21.15.84:0/167998549 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6280053d0 con 0x7fe624004d00 2023-10-11T06:11:30.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.480+0000 7fe6317fa640 1 -- 172.21.15.84:0/167998549 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628005540 con 0x7fe624004d00 2023-10-11T06:11:30.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.480+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6340c65f0 con 0x7fe624004d00 2023-10-11T06:11:30.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.480+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6340b2900 con 0x7fe624004d00 2023-10-11T06:11:30.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.481+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe634073ed0 con 0x7fe624004d00 2023-10-11T06:11:30.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.482+0000 7fe6317fa640 1 -- 172.21.15.84:0/167998549 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe628006a60 con 0x7fe624004d00 2023-10-11T06:11:30.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.483+0000 7fe6317fa640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe61806a830 0x7fe61806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:30.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.483+0000 7fe6317fa640 1 -- 172.21.15.84:0/167998549 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6280884d0 con 0x7fe624004d00 2023-10-11T06:11:30.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.483+0000 7fe633fff640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe61806a830 0x7fe61806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:30.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.484+0000 7fe633fff640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe61806a830 0x7fe61806cce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe634157f40 tx=0x7fe620009500 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:30.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.487+0000 7fe6317fa640 1 -- 172.21.15.84:0/167998549 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe628053890 con 0x7fe624004d00 2023-10-11T06:11:30.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.715+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe6340b09b0 con 0x7fe624004d00 2023-10-11T06:11:30.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.716+0000 7fe6317fa640 1 -- 172.21.15.84:0/167998549 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe628056eb0 con 0x7fe624004d00 2023-10-11T06:11:30.717 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:11:30.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe61806a830 msgr2=0x7fe61806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:30.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe61806a830 0x7fe61806cce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe634157f40 tx=0x7fe620009500 comp rx=0 tx=0).stop 2023-10-11T06:11:30.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 msgr2=0x7fe634150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:30.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe634150ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe62800d7a0 tx=0x7fe62800dc60 comp rx=0 tx=0).stop 2023-10-11T06:11:30.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 shutdown_connections 2023-10-11T06:11:30.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe61806a830 0x7fe61806cce0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6340cc800 0x7fe6340ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe6341577d0 0x7fe6340c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 --2- 172.21.15.84:0/167998549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe624004d00 0x7fe634150ac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:30.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.718+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 >> 172.21.15.84:0/167998549 conn(0x7fe634069c70 msgr2=0x7fe6340c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:30.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.719+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 shutdown_connections 2023-10-11T06:11:30.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:30.719+0000 7fe63b22d640 1 -- 172.21.15.84:0/167998549 wait complete. 2023-10-11T06:11:30.738 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:11:30.738 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:11:30.739 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:11:30.739 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T06:11:30.739 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:11:30.739 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:11:30.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:11:30.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:11:30.740 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:11:30.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:30.741 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:11:30.793 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:30.793 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:30.793 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:30.794 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:30.794 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:30.794 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:30.794 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:30.794 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:30.795 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:11:30.795 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:30.795 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:30.795 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:11:30.796 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:30.796 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:30.796 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:11:30.796 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:30.796 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:11:30.797 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42386sec preferred_lft 42386sec 2023-10-11T06:11:30.797 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:11:30.797 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:30.798 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-11T06:11:30.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:30.799 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:30.799 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:30.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add name ceph-brx type bridge 2023-10-11T06:11:30.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr flush dev ceph-brx 2023-10-11T06:11:30.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set ceph-brx up 2023-10-11T06:11:30.801 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-11T06:11:30.801 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:31.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:31.287 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:11:31.367 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:11:31.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:31.371 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:11:31.444 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:11:31.444 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:11:31.445 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:11:31.445 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:11:31.445 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:11:31.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:31.446 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:31.446 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:31.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:11:31.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:11:31.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:11:31.447 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:32.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:32.229 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:11:32.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:32.279 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2023-10-11T06:11:32.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:32.322 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:32.323 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:32.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:11:32.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-11T06:11:32.323 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:32.500 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-11T06:11:32.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:32.501 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:32.501 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:32.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-11T06:11:32.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-11T06:11:32.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-11T06:11:32.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-11T06:11:32.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-11T06:11:32.503 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:32.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:32.767 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:32.767 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:32.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-11T06:11:32.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.0 master ceph-brx 2023-10-11T06:11:32.768 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:32.925 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T06:11:32.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:32.925 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T06:11:32.968 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:11:32.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:32.969 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:11:33.058 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:11:33.183 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:33.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:33.253 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:33.256 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:33.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:33.257 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:11:33.303 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:11:33.303 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T06:11:33.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:11:33.429 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:33.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:33.501 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:33.503 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:33.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:33.505 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:11:33.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-10-11T06:11:33.581+0000 7f3153436480 -1 init, newargv = 0x55da737b3dc0 newargc=15 2023-10-11T06:11:33.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[95803]: starting ceph client 2023-10-11T06:11:33.596 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[95803]: starting fuse 2023-10-11T06:11:34.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:11:34.759 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:34.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:34.854 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:34.858 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:34.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:34.859 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:11:34.913 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:11:34.914 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-10-11T06:11:34.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:11:34.914 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:11:34.915 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:11:34.915 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:11:34.915 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:11:34.916 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:34.916 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:11:34.916 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T06:11:34.917 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:11:34.917 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T06:11:34.917 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:34.917 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:11:34.918 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:11:34.918 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:11:34.918 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:11:34.919 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:34.919 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:11:34.920 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:11:34.920 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:11:34.920 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:11:34.921 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:11:34.921 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:11:34.921 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:11:34.922 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:11:34.922 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:34.922 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-10-11T06:11:34.923 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:11:35.103 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.95803.asok 2023-10-11T06:11:35.284 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95803.asok 2023-10-11T06:11:35.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:35.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95803.asok status 2023-10-11T06:11:35.525 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:11:35.526 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:11:35.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:11:35.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:11:35.527 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-10-11T06:11:35.527 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:11:35.527 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T06:11:35.528 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "95803", 2023-10-11T06:11:35.528 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:11:35.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:35.529 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:11:35.529 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:11:35.529 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4722, 2023-10-11T06:11:35.530 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:11:35.530 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:11:35.530 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:11:35.531 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 4722 2023-10-11T06:11:35.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:35.531 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:11:35.531 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:11:35.531 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:11:35.531 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3144582178 2023-10-11T06:11:35.532 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:11:35.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:35.532 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:11:35.532 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:11:35.532 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:11:35.533 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3144582178 2023-10-11T06:11:35.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:35.533 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.4722 192.168.0.1:0/3144582178", 2023-10-11T06:11:35.533 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/3144582178", 2023-10-11T06:11:35.533 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:11:35.533 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 8, 2023-10-11T06:11:35.534 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 24, 2023-10-11T06:11:35.534 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:11:35.534 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:11:35.534 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:11:35.534 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:11:35.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:11:35.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.749+0000 7f3e53777640 1 Processor -- start 2023-10-11T06:11:35.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.749+0000 7f3e53777640 1 -- start start 2023-10-11T06:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e53777640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3e4c0ca300 0x7f3e4c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e53777640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e4c0c9360 0x7f3e4c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:35.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e53777640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c071a90 con 0x7f3e4c15d4f0 2023-10-11T06:11:35.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e53777640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c072310 con 0x7f3e4c0c9360 2023-10-11T06:11:35.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e53777640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c070280 con 0x7f3e4c0ca300 2023-10-11T06:11:35.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e51d14640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e4c0c9360 0x7f3e4c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:35.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e51d14640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e4c0c9360 0x7f3e4c0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35400/0 (socket says 172.21.15.84:35400) 2023-10-11T06:11:35.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.751+0000 7f3e51d14640 1 -- 172.21.15.84:0/2189858858 learned_addr learned my addr 172.21.15.84:0/2189858858 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:35.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.752+0000 7f3e50d12640 1 --2- 172.21.15.84:0/2189858858 >> v2:172.21.15.99:3300/0 conn(0x7f3e4c0ca300 0x7f3e4c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:35.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.752+0000 7f3e51d14640 1 -- 172.21.15.84:0/2189858858 >> v2:172.21.15.99:3300/0 conn(0x7f3e4c0ca300 msgr2=0x7f3e4c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:35.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.752+0000 7f3e51d14640 1 --2- 172.21.15.84:0/2189858858 >> v2:172.21.15.99:3300/0 conn(0x7f3e4c0ca300 0x7f3e4c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:35.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.752+0000 7f3e51d14640 1 -- 172.21.15.84:0/2189858858 >> v1:172.21.15.99:6789/0 conn(0x7f3e4c15d4f0 legacy=0x7f3e4c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:35.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.752+0000 7f3e51d14640 1 -- 172.21.15.84:0/2189858858 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c071cc0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.752+0000 7f3e50d12640 1 --2- 172.21.15.84:0/2189858858 >> v2:172.21.15.99:3300/0 conn(0x7f3e4c0ca300 0x7f3e4c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:35.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.753+0000 7f3e51d14640 1 --2- 172.21.15.84:0/2189858858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ce550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0b0980 tx=0x7f3e4402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b042251eceb888 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:35.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.753+0000 7f3e43fff640 1 -- 172.21.15.84:0/2189858858 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4402ca80 con 0x7f3e4c0c9360 2023-10-11T06:11:35.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.753+0000 7f3e43fff640 1 -- 172.21.15.84:0/2189858858 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4402cbe0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.753+0000 7f3e43fff640 1 -- 172.21.15.84:0/2189858858 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440343f0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 -- 172.21.15.84:0/2189858858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 msgr2=0x7f3e4c0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:35.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 --2- 172.21.15.84:0/2189858858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ce550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0b0980 tx=0x7f3e4402bee0 comp rx=0 tx=0).stop 2023-10-11T06:11:35.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 -- 172.21.15.84:0/2189858858 shutdown_connections 2023-10-11T06:11:35.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 --2- 172.21.15.84:0/2189858858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ce550 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:35.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 --2- 172.21.15.84:0/2189858858 >> v2:172.21.15.99:3300/0 conn(0x7f3e4c0ca300 0x7f3e4c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:35.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 -- 172.21.15.84:0/2189858858 >> 172.21.15.84:0/2189858858 conn(0x7f3e4c069c70 msgr2=0x7f3e4c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:35.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 -- 172.21.15.84:0/2189858858 shutdown_connections 2023-10-11T06:11:35.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.754+0000 7f3e53777640 1 -- 172.21.15.84:0/2189858858 wait complete. 2023-10-11T06:11:35.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.756+0000 7f3e53777640 1 Processor -- start 2023-10-11T06:11:35.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.756+0000 7f3e53777640 1 -- start start 2023-10-11T06:11:35.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.757+0000 7f3e53777640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:35.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e53777640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e4c0ca300 0x7f3e4c0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:35.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e53777640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e4c15d4f0 0x7f3e4c0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:35.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e53777640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0b21e0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e53777640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0c7b80 con 0x7f3e4c0ca300 2023-10-11T06:11:35.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e53777640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c14a380 con 0x7f3e4c15d4f0 2023-10-11T06:11:35.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:35.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35410/0 (socket says 172.21.15.84:35410) 2023-10-11T06:11:35.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 -- 172.21.15.84:0/1442035834 learned_addr learned my addr 172.21.15.84:0/1442035834 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:35.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51d14640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e4c15d4f0 0x7f3e4c0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:35.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e50d12640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e4c0ca300 0x7f3e4c0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:35.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 -- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e4c15d4f0 msgr2=0x7f3e4c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:35.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e4c15d4f0 0x7f3e4c0cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:35.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 -- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e4c0ca300 msgr2=0x7f3e4c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:35.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e4c0ca300 0x7f3e4c0b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:35.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51513640 1 -- 172.21.15.84:0/1442035834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44009b30 con 0x7f3e4c0c9360 2023-10-11T06:11:35.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e50d12640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e4c0ca300 0x7f3e4c0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:35.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.758+0000 7f3e51d14640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e4c15d4f0 0x7f3e4c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:35.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.759+0000 7f3e51513640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ca9e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e48004640 tx=0x7f3e4800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:35.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.759+0000 7f3e41ffb640 1 -- 172.21.15.84:0/1442035834 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48011790 con 0x7f3e4c0c9360 2023-10-11T06:11:35.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.759+0000 7f3e41ffb640 1 -- 172.21.15.84:0/1442035834 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e480118f0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.759+0000 7f3e41ffb640 1 -- 172.21.15.84:0/1442035834 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48011ce0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.759+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c0b15d0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.759+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c0b1b90 con 0x7f3e4c0c9360 2023-10-11T06:11:35.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.760+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c073ed0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.761+0000 7f3e41ffb640 1 -- 172.21.15.84:0/1442035834 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3e48012420 con 0x7f3e4c0c9360 2023-10-11T06:11:35.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.762+0000 7f3e41ffb640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3e3006a830 0x7f3e3006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:35.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.762+0000 7f3e41ffb640 1 -- 172.21.15.84:0/1442035834 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e48088c30 con 0x7f3e4c0c9360 2023-10-11T06:11:35.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.762+0000 7f3e50d12640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3e3006a830 0x7f3e3006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:35.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.763+0000 7f3e50d12640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3e3006a830 0x7f3e3006cce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c006fd0 tx=0x7f3e3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:35.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.767+0000 7f3e41ffb640 1 -- 172.21.15.84:0/1442035834 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3e48053ea0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.996+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3e4c0b09b0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.997+0000 7f3e41ffb640 1 -- 172.21.15.84:0/1442035834 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3e480574c0 con 0x7f3e4c0c9360 2023-10-11T06:11:35.998 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:11:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3e3006a830 msgr2=0x7f3e3006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3e3006a830 0x7f3e3006cce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c006fd0 tx=0x7f3e3c008040 comp rx=0 tx=0).stop 2023-10-11T06:11:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 msgr2=0x7f3e4c0ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ca9e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e48004640 tx=0x7f3e4800c8e0 comp rx=0 tx=0).stop 2023-10-11T06:11:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 shutdown_connections 2023-10-11T06:11:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3e3006a830 0x7f3e3006cce0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e4c15d4f0 0x7f3e4c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e4c0ca300 0x7f3e4c0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:35.999+0000 7f3e53777640 1 --2- 172.21.15.84:0/1442035834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e4c0c9360 0x7f3e4c0ca9e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:36.000+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 >> 172.21.15.84:0/1442035834 conn(0x7f3e4c069c70 msgr2=0x7f3e4c0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:36.000+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 shutdown_connections 2023-10-11T06:11:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:36.000+0000 7f3e53777640 1 -- 172.21.15.84:0/1442035834 wait complete. 2023-10-11T06:11:36.019 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:11:36.020 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:11:36.020 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:11:36.020 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-11T06:11:36.021 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-11T06:11:36.021 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:11:36.021 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:11:36.021 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:11:36.022 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-11T06:11:36.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.022 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:11:36.073 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:36.073 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:36.073 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:36.074 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.074 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:36.074 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.074 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:36.074 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:36.075 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:11:36.075 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:36.075 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:36.075 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:11:36.076 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:36.076 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:36.076 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:11:36.076 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:36.077 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:11:36.077 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42381sec preferred_lft 42381sec 2023-10-11T06:11:36.077 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:11:36.077 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.078 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:11:36.078 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:36.078 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:11:36.079 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.079 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1c33:d5ff:fea1:21f9/64 scope link 2023-10-11T06:11:36.079 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.079 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:11:36.080 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:11:36.080 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::d4ff:bcff:feac:c798/64 scope link 2023-10-11T06:11:36.080 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.081 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:11:36.170 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:11:36.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.174 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:11:36.228 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:11:36.229 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:11:36.229 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:11:36.229 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:11:36.229 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:11:36.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.231 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:36.231 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:36.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:11:36.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:11:36.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:11:36.232 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:36.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.421 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:11:36.470 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:11:36.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.472 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2023-10-11T06:11:36.523 INFO:teuthology.orchestra.run.smithi084.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-11T06:11:36.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.525 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:36.525 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:36.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:11:36.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-11T06:11:36.527 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:36.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.667 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-11T06:11:36.760 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:36.761 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:36.761 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:36.761 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.761 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:36.762 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.762 INFO:teuthology.orchestra.run.smithi084.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:11:36.762 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 16:52:42:94:2f:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T06:11:36.762 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T06:11:36.763 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.763 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1452:42ff:fe94:2fa9/64 scope link 2023-10-11T06:11:36.763 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-11T06:11:36.860 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:36.860 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:36.860 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:36.861 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.861 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:36.861 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.861 INFO:teuthology.orchestra.run.smithi084.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:11:36.861 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 16:52:42:94:2f:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T06:11:36.862 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T06:11:36.862 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.862 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1452:42ff:fe94:2fa9/64 scope link 2023-10-11T06:11:36.862 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:36.864 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-11T06:11:36.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:36.864 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:36.865 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:36.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-11T06:11:36.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-11T06:11:36.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-11T06:11:36.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-11T06:11:36.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-11T06:11:36.867 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:37.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:37.133 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:11:37.134 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:11:37.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.1 up 2023-10-11T06:11:37.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.1 master ceph-brx 2023-10-11T06:11:37.134 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:11:37.303 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-11T06:11:37.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:37.304 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-11T06:11:37.356 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-11T06:11:37.357 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:37.357 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-11T06:11:37.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:11:37.548 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:37.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:37.642 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:37.646 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:37.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:37.647 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:11:37.705 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:11:37.705 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-10-11T06:11:37.705 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-11T06:11:37.711 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:11:37.787 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:37.788 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:37.860 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:37.862 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:37.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:37.863 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:11:37.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi084.stderr:2023-10-11T06:11:37.899+0000 7f6486c3a480 -1 init, newargv = 0x555abd9898a0 newargc=15 2023-10-11T06:11:37.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi084.stderr:ceph-fuse[96350]: starting ceph client 2023-10-11T06:11:37.908 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:11:37.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi084.stderr:ceph-fuse[96350]: starting fuse 2023-10-11T06:11:39.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:11:39.115 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:39.115 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:39.210 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:39.214 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:39.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:39.215 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:11:39.267 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:11:39.267 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-10-11T06:11:39.268 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-10-11T06:11:39.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:11:39.268 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:11:39.269 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:11:39.269 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:11:39.269 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:11:39.270 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:39.270 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:11:39.270 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-11T06:11:39.270 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:11:39.271 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-11T06:11:39.271 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:39.271 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:11:39.272 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:11:39.272 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:11:39.272 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:11:39.272 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:39.273 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:11:39.273 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:11:39.273 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:11:39.273 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:11:39.274 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:11:39.274 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:11:39.274 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:11:39.274 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:11:39.275 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:11:39.275 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.2")) 2023-10-11T06:11:39.275 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:11:39.449 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.2.96350.asok 2023-10-11T06:11:39.630 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.96350.asok 2023-10-11T06:11:39.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:39.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.96350.asok status 2023-10-11T06:11:39.873 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:11:39.874 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:11:39.874 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:11:39.874 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:11:39.874 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "2", 2023-10-11T06:11:39.875 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:11:39.875 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-11T06:11:39.875 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "96350", 2023-10-11T06:11:39.875 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:11:39.875 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:39.876 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:11:39.876 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:11:39.876 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4734, 2023-10-11T06:11:39.876 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:11:39.876 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:11:39.877 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:11:39.877 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 4734 2023-10-11T06:11:39.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:39.877 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:11:39.877 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:11:39.877 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.2:0", 2023-10-11T06:11:39.878 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1949371710 2023-10-11T06:11:39.878 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:11:39.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:39.878 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:11:39.878 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:11:39.879 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.2:0", 2023-10-11T06:11:39.879 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1949371710 2023-10-11T06:11:39.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:11:39.879 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.4734 192.168.0.2:0/1949371710", 2023-10-11T06:11:39.879 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.2:0/1949371710", 2023-10-11T06:11:39.880 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:11:39.880 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 8, 2023-10-11T06:11:39.880 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 24, 2023-10-11T06:11:39.880 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:11:39.880 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:11:39.880 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:11:39.881 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:11:39.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:11:40.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.099+0000 7fa189ba4640 1 Processor -- start 2023-10-11T06:11:40.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.099+0000 7fa189ba4640 1 -- start start 2023-10-11T06:11:40.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.101+0000 7fa189ba4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa18415d4f0 0x7fa1840cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.101+0000 7fa189ba4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa184071a90 con 0x7fa1840ca300 2023-10-11T06:11:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.101+0000 7fa189ba4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa184070280 con 0x7fa1840c9360 2023-10-11T06:11:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.101+0000 7fa189ba4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa184072310 con 0x7fa18415d4f0 2023-10-11T06:11:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.101+0000 7fa1837fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa18415d4f0 0x7fa1840cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:40.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.101+0000 7fa1837fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa18415d4f0 0x7fa1840cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50460/0 (socket says 172.21.15.84:50460) 2023-10-11T06:11:40.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.101+0000 7fa1837fe640 1 -- 172.21.15.84:0/3427885827 learned_addr learned my addr 172.21.15.84:0/3427885827 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.102+0000 7fa1837fe640 1 -- 172.21.15.84:0/3427885827 >> v1:172.21.15.99:6789/0 conn(0x7fa1840c9360 legacy=0x7fa1840ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.102+0000 7fa1837fe640 1 -- 172.21.15.84:0/3427885827 >> v1:172.21.15.84:6789/0 conn(0x7fa1840ca300 legacy=0x7fa1840c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:40.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.102+0000 7fa1837fe640 1 -- 172.21.15.84:0/3427885827 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184071cc0 con 0x7fa18415d4f0 2023-10-11T06:11:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.103+0000 7fa1837fe640 1 --2- 172.21.15.84:0/3427885827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18415d4f0 0x7fa1840cb540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa174009850 tx=0x7fa17402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0413a56f3b87632 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.103+0000 7fa182ffd640 1 -- 172.21.15.84:0/3427885827 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174004230 con 0x7fa18415d4f0 2023-10-11T06:11:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.103+0000 7fa182ffd640 1 -- 172.21.15.84:0/3427885827 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174004390 con 0x7fa18415d4f0 2023-10-11T06:11:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.103+0000 7fa182ffd640 1 -- 172.21.15.84:0/3427885827 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174030840 con 0x7fa18415d4f0 2023-10-11T06:11:40.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.104+0000 7fa189ba4640 1 -- 172.21.15.84:0/3427885827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18415d4f0 msgr2=0x7fa1840cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.104+0000 7fa189ba4640 1 --2- 172.21.15.84:0/3427885827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18415d4f0 0x7fa1840cb540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa174009850 tx=0x7fa17402b980 comp rx=0 tx=0).stop 2023-10-11T06:11:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.104+0000 7fa189ba4640 1 -- 172.21.15.84:0/3427885827 shutdown_connections 2023-10-11T06:11:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.104+0000 7fa189ba4640 1 --2- 172.21.15.84:0/3427885827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa18415d4f0 0x7fa1840cb540 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.104+0000 7fa189ba4640 1 -- 172.21.15.84:0/3427885827 >> 172.21.15.84:0/3427885827 conn(0x7fa184069c70 msgr2=0x7fa18406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.104+0000 7fa189ba4640 1 -- 172.21.15.84:0/3427885827 shutdown_connections 2023-10-11T06:11:40.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.104+0000 7fa189ba4640 1 -- 172.21.15.84:0/3427885827 wait complete. 2023-10-11T06:11:40.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.106+0000 7fa189ba4640 1 Processor -- start 2023-10-11T06:11:40.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.106+0000 7fa189ba4640 1 -- start start 2023-10-11T06:11:40.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa189ba4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1840c9360 0x7fa1841493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:40.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa189ba4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1840ca300 0x7fa1840c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa189ba4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa18415d4f0 0x7fa18414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa189ba4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840c8d10 con 0x7fa1840c9360 2023-10-11T06:11:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa189ba4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840c8a30 con 0x7fa1840ca300 2023-10-11T06:11:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa189ba4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840b3030 con 0x7fa18415d4f0 2023-10-11T06:11:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1840c9360 0x7fa1841493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:40.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1840c9360 0x7fa1841493d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50476/0 (socket says 172.21.15.84:50476) 2023-10-11T06:11:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 -- 172.21.15.84:0/1055073675 learned_addr learned my addr 172.21.15.84:0/1055073675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa17bfff640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1840ca300 0x7fa1840c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 -- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa18415d4f0 msgr2=0x7fa18414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa18415d4f0 0x7fa18414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 -- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1840ca300 msgr2=0x7fa1840c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1840ca300 0x7fa1840c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa1837fe640 1 -- 172.21.15.84:0/1055073675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009460 con 0x7fa1840c9360 2023-10-11T06:11:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.107+0000 7fa17bfff640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1840ca300 0x7fa1840c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:40.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.108+0000 7fa1837fe640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1840c9360 0x7fa1841493d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa174000c00 tx=0x7fa1740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.108+0000 7fa180ff9640 1 -- 172.21.15.84:0/1055073675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17403e070 con 0x7fa1840c9360 2023-10-11T06:11:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.108+0000 7fa180ff9640 1 -- 172.21.15.84:0/1055073675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174004500 con 0x7fa1840c9360 2023-10-11T06:11:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.108+0000 7fa180ff9640 1 -- 172.21.15.84:0/1055073675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174030e40 con 0x7fa1840c9360 2023-10-11T06:11:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.108+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1840b3310 con 0x7fa1840c9360 2023-10-11T06:11:40.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.108+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa184149910 con 0x7fa1840c9360 2023-10-11T06:11:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.109+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1840b3830 con 0x7fa1840c9360 2023-10-11T06:11:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.110+0000 7fa180ff9640 1 -- 172.21.15.84:0/1055073675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa174003610 con 0x7fa1840c9360 2023-10-11T06:11:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.112+0000 7fa180ff9640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa16406a830 0x7fa16406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.112+0000 7fa180ff9640 1 -- 172.21.15.84:0/1055073675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa1740aaf30 con 0x7fa1840c9360 2023-10-11T06:11:40.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.112+0000 7fa17bfff640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa16406a830 0x7fa16406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:40.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.113+0000 7fa17bfff640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa16406a830 0x7fa16406cce0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa184159be0 tx=0x7fa16c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:40.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.117+0000 7fa180ff9640 1 -- 172.21.15.84:0/1055073675 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa1840b3830 con 0x7fa1840c9360 2023-10-11T06:11:40.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.351+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa1840b3830 con 0x7fa1840c9360 2023-10-11T06:11:40.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.352+0000 7fa180ff9640 1 -- 172.21.15.84:0/1055073675 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa174048060 con 0x7fa1840c9360 2023-10-11T06:11:40.354 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:11:40.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa16406a830 msgr2=0x7fa16406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:40.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa16406a830 0x7fa16406cce0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa184159be0 tx=0x7fa16c009040 comp rx=0 tx=0).stop 2023-10-11T06:11:40.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1840c9360 msgr2=0x7fa1841493d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:40.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1840c9360 0x7fa1841493d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa174000c00 tx=0x7fa1740046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 shutdown_connections 2023-10-11T06:11:40.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa16406a830 0x7fa16406cce0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa18415d4f0 0x7fa18414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1840ca300 0x7fa1840c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 --2- 172.21.15.84:0/1055073675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1840c9360 0x7fa1841493d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:40.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 >> 172.21.15.84:0/1055073675 conn(0x7fa184069c70 msgr2=0x7fa1840ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:40.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 shutdown_connections 2023-10-11T06:11:40.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:40.355+0000 7fa189ba4640 1 -- 172.21.15.84:0/1055073675 wait complete. 2023-10-11T06:11:40.374 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:11:40.374 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:11:40.375 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-11T06:11:40.375 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-11T06:11:40.375 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-11T06:11:40.375 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:11:40.376 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:11:40.376 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:11:40.376 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-11T06:11:40.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:40.377 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-11T06:11:40.435 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:40.435 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:40.435 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:40.436 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:40.436 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:40.436 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:40.436 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:40.436 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:40.437 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-11T06:11:40.437 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:40.437 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:40.437 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-11T06:11:40.438 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:40.438 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:40.438 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:11:40.438 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:40.439 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:11:40.439 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42394sec preferred_lft 42394sec 2023-10-11T06:11:40.439 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-11T06:11:40.439 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:40.440 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-11T06:11:40.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:40.441 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:40.441 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:40.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2023-10-11T06:11:40.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2023-10-11T06:11:40.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2023-10-11T06:11:40.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-11T06:11:40.443 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:40.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:40.898 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:11:40.982 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-11T06:11:40.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:40.986 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-11T06:11:41.044 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-11T06:11:41.045 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:11:41.045 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:11:41.045 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:11:41.045 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:11:41.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:41.046 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:41.046 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:41.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:11:41.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:11:41.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:11:41.047 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:41.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:41.821 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-11T06:11:41.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:41.876 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2023-10-11T06:11:41.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:41.930 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:41.930 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:41.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-11T06:11:41.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-11T06:11:41.931 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:42.070 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-11T06:11:42.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:42.071 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:42.071 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:42.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-11T06:11:42.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-11T06:11:42.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-11T06:11:42.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-11T06:11:42.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-11T06:11:42.075 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:42.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:42.355 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:42.355 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:42.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2023-10-11T06:11:42.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2023-10-11T06:11:42.356 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:42.522 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-11T06:11:42.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:42.523 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-11T06:11:42.573 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-11T06:11:42.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:42.574 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-11T06:11:42.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2023-10-11T06:11:42.717 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:42.718 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:42.808 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:42.811 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:42.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:42.812 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2023-10-11T06:11:42.862 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:11:42.862 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-11T06:11:42.911 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2023-10-11T06:11:42.993 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:42.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:43.066 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:43.069 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:43.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:43.069 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2023-10-11T06:11:43.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi099.stderr:2023-10-11T06:11:43.112+0000 7fdb9fa3a480 -1 init, newargv = 0x563f7d16c130 newargc=15 2023-10-11T06:11:43.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi099.stderr:ceph-fuse[93394]: starting ceph client 2023-10-11T06:11:43.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi099.stderr:ceph-fuse[93394]: starting fuse 2023-10-11T06:11:44.217 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2023-10-11T06:11:44.315 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:44.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:44.411 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:44.415 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:44.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:44.416 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2023-10-11T06:11:44.471 INFO:teuthology.orchestra.run.smithi099.stdout:39 2023-10-11T06:11:44.472 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-10-11T06:11:44.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:11:44.472 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2023-10-11T06:11:44.473 DEBUG:teuthology.orchestra.run.smithi099:> import re 2023-10-11T06:11:44.473 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-11T06:11:44.473 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2023-10-11T06:11:44.474 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:44.474 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2023-10-11T06:11:44.474 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-11T06:11:44.474 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2023-10-11T06:11:44.475 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-11T06:11:44.475 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:44.475 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2023-10-11T06:11:44.475 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2023-10-11T06:11:44.476 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2023-10-11T06:11:44.476 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2023-10-11T06:11:44.476 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:44.476 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2023-10-11T06:11:44.477 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:11:44.477 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:11:44.477 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:11:44.478 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2023-10-11T06:11:44.478 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2023-10-11T06:11:44.478 DEBUG:teuthology.orchestra.run.smithi099:> return f 2023-10-11T06:11:44.478 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:11:44.479 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:44.479 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.1")) 2023-10-11T06:11:44.479 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-11T06:11:44.657 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.1.93394.asok 2023-10-11T06:11:44.838 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.93394.asok 2023-10-11T06:11:44.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:44.839 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.93394.asok status 2023-10-11T06:11:45.079 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-11T06:11:45.080 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2023-10-11T06:11:45.080 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:11:45.080 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:11:45.080 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "1", 2023-10-11T06:11:45.081 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2023-10-11T06:11:45.081 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-11T06:11:45.081 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "93394", 2023-10-11T06:11:45.081 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2023-10-11T06:11:45.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:45.082 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2023-10-11T06:11:45.082 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2023-10-11T06:11:45.082 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4453, 2023-10-11T06:11:45.082 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2023-10-11T06:11:45.082 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2023-10-11T06:11:45.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2023-10-11T06:11:45.083 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4453 2023-10-11T06:11:45.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:45.083 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2023-10-11T06:11:45.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-11T06:11:45.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:11:45.084 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2215788772 2023-10-11T06:11:45.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-11T06:11:45.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:45.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2023-10-11T06:11:45.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-11T06:11:45.085 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:11:45.085 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2215788772 2023-10-11T06:11:45.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:45.085 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4453 192.168.0.1:0/2215788772", 2023-10-11T06:11:45.085 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2215788772", 2023-10-11T06:11:45.086 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2023-10-11T06:11:45.086 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 8, 2023-10-11T06:11:45.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 24, 2023-10-11T06:11:45.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:11:45.086 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2023-10-11T06:11:45.087 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2023-10-11T06:11:45.087 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-11T06:11:45.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:11:45.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.328+0000 7fa9b5b6e640 1 Processor -- start 2023-10-11T06:11:45.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.328+0000 7fa9b5b6e640 1 -- start start 2023-10-11T06:11:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.330+0000 7fa9b5b6e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.330+0000 7fa9b5b6e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9b00cc410 0x7fa9b00cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.330+0000 7fa9b5b6e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b00cb970 con 0x7fa9b015c290 2023-10-11T06:11:45.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.330+0000 7fa9b5b6e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b00cad70 con 0x7fa9b00cc410 2023-10-11T06:11:45.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.330+0000 7fa9b5b6e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b015fe40 con 0x7fa9b00cafb0 2023-10-11T06:11:45.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.330+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:45.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.330+0000 7fa9affff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9b00cc410 0x7fa9b00cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:45.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9affff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9b00cc410 0x7fa9b00cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:56746/0 (socket says 172.21.15.84:56746) 2023-10-11T06:11:45.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9af7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa9b015c290 0x7fa9b00c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:40908/0 (socket says 172.21.15.84:40908) 2023-10-11T06:11:45.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9affff640 1 -- 172.21.15.84:0/283326393 learned_addr learned my addr 172.21.15.84:0/283326393 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:45.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9ae7fc640 1 -- 172.21.15.84:0/283326393 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608606321 0 0) 0x7fa9b00cb970 con 0x7fa9b015c290 2023-10-11T06:11:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9aeffd640 1 -- 172.21.15.84:0/283326393 >> v2:172.21.15.99:3300/0 conn(0x7fa9b00cc410 msgr2=0x7fa9b00cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9aeffd640 1 --2- 172.21.15.84:0/283326393 >> v2:172.21.15.99:3300/0 conn(0x7fa9b00cc410 0x7fa9b00cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9aeffd640 1 -- 172.21.15.84:0/283326393 >> v1:172.21.15.99:6789/0 conn(0x7fa9b015c290 legacy=0x7fa9b00c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.331+0000 7fa9aeffd640 1 -- 172.21.15.84:0/283326393 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b006ff30 con 0x7fa9b00cafb0 2023-10-11T06:11:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.333+0000 7fa9aeffd640 1 --2- 172.21.15.84:0/283326393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00cb430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40097a0 tx=0x7fa9a402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f245c6c9bcfe6b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.333+0000 7fa9ae7fc640 1 -- 172.21.15.84:0/283326393 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4031830 con 0x7fa9b00cafb0 2023-10-11T06:11:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.333+0000 7fa9ae7fc640 1 -- 172.21.15.84:0/283326393 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a4031990 con 0x7fa9b00cafb0 2023-10-11T06:11:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9ae7fc640 1 -- 172.21.15.84:0/283326393 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4031d20 con 0x7fa9b00cafb0 2023-10-11T06:11:45.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/283326393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 msgr2=0x7fa9b00cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/283326393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00cb430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40097a0 tx=0x7fa9a402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/283326393 shutdown_connections 2023-10-11T06:11:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/283326393 >> v2:172.21.15.99:3300/0 conn(0x7fa9b00cc410 0x7fa9b00cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/283326393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00cb430 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/283326393 >> 172.21.15.84:0/283326393 conn(0x7fa9b0069c70 msgr2=0x7fa9b006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:45.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/283326393 shutdown_connections 2023-10-11T06:11:45.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.334+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/283326393 wait complete. 2023-10-11T06:11:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.336+0000 7fa9b5b6e640 1 Processor -- start 2023-10-11T06:11:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.336+0000 7fa9b5b6e640 1 -- start start 2023-10-11T06:11:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.337+0000 7fa9b5b6e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:45.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.337+0000 7fa9b5b6e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00cc410 0x7fa9b00ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.337+0000 7fa9b5b6e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b015c290 0x7fa9b01f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.337+0000 7fa9b5b6e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b0149390 con 0x7fa9b00cc410 2023-10-11T06:11:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.337+0000 7fa9b5b6e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b01490b0 con 0x7fa9b015c290 2023-10-11T06:11:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.337+0000 7fa9b5b6e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b01f5070 con 0x7fa9b00cafb0 2023-10-11T06:11:45.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.337+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00cc410 0x7fa9b00ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00cc410 0x7fa9b00ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50478/0 (socket says 172.21.15.84:50478) 2023-10-11T06:11:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 -- 172.21.15.84:0/1960799439 learned_addr learned my addr 172.21.15.84:0/1960799439 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9af7fe640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 -- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 msgr2=0x7fa9b00ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:45.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 -- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b015c290 msgr2=0x7fa9b01f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9affff640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b015c290 0x7fa9b01f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b015c290 0x7fa9b01f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 -- 172.21.15.84:0/1960799439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4009400 con 0x7fa9b00cc410 2023-10-11T06:11:45.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9af7fe640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9affff640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b015c290 0x7fa9b01f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.338+0000 7fa9aeffd640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00cc410 0x7fa9b00ae5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa9a402bee0 tx=0x7fa9a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.339+0000 7fa997fff640 1 -- 172.21.15.84:0/1960799439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a403b9f0 con 0x7fa9b00cc410 2023-10-11T06:11:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.339+0000 7fa997fff640 1 -- 172.21.15.84:0/1960799439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a403bb70 con 0x7fa9b00cc410 2023-10-11T06:11:45.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.339+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b01f5350 con 0x7fa9b00cc410 2023-10-11T06:11:45.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.339+0000 7fa997fff640 1 -- 172.21.15.84:0/1960799439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40453f0 con 0x7fa9b00cc410 2023-10-11T06:11:45.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.339+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b01f5b50 con 0x7fa9b00cc410 2023-10-11T06:11:45.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.340+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b01f4ed0 con 0x7fa9b00cc410 2023-10-11T06:11:45.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.341+0000 7fa997fff640 1 -- 172.21.15.84:0/1960799439 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa9a404e450 con 0x7fa9b00cc410 2023-10-11T06:11:45.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.342+0000 7fa997fff640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa99c06a830 0x7fa99c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:45.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.342+0000 7fa997fff640 1 -- 172.21.15.84:0/1960799439 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9a40ab1a0 con 0x7fa9b00cc410 2023-10-11T06:11:45.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.342+0000 7fa9af7fe640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa99c06a830 0x7fa99c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:45.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.343+0000 7fa9af7fe640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa99c06a830 0x7fa99c06cce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0007fd0 tx=0x7fa9a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:45.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.347+0000 7fa997fff640 1 -- 172.21.15.84:0/1960799439 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa9b01f4ed0 con 0x7fa9b00cc410 2023-10-11T06:11:45.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.576+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9b01f5870 con 0x7fa9b00cc410 2023-10-11T06:11:45.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.576+0000 7fa997fff640 1 -- 172.21.15.84:0/1960799439 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9b01f5870 con 0x7fa9b00cc410 2023-10-11T06:11:45.578 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:11:45.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa99c06a830 msgr2=0x7fa99c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:45.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa99c06a830 0x7fa99c06cce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0007fd0 tx=0x7fa9a0009040 comp rx=0 tx=0).stop 2023-10-11T06:11:45.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00cc410 msgr2=0x7fa9b00ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:45.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00cc410 0x7fa9b00ae5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa9a402bee0 tx=0x7fa9a40046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 shutdown_connections 2023-10-11T06:11:45.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa99c06a830 0x7fa99c06cce0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b015c290 0x7fa9b01f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9b00cc410 0x7fa9b00ae5a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 --2- 172.21.15.84:0/1960799439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b00cafb0 0x7fa9b00ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:45.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.579+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 >> 172.21.15.84:0/1960799439 conn(0x7fa9b0069c70 msgr2=0x7fa9b00cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:45.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.580+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 shutdown_connections 2023-10-11T06:11:45.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:45.580+0000 7fa9b5b6e640 1 -- 172.21.15.84:0/1960799439 wait complete. 2023-10-11T06:11:45.599 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:11:45.599 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:11:45.600 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2023-10-11T06:11:45.600 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-11T06:11:45.600 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-11T06:11:45.600 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:11:45.601 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:11:45.601 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:11:45.601 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-11T06:11:45.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:45.602 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2023-10-11T06:11:45.662 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:45.663 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:45.663 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:45.663 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:45.663 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:45.664 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:45.664 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:45.664 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:45.664 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2023-10-11T06:11:45.664 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:45.664 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:45.665 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2023-10-11T06:11:45.665 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:11:45.665 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:45.665 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:11:45.665 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:45.665 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:11:45.666 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42389sec preferred_lft 42389sec 2023-10-11T06:11:45.666 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2023-10-11T06:11:45.666 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:45.666 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:11:45.667 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 4e:c2:04:fc:13:64 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:11:45.667 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:11:45.667 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:45.667 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::aca3:c2ff:fe44:6a3b/64 scope link 2023-10-11T06:11:45.667 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:45.667 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:11:45.668 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 4e:c2:04:fc:13:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-11T06:11:45.668 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::4cc2:4ff:fefc:1364/64 scope link 2023-10-11T06:11:45.668 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:45.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:45.669 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:11:45.761 INFO:teuthology.orchestra.run.smithi099.stdout:1 2023-10-11T06:11:45.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:45.765 DEBUG:teuthology.orchestra.run.smithi099:> route 2023-10-11T06:11:45.819 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2023-10-11T06:11:45.819 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:11:45.819 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:11:45.819 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:11:45.820 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:11:45.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:45.822 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:45.823 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:45.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:11:45.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:11:45.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:11:45.824 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:46.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:46.009 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-11T06:11:46.062 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-11T06:11:46.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:46.063 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2023-10-11T06:11:46.115 INFO:teuthology.orchestra.run.smithi099.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-11T06:11:46.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:46.116 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:46.117 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:46.117 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-11T06:11:46.117 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-11T06:11:46.118 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:46.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:46.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-11T06:11:46.354 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:46.354 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:46.354 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:46.354 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.355 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:46.355 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.355 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:11:46.355 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 52:64:45:a9:00:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T06:11:46.355 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T06:11:46.355 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.356 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5064:45ff:fea9:61/64 scope link 2023-10-11T06:11:46.356 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:46.358 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-11T06:11:46.451 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:11:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:11:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:11:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2023-10-11T06:11:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.453 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:11:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 52:64:45:a9:00:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-11T06:11:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-11T06:11:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5064:45ff:fea9:61/64 scope link 2023-10-11T06:11:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:11:46.455 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-11T06:11:46.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:46.456 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:46.456 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:46.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-11T06:11:46.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-11T06:11:46.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-11T06:11:46.458 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-11T06:11:46.458 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-11T06:11:46.458 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:46.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:46.713 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2023-10-11T06:11:46.714 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2023-10-11T06:11:46.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.1 up 2023-10-11T06:11:46.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.1 master ceph-brx 2023-10-11T06:11:46.714 DEBUG:teuthology.orchestra.run.smithi099:> ') 2023-10-11T06:11:46.876 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-11T06:11:46.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:46.876 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-11T06:11:46.926 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-11T06:11:46.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:11:46.927 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-11T06:11:46.977 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2023-10-11T06:11:47.076 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:47.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:47.172 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:47.175 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:47.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:47.176 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2023-10-11T06:11:47.232 INFO:teuthology.orchestra.run.smithi099.stdout:39 2023-10-11T06:11:47.232 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-10-11T06:11:47.233 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-11T06:11:47.241 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2023-10-11T06:11:47.327 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:11:47.328 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:11:47.402 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:11:47.405 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:11:47.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:47.406 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2023-10-11T06:11:47.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi099.stderr:2023-10-11T06:11:47.430+0000 7f0043963480 -1 init, newargv = 0x55d01a5cfa00 newargc=15 2023-10-11T06:11:47.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi099.stderr:ceph-fuse[93914]: starting ceph client 2023-10-11T06:11:47.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi099.stderr:ceph-fuse[93914]: starting fuse 2023-10-11T06:11:47.449 INFO:teuthology.orchestra.run.smithi099.stdout:39 2023-10-11T06:11:47.449 INFO:teuthology.orchestra.run.smithi099.stdout:42 2023-10-11T06:11:47.450 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 42] 2023-10-11T06:11:47.450 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:11:47.450 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2023-10-11T06:11:47.451 DEBUG:teuthology.orchestra.run.smithi099:> import re 2023-10-11T06:11:47.451 DEBUG:teuthology.orchestra.run.smithi099:> import os 2023-10-11T06:11:47.451 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2023-10-11T06:11:47.452 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:47.452 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2023-10-11T06:11:47.452 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-11T06:11:47.452 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2023-10-11T06:11:47.453 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-11T06:11:47.453 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:47.453 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2023-10-11T06:11:47.453 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2023-10-11T06:11:47.454 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2023-10-11T06:11:47.454 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2023-10-11T06:11:47.454 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:47.455 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2023-10-11T06:11:47.455 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:11:47.455 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:11:47.456 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:11:47.456 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2023-10-11T06:11:47.456 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2023-10-11T06:11:47.456 DEBUG:teuthology.orchestra.run.smithi099:> return f 2023-10-11T06:11:47.457 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:11:47.457 DEBUG:teuthology.orchestra.run.smithi099:> 2023-10-11T06:11:47.457 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.3")) 2023-10-11T06:11:47.458 DEBUG:teuthology.orchestra.run.smithi099:> ' 2023-10-11T06:11:47.650 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.3.93914.asok 2023-10-11T06:11:47.830 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.93914.asok 2023-10-11T06:11:47.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:47.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.93914.asok status 2023-10-11T06:11:48.069 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2023-10-11T06:11:48.070 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2023-10-11T06:11:48.070 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:11:48.070 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:11:48.070 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "3", 2023-10-11T06:11:48.071 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2023-10-11T06:11:48.071 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-11T06:11:48.071 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "93914", 2023-10-11T06:11:48.071 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2023-10-11T06:11:48.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:48.072 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2023-10-11T06:11:48.072 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2023-10-11T06:11:48.072 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4379, 2023-10-11T06:11:48.072 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2023-10-11T06:11:48.072 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2023-10-11T06:11:48.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2023-10-11T06:11:48.073 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4379 2023-10-11T06:11:48.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:48.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2023-10-11T06:11:48.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-11T06:11:48.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.2:0", 2023-10-11T06:11:48.074 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3133043396 2023-10-11T06:11:48.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2023-10-11T06:11:48.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:48.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2023-10-11T06:11:48.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2023-10-11T06:11:48.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.2:0", 2023-10-11T06:11:48.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3133043396 2023-10-11T06:11:48.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2023-10-11T06:11:48.075 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4379 192.168.0.2:0/3133043396", 2023-10-11T06:11:48.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.2:0/3133043396", 2023-10-11T06:11:48.076 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2023-10-11T06:11:48.076 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 8, 2023-10-11T06:11:48.076 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 24, 2023-10-11T06:11:48.076 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:11:48.076 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2023-10-11T06:11:48.077 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2023-10-11T06:11:48.077 INFO:teuthology.orchestra.run.smithi099.stdout:} 2023-10-11T06:11:48.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.084 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:11:48.085 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T06:11:48.147 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:11:48.147 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T06:11:48.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:11:48.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.248 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:11:48.248 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-11T06:11:48.302 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:11:48.303 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-11T06:11:48.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-11T06:11:48.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.393 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2023-10-11T06:11:48.393 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-11T06:11:48.455 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2023-10-11T06:11:48.456 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-11T06:11:48.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-11T06:11:48.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.545 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2023-10-11T06:11:48.545 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-11T06:11:48.601 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2023-10-11T06:11:48.602 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-11T06:11:48.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:11:48.602 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-11T06:11:48.699 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-11T06:11:48.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.941+0000 7f60931c9640 1 Processor -- start 2023-10-11T06:11:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.941+0000 7f60931c9640 1 -- start start 2023-10-11T06:11:48.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.942+0000 7f60931c9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f608c0cafb0 0x7f608c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.942+0000 7f60931c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0cb970 con 0x7f608c15c290 2023-10-11T06:11:48.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.943+0000 7f60931c9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c15fe40 con 0x7f608c0cc410 2023-10-11T06:11:48.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.943+0000 7f60931c9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c0cad70 con 0x7f608c0cafb0 2023-10-11T06:11:48.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.943+0000 7f6090f65640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f608c15c290 0x7f608c0c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39354/0 (socket says 172.21.15.84:39354) 2023-10-11T06:11:48.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.943+0000 7f6090f65640 1 -- 172.21.15.84:0/1728774360 learned_addr learned my addr 172.21.15.84:0/1728774360 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:48.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.943+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461201226 0 0) 0x7f608c0cb970 con 0x7f608c15c290 2023-10-11T06:11:48.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.943+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6074003520 con 0x7f608c15c290 2023-10-11T06:11:48.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.943+0000 7f608bfff640 1 --2- 172.21.15.84:0/1728774360 >> v2:172.21.15.99:3300/0 conn(0x7f608c0cafb0 0x7f608c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2751100334 0 0) 0x7f6074003520 con 0x7f608c15c290 2023-10-11T06:11:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 >> v2:172.21.15.99:3300/0 conn(0x7f608c0cafb0 msgr2=0x7f608c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 --2- 172.21.15.84:0/1728774360 >> v2:172.21.15.99:3300/0 conn(0x7f608c0cafb0 0x7f608c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 >> v1:172.21.15.99:6789/0 conn(0x7f608c0cc410 legacy=0x7f608c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:48.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c06ff30 con 0x7f608c15c290 2023-10-11T06:11:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f60800017f0 con 0x7f608c15c290 2023-10-11T06:11:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 >> v1:172.21.15.84:6789/0 conn(0x7f608c15c290 legacy=0x7f608c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6074003970 0x7f6074003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60740047c0 0x7f6074004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6074005170 0x7f607400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cad70 con 0x7f60740047c0 2023-10-11T06:11:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607400bff0 con 0x7f6074005170 2023-10-11T06:11:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607400c1b0 con 0x7f6074003970 2023-10-11T06:11:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.944+0000 7f608bfff640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60740047c0 0x7f6074004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.945+0000 7f608bfff640 1 -- 172.21.15.84:0/1728774360 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6074003970 msgr2=0x7f6074003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.945+0000 7f608bfff640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6074003970 0x7f6074003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.945+0000 7f608bfff640 1 -- 172.21.15.84:0/1728774360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6074005170 msgr2=0x7f607400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.945+0000 7f608bfff640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6074005170 0x7f607400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.945+0000 7f608bfff640 1 -- 172.21.15.84:0/1728774360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6074003520 con 0x7f60740047c0 2023-10-11T06:11:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.945+0000 7f6091766640 1 -- 172.21.15.84:0/1728774360 reap_dead start 2023-10-11T06:11:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.946+0000 7f608bfff640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60740047c0 0x7f6074004c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f607c009ac0 tx=0x7f60740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babc97797c09c25b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.946+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c02c6d0 con 0x7f60740047c0 2023-10-11T06:11:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.946+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c02c830 con 0x7f60740047c0 2023-10-11T06:11:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.946+0000 7f608b7fe640 1 -- 172.21.15.84:0/1728774360 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c02cbc0 con 0x7f60740047c0 2023-10-11T06:11:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.946+0000 7f60931c9640 1 -- 172.21.15.84:0/1728774360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60740047c0 msgr2=0x7f6074004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.946+0000 7f60931c9640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60740047c0 0x7f6074004c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f607c009ac0 tx=0x7f60740057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.947+0000 7f60931c9640 1 -- 172.21.15.84:0/1728774360 shutdown_connections 2023-10-11T06:11:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.947+0000 7f60931c9640 1 --2- 172.21.15.84:0/1728774360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60740047c0 0x7f6074004c30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:48.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.947+0000 7f60931c9640 1 -- 172.21.15.84:0/1728774360 >> 172.21.15.84:0/1728774360 conn(0x7f608c069c70 msgr2=0x7f608c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:48.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.947+0000 7f60931c9640 1 -- 172.21.15.84:0/1728774360 shutdown_connections 2023-10-11T06:11:48.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.947+0000 7f60931c9640 1 -- 172.21.15.84:0/1728774360 wait complete. 2023-10-11T06:11:48.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.949+0000 7f60931c9640 1 Processor -- start 2023-10-11T06:11:48.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.949+0000 7f60931c9640 1 -- start start 2023-10-11T06:11:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f60931c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60740047c0 0x7f608c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f60931c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c15e830 0x7f608c15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f60931c9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f608c152810 0x7f608c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f60931c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c15f1e0 con 0x7f608c15e830 2023-10-11T06:11:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f60931c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b1430 con 0x7f60740047c0 2023-10-11T06:11:48.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f60931c9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b2290 con 0x7f608c152810 2023-10-11T06:11:48.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c15e830 0x7f608c15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:48.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c15e830 0x7f608c15eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55998/0 (socket says 172.21.15.84:55998) 2023-10-11T06:11:48.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 -- 172.21.15.84:0/4200977371 learned_addr learned my addr 172.21.15.84:0/4200977371 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:48.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f6090f65640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60740047c0 0x7f608c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:48.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 -- 172.21.15.84:0/4200977371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f608c152810 msgr2=0x7f608c152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:48.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f608c152810 0x7f608c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:48.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 -- 172.21.15.84:0/4200977371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60740047c0 msgr2=0x7f608c150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:48.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60740047c0 0x7f608c150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:48.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.950+0000 7f608bfff640 1 -- 172.21.15.84:0/4200977371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c150fe0 con 0x7f608c15e830 2023-10-11T06:11:48.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.951+0000 7f608bfff640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c15e830 0x7f608c15eca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f607c009ac0 tx=0x7f607c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:48.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.951+0000 7f60897fa640 1 -- 172.21.15.84:0/4200977371 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c003810 con 0x7f608c15e830 2023-10-11T06:11:48.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.951+0000 7f60897fa640 1 -- 172.21.15.84:0/4200977371 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c003970 con 0x7f608c15e830 2023-10-11T06:11:48.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.951+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c1531c0 con 0x7f608c15e830 2023-10-11T06:11:48.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.951+0000 7f60897fa640 1 -- 172.21.15.84:0/4200977371 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c003d00 con 0x7f608c15e830 2023-10-11T06:11:48.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.951+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c0b1960 con 0x7f608c15e830 2023-10-11T06:11:48.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.952+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c073ed0 con 0x7f608c15e830 2023-10-11T06:11:48.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.953+0000 7f60897fa640 1 -- 172.21.15.84:0/4200977371 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f607c040450 con 0x7f608c15e830 2023-10-11T06:11:48.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.954+0000 7f60897fa640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f607806a830 0x7f607806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:48.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.954+0000 7f60897fa640 1 -- 172.21.15.84:0/4200977371 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f607c0aafc0 con 0x7f608c15e830 2023-10-11T06:11:48.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.955+0000 7f6090f65640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f607806a830 0x7f607806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:48.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.955+0000 7f6090f65640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f607806a830 0x7f607806cce0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f608c0b2740 tx=0x7f6080009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:48.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:48.959+0000 7f60897fa640 1 -- 172.21.15.84:0/4200977371 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f607c0ddba0 con 0x7f608c15e830 2023-10-11T06:11:49.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.150+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f608c0b0050 con 0x7f608c15e830 2023-10-11T06:11:49.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.153+0000 7f60897fa640 1 -- 172.21.15.84:0/4200977371 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17763 (secure 0 0 0) 0x7f607c079cd0 con 0x7f608c15e830 2023-10-11T06:11:49.155 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:49.156 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:14.671090+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:49.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f607806a830 msgr2=0x7f607806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f607806a830 0x7f607806cce0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f608c0b2740 tx=0x7f6080009040 comp rx=0 tx=0).stop 2023-10-11T06:11:49.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c15e830 msgr2=0x7f608c15eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c15e830 0x7f608c15eca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f607c009ac0 tx=0x7f607c004730 comp rx=0 tx=0).stop 2023-10-11T06:11:49.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 shutdown_connections 2023-10-11T06:11:49.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f607806a830 0x7f607806cce0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f608c152810 0x7f608c152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c15e830 0x7f608c15eca0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 --2- 172.21.15.84:0/4200977371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60740047c0 0x7f608c150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.156+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 >> 172.21.15.84:0/4200977371 conn(0x7f608c069c70 msgr2=0x7f608c0ac470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:49.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.157+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 shutdown_connections 2023-10-11T06:11:49.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.157+0000 7f60931c9640 1 -- 172.21.15.84:0/4200977371 wait complete. 2023-10-11T06:11:49.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:49.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:49.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.393+0000 7f40f1a41640 1 Processor -- start 2023-10-11T06:11:49.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.393+0000 7f40f1a41640 1 -- start start 2023-10-11T06:11:49.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.394+0000 7f40f1a41640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40ec0ced30 0x7f40ec0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40f1a41640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0c8d00 0x7f40ec06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40f1a41640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec071380 con 0x7f40ec15a0b0 2023-10-11T06:11:49.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40f1a41640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec06e0f0 con 0x7f40ec0ced30 2023-10-11T06:11:49.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40f1a41640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0cbae0 con 0x7f40ec0c8d00 2023-10-11T06:11:49.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40eaffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f40ec15a0b0 0x7f40ec0ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39356/0 (socket says 172.21.15.84:39356) 2023-10-11T06:11:49.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40eaffd640 1 -- 172.21.15.84:0/1519123472 learned_addr learned my addr 172.21.15.84:0/1519123472 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:49.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40e3fff640 1 --2- 172.21.15.84:0/1519123472 >> v2:172.21.15.84:3300/0 conn(0x7f40ec0ced30 0x7f40ec0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40eb7fe640 1 --2- 172.21.15.84:0/1519123472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0c8d00 0x7f40ec06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40ea7fc640 1 -- 172.21.15.84:0/1519123472 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547517717 0 0) 0x7f40ec071380 con 0x7f40ec15a0b0 2023-10-11T06:11:49.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40ea7fc640 1 -- 172.21.15.84:0/1519123472 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d8003520 con 0x7f40ec15a0b0 2023-10-11T06:11:49.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.395+0000 7f40e3fff640 1 -- 172.21.15.84:0/1519123472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0c8d00 msgr2=0x7f40ec06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:49.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.396+0000 7f40e3fff640 1 --2- 172.21.15.84:0/1519123472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0c8d00 0x7f40ec06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.396+0000 7f40e3fff640 1 -- 172.21.15.84:0/1519123472 >> v1:172.21.15.84:6789/0 conn(0x7f40ec15a0b0 legacy=0x7f40ec0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.396+0000 7f40e3fff640 1 -- 172.21.15.84:0/1519123472 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec06de10 con 0x7f40ec0ced30 2023-10-11T06:11:49.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.397+0000 7f40e3fff640 1 --2- 172.21.15.84:0/1519123472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec0ced30 0x7f40ec0cb5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097d0 tx=0x7f40d402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ced73299eb62f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:49.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.397+0000 7f40ea7fc640 1 -- 172.21.15.84:0/1519123472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40316f0 con 0x7f40ec0ced30 2023-10-11T06:11:49.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.397+0000 7f40ea7fc640 1 -- 172.21.15.84:0/1519123472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4031850 con 0x7f40ec0ced30 2023-10-11T06:11:49.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.397+0000 7f40ea7fc640 1 -- 172.21.15.84:0/1519123472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4031bc0 con 0x7f40ec0ced30 2023-10-11T06:11:49.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.397+0000 7f40f1a41640 1 -- 172.21.15.84:0/1519123472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec0ced30 msgr2=0x7f40ec0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.397+0000 7f40f1a41640 1 --2- 172.21.15.84:0/1519123472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec0ced30 0x7f40ec0cb5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097d0 tx=0x7f40d402b790 comp rx=0 tx=0).stop 2023-10-11T06:11:49.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.398+0000 7f40f1a41640 1 -- 172.21.15.84:0/1519123472 shutdown_connections 2023-10-11T06:11:49.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.398+0000 7f40f1a41640 1 --2- 172.21.15.84:0/1519123472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0c8d00 0x7f40ec06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.398+0000 7f40f1a41640 1 --2- 172.21.15.84:0/1519123472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec0ced30 0x7f40ec0cb5a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.398+0000 7f40f1a41640 1 -- 172.21.15.84:0/1519123472 >> 172.21.15.84:0/1519123472 conn(0x7f40ec069c70 msgr2=0x7f40ec06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:49.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.398+0000 7f40f1a41640 1 -- 172.21.15.84:0/1519123472 shutdown_connections 2023-10-11T06:11:49.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.398+0000 7f40f1a41640 1 -- 172.21.15.84:0/1519123472 wait complete. 2023-10-11T06:11:49.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.399+0000 7f40f1a41640 1 Processor -- start 2023-10-11T06:11:49.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.400+0000 7f40f1a41640 1 -- start start 2023-10-11T06:11:49.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.400+0000 7f40f1a41640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40ec0c8d00 0x7f40ec152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.400+0000 7f40f1a41640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0ced30 0x7f40ec145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40f1a41640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec15a0b0 0x7f40ec146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40f1a41640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec1466a0 con 0x7f40ec15a0b0 2023-10-11T06:11:49.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40f1a41640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec155260 con 0x7f40ec0c8d00 2023-10-11T06:11:49.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40f1a41640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec1549c0 con 0x7f40ec0ced30 2023-10-11T06:11:49.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec15a0b0 0x7f40ec146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec15a0b0 0x7f40ec146160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56016/0 (socket says 172.21.15.84:56016) 2023-10-11T06:11:49.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 -- 172.21.15.84:0/4254091247 learned_addr learned my addr 172.21.15.84:0/4254091247 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:49.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 -- 172.21.15.84:0/4254091247 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0ced30 msgr2=0x7f40ec145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0ced30 0x7f40ec145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 -- 172.21.15.84:0/4254091247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40ec0c8d00 msgr2=0x7f40ec152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eaffd640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40ec0c8d00 0x7f40ec152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40ec0c8d00 0x7f40ec152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 -- 172.21.15.84:0/4254091247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4009400 con 0x7f40ec15a0b0 2023-10-11T06:11:49.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.401+0000 7f40eb7fe640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec15a0b0 0x7f40ec146160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f40e4007e40 tx=0x7f40e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:49.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.402+0000 7f40e8ff9640 1 -- 172.21.15.84:0/4254091247 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4014620 con 0x7f40ec15a0b0 2023-10-11T06:11:49.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.402+0000 7f40e8ff9640 1 -- 172.21.15.84:0/4254091247 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e400ee90 con 0x7f40ec15a0b0 2023-10-11T06:11:49.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.402+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec154ca0 con 0x7f40ec15a0b0 2023-10-11T06:11:49.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.402+0000 7f40e8ff9640 1 -- 172.21.15.84:0/4254091247 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e40149e0 con 0x7f40ec15a0b0 2023-10-11T06:11:49.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.402+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec150cd0 con 0x7f40ec15a0b0 2023-10-11T06:11:49.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.403+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec073ed0 con 0x7f40ec15a0b0 2023-10-11T06:11:49.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.404+0000 7f40e8ff9640 1 -- 172.21.15.84:0/4254091247 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f40e4014b40 con 0x7f40ec15a0b0 2023-10-11T06:11:49.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.405+0000 7f40e8ff9640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f40cc06a860 0x7f40cc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.405+0000 7f40e8ff9640 1 -- 172.21.15.84:0/4254091247 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40e4089950 con 0x7f40ec15a0b0 2023-10-11T06:11:49.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.405+0000 7f40eaffd640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f40cc06a860 0x7f40cc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.406+0000 7f40eaffd640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f40cc06a860 0x7f40cc06cd10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f40dc000f60 tx=0x7f40dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:49.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.409+0000 7f40e8ff9640 1 -- 172.21.15.84:0/4254091247 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f40e4054bc0 con 0x7f40ec15a0b0 2023-10-11T06:11:49.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.597+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40ec0c5540 con 0x7f40ec15a0b0 2023-10-11T06:11:49.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.598+0000 7f40e8ff9640 1 -- 172.21.15.84:0/4254091247 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40e40581e0 con 0x7f40ec15a0b0 2023-10-11T06:11:49.599 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:49.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.600+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f40cc06a860 msgr2=0x7f40cc06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.600+0000 7f40f1a41640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f40cc06a860 0x7f40cc06cd10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f40dc000f60 tx=0x7f40dc009040 comp rx=0 tx=0).stop 2023-10-11T06:11:49.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.600+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec15a0b0 msgr2=0x7f40ec146160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.600+0000 7f40f1a41640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec15a0b0 0x7f40ec146160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f40e4007e40 tx=0x7f40e400e4b0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 shutdown_connections 2023-10-11T06:11:49.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f40cc06a860 0x7f40cc06cd10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40ec15a0b0 0x7f40ec146160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40ec0ced30 0x7f40ec145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 --2- 172.21.15.84:0/4254091247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40ec0c8d00 0x7f40ec152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 >> 172.21.15.84:0/4254091247 conn(0x7f40ec069c70 msgr2=0x7f40ec0c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 shutdown_connections 2023-10-11T06:11:49.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.601+0000 7f40f1a41640 1 -- 172.21.15.84:0/4254091247 wait complete. 2023-10-11T06:11:49.622 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:49.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.835+0000 7efe82e29640 1 Processor -- start 2023-10-11T06:11:49.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.835+0000 7efe82e29640 1 -- start start 2023-10-11T06:11:49.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe82e29640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe7c158e20 0x7efe7c0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe82e29640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c1591f0 0x7efe7c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe82e29640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c0cb8e0 con 0x7efe7c0cc890 2023-10-11T06:11:49.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe82e29640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7c15cd40 con 0x7efe7c158e20 2023-10-11T06:11:49.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe82e29640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0ce2c0 con 0x7efe7c1591f0 2023-10-11T06:11:49.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe81e27640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe7c158e20 0x7efe7c0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe81e27640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe7c158e20 0x7efe7c0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:46344/0 (socket says 172.21.15.84:46344) 2023-10-11T06:11:49.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe81626640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efe7c0cc890 0x7efe7c069c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:55584/0 (socket says 172.21.15.84:55584) 2023-10-11T06:11:49.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe81e27640 1 -- 172.21.15.84:0/327157292 learned_addr learned my addr 172.21.15.84:0/327157292 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:49.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.837+0000 7efe82628640 1 --2- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c1591f0 0x7efe7c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.838+0000 7efe81e27640 1 -- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c1591f0 msgr2=0x7efe7c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:49.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.838+0000 7efe81e27640 1 --2- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c1591f0 0x7efe7c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.838+0000 7efe81e27640 1 -- 172.21.15.84:0/327157292 >> v1:172.21.15.99:6789/0 conn(0x7efe7c0cc890 legacy=0x7efe7c069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.838+0000 7efe81e27640 1 -- 172.21.15.84:0/327157292 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c15cfe0 con 0x7efe7c158e20 2023-10-11T06:11:49.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.838+0000 7efe82628640 1 --2- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c1591f0 0x7efe7c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:11:49.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.838+0000 7efe80e25640 1 -- 172.21.15.84:0/327157292 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312298142 0 0) 0x7efe7c0cb8e0 con 0x7efe7c0cc890 2023-10-11T06:11:49.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.840+0000 7efe81e27640 1 --2- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c158e20 0x7efe7c0cae60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe78009870 tx=0x7efe7802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71113623fdafcea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:49.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.840+0000 7efe80e25640 1 -- 172.21.15.84:0/327157292 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78031640 con 0x7efe7c158e20 2023-10-11T06:11:49.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.840+0000 7efe80e25640 1 -- 172.21.15.84:0/327157292 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe780317a0 con 0x7efe7c158e20 2023-10-11T06:11:49.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.840+0000 7efe80e25640 1 -- 172.21.15.84:0/327157292 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78031b30 con 0x7efe7c158e20 2023-10-11T06:11:49.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 -- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c158e20 msgr2=0x7efe7c0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 --2- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c158e20 0x7efe7c0cae60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe78009870 tx=0x7efe7802b8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 -- 172.21.15.84:0/327157292 shutdown_connections 2023-10-11T06:11:49.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 --2- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c1591f0 0x7efe7c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 --2- 172.21.15.84:0/327157292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c158e20 0x7efe7c0cae60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 -- 172.21.15.84:0/327157292 >> 172.21.15.84:0/327157292 conn(0x7efe7c06bfb0 msgr2=0x7efe7c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:49.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 -- 172.21.15.84:0/327157292 shutdown_connections 2023-10-11T06:11:49.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.841+0000 7efe82e29640 1 -- 172.21.15.84:0/327157292 wait complete. 2023-10-11T06:11:49.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.843+0000 7efe82e29640 1 Processor -- start 2023-10-11T06:11:49.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.843+0000 7efe82e29640 1 -- start start 2023-10-11T06:11:49.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82e29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c0cc890 0x7efe7c14a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82e29640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c158e20 0x7efe7c1f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82e29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe7c1591f0 0x7efe7c0ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82e29640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0c4860 con 0x7efe7c1591f0 2023-10-11T06:11:49.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82e29640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c14b080 con 0x7efe7c0cc890 2023-10-11T06:11:49.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82e29640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0ac2a0 con 0x7efe7c158e20 2023-10-11T06:11:49.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe7c1591f0 0x7efe7c0ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe7c1591f0 0x7efe7c0ac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56024/0 (socket says 172.21.15.84:56024) 2023-10-11T06:11:49.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 -- 172.21.15.84:0/3117373288 learned_addr learned my addr 172.21.15.84:0/3117373288 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:49.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe81e27640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c0cc890 0x7efe7c14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe81626640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c158e20 0x7efe7c1f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 -- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c158e20 msgr2=0x7efe7c1f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c158e20 0x7efe7c1f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 -- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c0cc890 msgr2=0x7efe7c14a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:49.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c0cc890 0x7efe7c14a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.845+0000 7efe82628640 1 -- 172.21.15.84:0/3117373288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7400ba70 con 0x7efe7c1591f0 2023-10-11T06:11:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.846+0000 7efe82628640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe7c1591f0 0x7efe7c0ac670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe740082e0 tx=0x7efe740087a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.846+0000 7efe727fc640 1 -- 172.21.15.84:0/3117373288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74010040 con 0x7efe7c1591f0 2023-10-11T06:11:49.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.846+0000 7efe727fc640 1 -- 172.21.15.84:0/3117373288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe74003b90 con 0x7efe7c1591f0 2023-10-11T06:11:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.846+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe780094d0 con 0x7efe7c1591f0 2023-10-11T06:11:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.846+0000 7efe727fc640 1 -- 172.21.15.84:0/3117373288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7400be10 con 0x7efe7c1591f0 2023-10-11T06:11:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.846+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c0c3be0 con 0x7efe7c1591f0 2023-10-11T06:11:49.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.848+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c0c49a0 con 0x7efe7c1591f0 2023-10-11T06:11:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.848+0000 7efe727fc640 1 -- 172.21.15.84:0/3117373288 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7efe74004030 con 0x7efe7c1591f0 2023-10-11T06:11:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.849+0000 7efe727fc640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efe6006a830 0x7efe6006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:49.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.849+0000 7efe727fc640 1 -- 172.21.15.84:0/3117373288 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe74089490 con 0x7efe7c1591f0 2023-10-11T06:11:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.850+0000 7efe81e27640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efe6006a830 0x7efe6006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.850+0000 7efe81e27640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efe6006a830 0x7efe6006cce0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe78036000 tx=0x7efe7803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:49.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:49.854+0000 7efe727fc640 1 -- 172.21.15.84:0/3117373288 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7efe7c0c49a0 con 0x7efe7c1591f0 2023-10-11T06:11:50.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.040+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efe7c0c49a0 con 0x7efe7c1591f0 2023-10-11T06:11:50.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.041+0000 7efe727fc640 1 -- 172.21.15.84:0/3117373288 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7efe7c0c49a0 con 0x7efe7c1591f0 2023-10-11T06:11:50.043 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:50.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.043+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efe6006a830 msgr2=0x7efe6006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.043+0000 7efe82e29640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efe6006a830 0x7efe6006cce0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe78036000 tx=0x7efe7803c040 comp rx=0 tx=0).stop 2023-10-11T06:11:50.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.043+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe7c1591f0 msgr2=0x7efe7c0ac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.043+0000 7efe82e29640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe7c1591f0 0x7efe7c0ac670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe740082e0 tx=0x7efe740087a0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 shutdown_connections 2023-10-11T06:11:50.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efe6006a830 0x7efe6006cce0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe7c1591f0 0x7efe7c0ac670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe7c158e20 0x7efe7c1f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 --2- 172.21.15.84:0/3117373288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe7c0cc890 0x7efe7c14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 >> 172.21.15.84:0/3117373288 conn(0x7efe7c06bfb0 msgr2=0x7efe7c0c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:50.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 shutdown_connections 2023-10-11T06:11:50.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.044+0000 7efe82e29640 1 -- 172.21.15.84:0/3117373288 wait complete. 2023-10-11T06:11:50.064 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:11:50.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.281+0000 7f43366f7640 1 Processor -- start 2023-10-11T06:11:50.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.281+0000 7f43366f7640 1 -- start start 2023-10-11T06:11:50.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f43366f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4330150300 0x7f43300c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f43366f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43300ce510 0x7f43300ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f43366f7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43300c9b90 con 0x7f433014ff30 2023-10-11T06:11:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f43366f7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433015da10 con 0x7f4330150300 2023-10-11T06:11:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f43366f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43300cc990 con 0x7f43300ce510 2023-10-11T06:11:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f432ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f433014ff30 0x7f43300cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39362/0 (socket says 172.21.15.84:39362) 2023-10-11T06:11:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f432ffff640 1 -- 172.21.15.84:0/2454229834 learned_addr learned my addr 172.21.15.84:0/2454229834 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.283+0000 7f432f7fe640 1 --2- 172.21.15.84:0/2454229834 >> v2:172.21.15.84:3300/0 conn(0x7f4330150300 0x7f43300c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.284+0000 7f432effd640 1 -- 172.21.15.84:0/2454229834 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260469265 0 0) 0x7f43300c9b90 con 0x7f433014ff30 2023-10-11T06:11:50.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.284+0000 7f432effd640 1 -- 172.21.15.84:0/2454229834 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4318005730 con 0x7f433014ff30 2023-10-11T06:11:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.284+0000 7f432f7fe640 1 -- 172.21.15.84:0/2454229834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43300ce510 msgr2=0x7f43300ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.284+0000 7f4334c94640 1 --2- 172.21.15.84:0/2454229834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43300ce510 0x7f43300ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.284+0000 7f432f7fe640 1 --2- 172.21.15.84:0/2454229834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43300ce510 0x7f43300ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.284+0000 7f432f7fe640 1 -- 172.21.15.84:0/2454229834 >> v1:172.21.15.84:6789/0 conn(0x7f433014ff30 legacy=0x7f43300cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.284+0000 7f432f7fe640 1 -- 172.21.15.84:0/2454229834 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43300ceed0 con 0x7f4330150300 2023-10-11T06:11:50.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.285+0000 7f432f7fe640 1 --2- 172.21.15.84:0/2454229834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f43300c9650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43240097a0 tx=0x7f432402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea243410354741c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:50.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.285+0000 7f432effd640 1 -- 172.21.15.84:0/2454229834 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324031780 con 0x7f4330150300 2023-10-11T06:11:50.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.285+0000 7f432effd640 1 -- 172.21.15.84:0/2454229834 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43240318e0 con 0x7f4330150300 2023-10-11T06:11:50.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.285+0000 7f432effd640 1 -- 172.21.15.84:0/2454229834 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324031c70 con 0x7f4330150300 2023-10-11T06:11:50.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 -- 172.21.15.84:0/2454229834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 msgr2=0x7f43300c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 --2- 172.21.15.84:0/2454229834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f43300c9650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43240097a0 tx=0x7f432402b820 comp rx=0 tx=0).stop 2023-10-11T06:11:50.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 -- 172.21.15.84:0/2454229834 shutdown_connections 2023-10-11T06:11:50.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 --2- 172.21.15.84:0/2454229834 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43300ce510 0x7f43300ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 --2- 172.21.15.84:0/2454229834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f43300c9650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 -- 172.21.15.84:0/2454229834 >> 172.21.15.84:0/2454229834 conn(0x7f4330069c70 msgr2=0x7f433006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:50.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 -- 172.21.15.84:0/2454229834 shutdown_connections 2023-10-11T06:11:50.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.286+0000 7f43366f7640 1 -- 172.21.15.84:0/2454229834 wait complete. 2023-10-11T06:11:50.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.288+0000 7f43366f7640 1 Processor -- start 2023-10-11T06:11:50.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.288+0000 7f43366f7640 1 -- start start 2023-10-11T06:11:50.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f43366f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43300ce510 0x7f43300af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f43366f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f433014ff30 0x7f43300c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f43366f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f433014dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f43366f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43300c5cc0 con 0x7f4330150300 2023-10-11T06:11:50.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f43366f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301f1af0 con 0x7f43300ce510 2023-10-11T06:11:50.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f43366f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43301f1dd0 con 0x7f433014ff30 2023-10-11T06:11:50.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f4334c94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f433014dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f4334c94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f433014dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56042/0 (socket says 172.21.15.84:56042) 2023-10-11T06:11:50.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f4334c94640 1 -- 172.21.15.84:0/1989860458 learned_addr learned my addr 172.21.15.84:0/1989860458 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:50.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.289+0000 7f432ffff640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43300ce510 0x7f43300af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f432f7fe640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f433014ff30 0x7f43300c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f4334c94640 1 -- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f433014ff30 msgr2=0x7f43300c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f4334c94640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f433014ff30 0x7f43300c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f4334c94640 1 -- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43300ce510 msgr2=0x7f43300af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f4334c94640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43300ce510 0x7f43300af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f4334c94640 1 -- 172.21.15.84:0/1989860458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324009400 con 0x7f4330150300 2023-10-11T06:11:50.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f432ffff640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43300ce510 0x7f43300af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:50.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f4334c94640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f433014dd10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f432800cd70 tx=0x7f432800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:50.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f432cff9640 1 -- 172.21.15.84:0/1989860458 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328003b90 con 0x7f4330150300 2023-10-11T06:11:50.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f432cff9640 1 -- 172.21.15.84:0/1989860458 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4328003cf0 con 0x7f4330150300 2023-10-11T06:11:50.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f432cff9640 1 -- 172.21.15.84:0/1989860458 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328004240 con 0x7f4330150300 2023-10-11T06:11:50.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43301f20b0 con 0x7f4330150300 2023-10-11T06:11:50.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.290+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43301f1690 con 0x7f4330150300 2023-10-11T06:11:50.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.292+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43301f18b0 con 0x7f4330150300 2023-10-11T06:11:50.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.292+0000 7f432cff9640 1 -- 172.21.15.84:0/1989860458 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f43280043a0 con 0x7f4330150300 2023-10-11T06:11:50.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.294+0000 7f432cff9640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f431c06a830 0x7f431c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.294+0000 7f432cff9640 1 -- 172.21.15.84:0/1989860458 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4328012070 con 0x7f4330150300 2023-10-11T06:11:50.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.294+0000 7f432ffff640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f431c06a830 0x7f431c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.295+0000 7f432ffff640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f431c06a830 0x7f431c06cce0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4320007fd0 tx=0x7f4320009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:50.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.298+0000 7f432cff9640 1 -- 172.21.15.84:0/1989860458 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f43301f18b0 con 0x7f4330150300 2023-10-11T06:11:50.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.479+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43301f18b0 con 0x7f4330150300 2023-10-11T06:11:50.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.480+0000 7f432cff9640 1 -- 172.21.15.84:0/1989860458 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43301f18b0 con 0x7f4330150300 2023-10-11T06:11:50.481 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:11:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f431c06a830 msgr2=0x7f431c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f431c06a830 0x7f431c06cce0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4320007fd0 tx=0x7f4320009040 comp rx=0 tx=0).stop 2023-10-11T06:11:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 msgr2=0x7f433014dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f433014dd10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f432800cd70 tx=0x7f432800a660 comp rx=0 tx=0).stop 2023-10-11T06:11:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 shutdown_connections 2023-10-11T06:11:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f431c06a830 0x7f431c06cce0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4330150300 0x7f433014dd10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f433014ff30 0x7f43300c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 --2- 172.21.15.84:0/1989860458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43300ce510 0x7f43300af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.482+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 >> 172.21.15.84:0/1989860458 conn(0x7f4330069c70 msgr2=0x7f43300cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.483+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 shutdown_connections 2023-10-11T06:11:50.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.483+0000 7f43366f7640 1 -- 172.21.15.84:0/1989860458 wait complete. 2023-10-11T06:11:50.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:11:50.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.706+0000 7f734347d640 1 Processor -- start 2023-10-11T06:11:50.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.707+0000 7f734347d640 1 -- start start 2023-10-11T06:11:50.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.708+0000 7f734347d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f733c0cb5a0 0x7f733c0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.708+0000 7f734347d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c0c9240 con 0x7f733c15a0b0 2023-10-11T06:11:50.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.708+0000 7f734347d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c071380 con 0x7f733c0ced30 2023-10-11T06:11:50.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.708+0000 7f734347d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c06e0f0 con 0x7f733c0cb5a0 2023-10-11T06:11:50.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7341219640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f733c15a0b0 0x7f733c0ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39378/0 (socket says 172.21.15.84:39378) 2023-10-11T06:11:50.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7341219640 1 -- 172.21.15.84:0/2345951067 learned_addr learned my addr 172.21.15.84:0/2345951067 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7341a1a640 1 --2- 172.21.15.84:0/2345951067 >> v2:172.21.15.99:3300/0 conn(0x7f733c0cb5a0 0x7f733c0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439221417 0 0) 0x7f7338001110 con 0x7f733c15a0b0 2023-10-11T06:11:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7328003520 con 0x7f733c15a0b0 2023-10-11T06:11:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1238339092 0 0) 0x7f7328003520 con 0x7f733c15a0b0 2023-10-11T06:11:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 >> v2:172.21.15.99:3300/0 conn(0x7f733c0cb5a0 msgr2=0x7f733c0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:50.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.709+0000 7f7333fff640 1 --2- 172.21.15.84:0/2345951067 >> v2:172.21.15.99:3300/0 conn(0x7f733c0cb5a0 0x7f733c0c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 >> v1:172.21.15.99:6789/0 conn(0x7f733c0ced30 legacy=0x7f733c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c06de10 con 0x7f733c15a0b0 2023-10-11T06:11:50.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f7338001800 con 0x7f733c15a0b0 2023-10-11T06:11:50.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 >> v1:172.21.15.84:6789/0 conn(0x7f733c15a0b0 legacy=0x7f733c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7328003970 0x7f7328003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73280047c0 0x7f7328004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7328005290 0x7f732800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c06e0f0 con 0x7f73280047c0 2023-10-11T06:11:50.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732800c1c0 con 0x7f7328003970 2023-10-11T06:11:50.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732800c380 con 0x7f7328005290 2023-10-11T06:11:50.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7340a18640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73280047c0 0x7f7328004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7341219640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7328003970 0x7f7328003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7340a18640 1 -- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7328005290 msgr2=0x7f732800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:50.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7340a18640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7328005290 0x7f732800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7340a18640 1 -- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7328003970 msgr2=0x7f7328003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7340a18640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7328003970 0x7f7328003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7340a18640 1 -- 172.21.15.84:0/2345951067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7328003520 con 0x7f73280047c0 2023-10-11T06:11:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7341219640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7328003970 0x7f7328003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.710+0000 7f7341a1a640 1 -- 172.21.15.84:0/2345951067 reap_dead start 2023-10-11T06:11:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.711+0000 7f7340a18640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73280047c0 0x7f7328004c30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f733c0cf120 tx=0x7f73280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a33f9972bc4ad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c0052d0 con 0x7f73280047c0 2023-10-11T06:11:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c02e050 con 0x7f73280047c0 2023-10-11T06:11:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f7333fff640 1 -- 172.21.15.84:0/2345951067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c003830 con 0x7f73280047c0 2023-10-11T06:11:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f734347d640 1 -- 172.21.15.84:0/2345951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73280047c0 msgr2=0x7f7328004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f734347d640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73280047c0 0x7f7328004c30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f733c0cf120 tx=0x7f73280057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f734347d640 1 -- 172.21.15.84:0/2345951067 shutdown_connections 2023-10-11T06:11:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f734347d640 1 --2- 172.21.15.84:0/2345951067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73280047c0 0x7f7328004c30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f734347d640 1 -- 172.21.15.84:0/2345951067 >> 172.21.15.84:0/2345951067 conn(0x7f733c069c70 msgr2=0x7f733c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.712+0000 7f734347d640 1 -- 172.21.15.84:0/2345951067 shutdown_connections 2023-10-11T06:11:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.713+0000 7f734347d640 1 -- 172.21.15.84:0/2345951067 wait complete. 2023-10-11T06:11:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.714+0000 7f734347d640 1 Processor -- start 2023-10-11T06:11:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.714+0000 7f734347d640 1 -- start start 2023-10-11T06:11:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.715+0000 7f734347d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280047c0 0x7f733c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.715+0000 7f734347d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f733c146c90 0x7f733c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.715+0000 7f734347d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c158cc0 0x7f733c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.715+0000 7f734347d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0b42d0 con 0x7f733c158cc0 2023-10-11T06:11:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.715+0000 7f734347d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0c8230 con 0x7f73280047c0 2023-10-11T06:11:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.715+0000 7f734347d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0c8510 con 0x7f733c146c90 2023-10-11T06:11:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c158cc0 0x7f733c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c158cc0 0x7f733c0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56060/0 (socket says 172.21.15.84:56060) 2023-10-11T06:11:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 -- 172.21.15.84:0/3159848228 learned_addr learned my addr 172.21.15.84:0/3159848228 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341219640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280047c0 0x7f733c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7340a18640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f733c146c90 0x7f733c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 -- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f733c146c90 msgr2=0x7f733c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f733c146c90 0x7f733c158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 -- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280047c0 msgr2=0x7f733c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280047c0 0x7f733c158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341a1a640 1 -- 172.21.15.84:0/3159848228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0b3c80 con 0x7f733c158cc0 2023-10-11T06:11:50.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7341219640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280047c0 0x7f733c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:50.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.716+0000 7f7340a18640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f733c146c90 0x7f733c158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:50.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.717+0000 7f7341a1a640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c158cc0 0x7f733c0c6dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f733c0c8c80 tx=0x7f733400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:50.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.717+0000 7f7331ffb640 1 -- 172.21.15.84:0/3159848228 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334010040 con 0x7f733c158cc0 2023-10-11T06:11:50.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.717+0000 7f7331ffb640 1 -- 172.21.15.84:0/3159848228 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334014810 con 0x7f733c158cc0 2023-10-11T06:11:50.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.717+0000 7f7331ffb640 1 -- 172.21.15.84:0/3159848228 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334009b10 con 0x7f733c158cc0 2023-10-11T06:11:50.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.717+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c0b3f60 con 0x7f733c158cc0 2023-10-11T06:11:50.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.717+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c132af0 con 0x7f733c158cc0 2023-10-11T06:11:50.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.718+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c0b4440 con 0x7f733c158cc0 2023-10-11T06:11:50.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.719+0000 7f7331ffb640 1 -- 172.21.15.84:0/3159848228 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f7334014bc0 con 0x7f733c158cc0 2023-10-11T06:11:50.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.720+0000 7f7331ffb640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f732006a830 0x7f732006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:50.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.720+0000 7f7331ffb640 1 -- 172.21.15.84:0/3159848228 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7334089060 con 0x7f733c158cc0 2023-10-11T06:11:50.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.720+0000 7f7341219640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f732006a830 0x7f732006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:50.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.721+0000 7f7341219640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f732006a830 0x7f732006cce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f733c063890 tx=0x7f7338009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:50.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.724+0000 7f7331ffb640 1 -- 172.21.15.84:0/3159848228 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f73340542d0 con 0x7f733c158cc0 2023-10-11T06:11:50.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.922+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f733c073ed0 con 0x7f733c158cc0 2023-10-11T06:11:50.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.922+0000 7f7331ffb640 1 -- 172.21.15.84:0/3159848228 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73340180a0 con 0x7f733c158cc0 2023-10-11T06:11:50.924 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:11:50.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f732006a830 msgr2=0x7f732006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f732006a830 0x7f732006cce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f733c063890 tx=0x7f7338009040 comp rx=0 tx=0).stop 2023-10-11T06:11:50.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c158cc0 msgr2=0x7f733c0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:50.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c158cc0 0x7f733c0c6dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f733c0c8c80 tx=0x7f733400ef10 comp rx=0 tx=0).stop 2023-10-11T06:11:50.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 shutdown_connections 2023-10-11T06:11:50.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f732006a830 0x7f732006cce0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f733c158cc0 0x7f733c0c6dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f733c146c90 0x7f733c158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 --2- 172.21.15.84:0/3159848228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280047c0 0x7f733c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:50.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.925+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 >> 172.21.15.84:0/3159848228 conn(0x7f733c069c70 msgr2=0x7f733c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:50.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.926+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 shutdown_connections 2023-10-11T06:11:50.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:50.926+0000 7f734347d640 1 -- 172.21.15.84:0/3159848228 wait complete. 2023-10-11T06:11:50.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:51.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.137+0000 7fc42cdf5640 1 Processor -- start 2023-10-11T06:11:51.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.137+0000 7fc42cdf5640 1 -- start start 2023-10-11T06:11:51.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc42cdf5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 0x7fc4280caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc42cdf5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4281593f0 con 0x7fc4280cd300 2023-10-11T06:11:51.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc42cdf5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4281596c0 con 0x7fc4280ce0d0 2023-10-11T06:11:51.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc42cdf5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42815a340 con 0x7fc4280adb90 2023-10-11T06:11:51.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc426d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc4280cd300 0x7fc42815cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39392/0 (socket says 172.21.15.84:39392) 2023-10-11T06:11:51.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc426d9d640 1 -- 172.21.15.84:0/3674899288 learned_addr learned my addr 172.21.15.84:0/3674899288 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc41ffff640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 0x7fc4280caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487946976 0 0) 0x7fc4281593f0 con 0x7fc4280cd300 2023-10-11T06:11:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.139+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc414003520 con 0x7fc4280cd300 2023-10-11T06:11:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218479101 0 0) 0x7fc414003520 con 0x7fc4280cd300 2023-10-11T06:11:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 msgr2=0x7fc4280caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 0x7fc4280caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 >> v1:172.21.15.99:6789/0 conn(0x7fc4280ce0d0 legacy=0x7fc4280c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4281599a0 con 0x7fc4280cd300 2023-10-11T06:11:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fc4200019f0 con 0x7fc4280cd300 2023-10-11T06:11:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 >> v1:172.21.15.84:6789/0 conn(0x7fc4280cd300 legacy=0x7fc42815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4140039a0 0x7fc414003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 0x7fc4140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc414005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42815a340 con 0x7fc414004d00 2023-10-11T06:11:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41400dca0 con 0x7fc4140039a0 2023-10-11T06:11:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41400de60 con 0x7fc4280adb90 2023-10-11T06:11:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc426d9d640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc414005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc41ffff640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 0x7fc4140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.141+0000 7fc426d9d640 1 -- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 msgr2=0x7fc4140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.140+0000 7fc42659c640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4140039a0 0x7fc414003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.141+0000 7fc426d9d640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 0x7fc4140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.141+0000 7fc426d9d640 1 -- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4140039a0 msgr2=0x7fc414003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.141+0000 7fc426d9d640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4140039a0 0x7fc414003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.141+0000 7fc426d9d640 1 -- 172.21.15.84:0/3674899288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414003520 con 0x7fc414004d00 2023-10-11T06:11:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.141+0000 7fc42659c640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4140039a0 0x7fc414003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.141+0000 7fc41ffff640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4280adb90 0x7fc4140047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.142+0000 7fc426d9d640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc414005170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc42000a830 tx=0x7fc4140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcdf132ebf63682 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.142+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4200072d0 con 0x7fc414004d00 2023-10-11T06:11:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.142+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42002f070 con 0x7fc414004d00 2023-10-11T06:11:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.142+0000 7fc425d9b640 1 -- 172.21.15.84:0/3674899288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4200076b0 con 0x7fc414004d00 2023-10-11T06:11:51.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.143+0000 7fc42cdf5640 1 -- 172.21.15.84:0/3674899288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 msgr2=0x7fc414005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.143+0000 7fc42cdf5640 1 --2- 172.21.15.84:0/3674899288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc414005170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc42000a830 tx=0x7fc4140057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.143+0000 7fc426d9d640 1 -- 172.21.15.84:0/3674899288 reap_dead start 2023-10-11T06:11:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.143+0000 7fc42cdf5640 1 -- 172.21.15.84:0/3674899288 shutdown_connections 2023-10-11T06:11:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.143+0000 7fc42cdf5640 1 -- 172.21.15.84:0/3674899288 >> 172.21.15.84:0/3674899288 conn(0x7fc428069c70 msgr2=0x7fc42806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.143+0000 7fc42cdf5640 1 -- 172.21.15.84:0/3674899288 shutdown_connections 2023-10-11T06:11:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.143+0000 7fc42cdf5640 1 -- 172.21.15.84:0/3674899288 wait complete. 2023-10-11T06:11:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.145+0000 7fc42cdf5640 1 Processor -- start 2023-10-11T06:11:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.145+0000 7fc42cdf5640 1 -- start start 2023-10-11T06:11:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42cdf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc4280c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42cdf5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4280b1430 0x7fc4280b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42cdf5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc428152ec0 0x7fc428153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42cdf5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280b2920 con 0x7fc414004d00 2023-10-11T06:11:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42cdf5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc428153870 con 0x7fc4280b1430 2023-10-11T06:11:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42cdf5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280b1ca0 con 0x7fc428152ec0 2023-10-11T06:11:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc4280c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc4280c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56078/0 (socket says 172.21.15.84:56078) 2023-10-11T06:11:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 -- 172.21.15.84:0/449808427 learned_addr learned my addr 172.21.15.84:0/449808427 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 -- 172.21.15.84:0/449808427 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc428152ec0 msgr2=0x7fc428153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc428152ec0 0x7fc428153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 -- 172.21.15.84:0/449808427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4280b1430 msgr2=0x7fc4280b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4280b1430 0x7fc4280b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 -- 172.21.15.84:0/449808427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4280b1f80 con 0x7fc414004d00 2023-10-11T06:11:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.146+0000 7fc42659c640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc4280c4390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d5f0 tx=0x7fc41800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.147+0000 7fc41effd640 1 -- 172.21.15.84:0/449808427 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418017070 con 0x7fc414004d00 2023-10-11T06:11:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.147+0000 7fc41effd640 1 -- 172.21.15.84:0/449808427 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc418005280 con 0x7fc414004d00 2023-10-11T06:11:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.147+0000 7fc41effd640 1 -- 172.21.15.84:0/449808427 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418006820 con 0x7fc414004d00 2023-10-11T06:11:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.147+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4280b2260 con 0x7fc414004d00 2023-10-11T06:11:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.147+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4280b2d40 con 0x7fc414004d00 2023-10-11T06:11:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.148+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4280b2f60 con 0x7fc414004d00 2023-10-11T06:11:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.149+0000 7fc41effd640 1 -- 172.21.15.84:0/449808427 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc418006980 con 0x7fc414004d00 2023-10-11T06:11:51.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.150+0000 7fc41effd640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc40806a830 0x7fc40806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.150+0000 7fc41effd640 1 -- 172.21.15.84:0/449808427 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc4180888e0 con 0x7fc414004d00 2023-10-11T06:11:51.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.150+0000 7fc41ffff640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc40806a830 0x7fc40806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.151+0000 7fc41ffff640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc40806a830 0x7fc40806cce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4280b3100 tx=0x7fc410009350 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:51.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.154+0000 7fc41effd640 1 -- 172.21.15.84:0/449808427 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc4280b2f60 con 0x7fc414004d00 2023-10-11T06:11:51.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.345+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4280b2f60 con 0x7fc414004d00 2023-10-11T06:11:51.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.347+0000 7fc41effd640 1 -- 172.21.15.84:0/449808427 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fc4280b2f60 con 0x7fc414004d00 2023-10-11T06:11:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:51.349 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:50.855657+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:51.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.349+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc40806a830 msgr2=0x7fc40806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.349+0000 7fc42cdf5640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc40806a830 0x7fc40806cce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4280b3100 tx=0x7fc410009350 comp rx=0 tx=0).stop 2023-10-11T06:11:51.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.349+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 msgr2=0x7fc4280c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.349+0000 7fc42cdf5640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc4280c4390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc41800d5f0 tx=0x7fc41800dab0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 shutdown_connections 2023-10-11T06:11:51.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc40806a830 0x7fc40806cce0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc428152ec0 0x7fc428153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4280b1430 0x7fc4280b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 --2- 172.21.15.84:0/449808427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc414004d00 0x7fc4280c4390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 >> 172.21.15.84:0/449808427 conn(0x7fc428069c70 msgr2=0x7fc4280cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:51.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 shutdown_connections 2023-10-11T06:11:51.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.350+0000 7fc42cdf5640 1 -- 172.21.15.84:0/449808427 wait complete. 2023-10-11T06:11:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:51.371 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:51.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.586+0000 7ffbddb77640 1 Processor -- start 2023-10-11T06:11:51.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.587+0000 7ffbddb77640 1 -- start start 2023-10-11T06:11:51.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbddb77640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ffbd815c2b0 0x7ffbd806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbddb77640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffbd80ceda0 0x7ffbd80cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbddb77640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd8160890 con 0x7ffbd80cf950 2023-10-11T06:11:51.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbddb77640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd80cf220 con 0x7ffbd80ceda0 2023-10-11T06:11:51.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbddb77640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd80cd820 con 0x7ffbd815c2b0 2023-10-11T06:11:51.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbd7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffbd80cf950 0x7ffbd80b44f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39406/0 (socket says 172.21.15.84:39406) 2023-10-11T06:11:51.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbd7fff640 1 -- 172.21.15.84:0/2904694478 learned_addr learned my addr 172.21.15.84:0/2904694478 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:51.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.588+0000 7ffbd6ffd640 1 --2- 172.21.15.84:0/2904694478 >> v2:172.21.15.84:3300/0 conn(0x7ffbd80ceda0 0x7ffbd80cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.589+0000 7ffbd77fe640 1 --2- 172.21.15.84:0/2904694478 >> v2:172.21.15.99:3300/0 conn(0x7ffbd815c2b0 0x7ffbd806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.589+0000 7ffbd67fc640 1 -- 172.21.15.84:0/2904694478 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101160972 0 0) 0x7ffbd8160890 con 0x7ffbd80cf950 2023-10-11T06:11:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.589+0000 7ffbd67fc640 1 -- 172.21.15.84:0/2904694478 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc4003520 con 0x7ffbd80cf950 2023-10-11T06:11:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.589+0000 7ffbd6ffd640 1 -- 172.21.15.84:0/2904694478 >> v2:172.21.15.99:3300/0 conn(0x7ffbd815c2b0 msgr2=0x7ffbd806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.589+0000 7ffbd6ffd640 1 --2- 172.21.15.84:0/2904694478 >> v2:172.21.15.99:3300/0 conn(0x7ffbd815c2b0 0x7ffbd806ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.589+0000 7ffbd6ffd640 1 -- 172.21.15.84:0/2904694478 >> v1:172.21.15.84:6789/0 conn(0x7ffbd80cf950 legacy=0x7ffbd80b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.589+0000 7ffbd6ffd640 1 -- 172.21.15.84:0/2904694478 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8160dc0 con 0x7ffbd80ceda0 2023-10-11T06:11:51.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.590+0000 7ffbd6ffd640 1 --2- 172.21.15.84:0/2904694478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd80ceda0 0x7ffbd80cf410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80b55d0 tx=0x7ffbc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89571f934891daf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:51.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.590+0000 7ffbd67fc640 1 -- 172.21.15.84:0/2904694478 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0031830 con 0x7ffbd80ceda0 2023-10-11T06:11:51.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.590+0000 7ffbd67fc640 1 -- 172.21.15.84:0/2904694478 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc0031990 con 0x7ffbd80ceda0 2023-10-11T06:11:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.590+0000 7ffbd67fc640 1 -- 172.21.15.84:0/2904694478 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0031d00 con 0x7ffbd80ceda0 2023-10-11T06:11:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 -- 172.21.15.84:0/2904694478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd80ceda0 msgr2=0x7ffbd80cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 --2- 172.21.15.84:0/2904694478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd80ceda0 0x7ffbd80cf410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80b55d0 tx=0x7ffbc002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 -- 172.21.15.84:0/2904694478 shutdown_connections 2023-10-11T06:11:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 --2- 172.21.15.84:0/2904694478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd80ceda0 0x7ffbd80cf410 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 --2- 172.21.15.84:0/2904694478 >> v2:172.21.15.99:3300/0 conn(0x7ffbd815c2b0 0x7ffbd806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 -- 172.21.15.84:0/2904694478 >> 172.21.15.84:0/2904694478 conn(0x7ffbd806af40 msgr2=0x7ffbd806b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:51.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 -- 172.21.15.84:0/2904694478 shutdown_connections 2023-10-11T06:11:51.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.591+0000 7ffbddb77640 1 -- 172.21.15.84:0/2904694478 wait complete. 2023-10-11T06:11:51.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.593+0000 7ffbddb77640 1 Processor -- start 2023-10-11T06:11:51.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.593+0000 7ffbddb77640 1 -- start start 2023-10-11T06:11:51.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.594+0000 7ffbddb77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd80ceda0 0x7ffbd81f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbddb77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbd80cf950 0x7ffbd80cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbddb77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd815c2b0 0x7ffbd80cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbddb77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd80b3150 con 0x7ffbd815c2b0 2023-10-11T06:11:51.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbddb77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd80b53d0 con 0x7ffbd80ceda0 2023-10-11T06:11:51.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbddb77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81f2250 con 0x7ffbd80cf950 2023-10-11T06:11:51.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd815c2b0 0x7ffbd80cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd815c2b0 0x7ffbd80cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56106/0 (socket says 172.21.15.84:56106) 2023-10-11T06:11:51.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 -- 172.21.15.84:0/1322131187 learned_addr learned my addr 172.21.15.84:0/1322131187 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:51.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 -- 172.21.15.84:0/1322131187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbd80cf950 msgr2=0x7ffbd80cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbd80cf950 0x7ffbd80cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 -- 172.21.15.84:0/1322131187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd80ceda0 msgr2=0x7ffbd81f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd80ceda0 0x7ffbd81f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 -- 172.21.15.84:0/1322131187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00093a0 con 0x7ffbd815c2b0 2023-10-11T06:11:51.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.595+0000 7ffbd7fff640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd815c2b0 0x7ffbd80cbd60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c430 tx=0x7ffbd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:51.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.596+0000 7ffbcffff640 1 -- 172.21.15.84:0/1322131187 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0005ce0 con 0x7ffbd815c2b0 2023-10-11T06:11:51.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.596+0000 7ffbcffff640 1 -- 172.21.15.84:0/1322131187 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd0005e40 con 0x7ffbd815c2b0 2023-10-11T06:11:51.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.596+0000 7ffbcffff640 1 -- 172.21.15.84:0/1322131187 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0007310 con 0x7ffbd815c2b0 2023-10-11T06:11:51.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.596+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd8156760 con 0x7ffbd815c2b0 2023-10-11T06:11:51.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.596+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd815eb10 con 0x7ffbd815c2b0 2023-10-11T06:11:51.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.597+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd815e970 con 0x7ffbd815c2b0 2023-10-11T06:11:51.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.598+0000 7ffbcffff640 1 -- 172.21.15.84:0/1322131187 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ffbd0007470 con 0x7ffbd815c2b0 2023-10-11T06:11:51.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.599+0000 7ffbcffff640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffbb806a860 0x7ffbb806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:51.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.599+0000 7ffbcffff640 1 -- 172.21.15.84:0/1322131187 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ffbd0088740 con 0x7ffbd815c2b0 2023-10-11T06:11:51.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.599+0000 7ffbd77fe640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffbb806a860 0x7ffbb806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.600+0000 7ffbd77fe640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffbb806a860 0x7ffbb806cd10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80070b0 tx=0x7ffbc8007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.604+0000 7ffbcffff640 1 -- 172.21.15.84:0/1322131187 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ffbd815e970 con 0x7ffbd815c2b0 2023-10-11T06:11:51.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.791+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbd815e970 con 0x7ffbd815c2b0 2023-10-11T06:11:51.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.792+0000 7ffbcffff640 1 -- 172.21.15.84:0/1322131187 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbd815e970 con 0x7ffbd815c2b0 2023-10-11T06:11:51.793 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.794+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffbb806a860 msgr2=0x7ffbb806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.794+0000 7ffbddb77640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffbb806a860 0x7ffbb806cd10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80070b0 tx=0x7ffbc8007040 comp rx=0 tx=0).stop 2023-10-11T06:11:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.794+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd815c2b0 msgr2=0x7ffbd80cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.794+0000 7ffbddb77640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd815c2b0 0x7ffbd80cbd60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c430 tx=0x7ffbd000c8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 shutdown_connections 2023-10-11T06:11:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffbb806a860 0x7ffbb806cd10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbd815c2b0 0x7ffbd80cbd60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbd80cf950 0x7ffbd80cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 --2- 172.21.15.84:0/1322131187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd80ceda0 0x7ffbd81f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:51.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 >> 172.21.15.84:0/1322131187 conn(0x7ffbd806af40 msgr2=0x7ffbd80c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:51.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 shutdown_connections 2023-10-11T06:11:51.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:51.795+0000 7ffbddb77640 1 -- 172.21.15.84:0/1322131187 wait complete. 2023-10-11T06:11:51.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:52.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.023+0000 7f12673cb640 1 Processor -- start 2023-10-11T06:11:52.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.023+0000 7f12673cb640 1 -- start start 2023-10-11T06:11:52.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.024+0000 7f12673cb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f126015b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.024+0000 7f12673cb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f126006fee0 0x7f12600affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f12673cb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12600ccb60 con 0x7f1260154940 2023-10-11T06:11:52.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f12673cb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126015b680 con 0x7f126006fee0 2023-10-11T06:11:52.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f12673cb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1260070420 con 0x7f1260154570 2023-10-11T06:11:52.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f1266bca640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f126006fee0 0x7f12600affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f1265bc8640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1260154940 0x7f12600cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39416/0 (socket says 172.21.15.84:39416) 2023-10-11T06:11:52.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f1265bc8640 1 -- 172.21.15.84:0/1557110036 learned_addr learned my addr 172.21.15.84:0/1557110036 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:52.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f12663c9640 1 --2- 172.21.15.84:0/1557110036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f126015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f1266bca640 1 -- 172.21.15.84:0/1557110036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 msgr2=0x7f126015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:52.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f1266bca640 1 --2- 172.21.15.84:0/1557110036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f126015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f1266bca640 1 -- 172.21.15.84:0/1557110036 >> v1:172.21.15.84:6789/0 conn(0x7f1260154940 legacy=0x7f12600cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f12653c7640 1 -- 172.21.15.84:0/1557110036 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115863581 0 0) 0x7f12600ccb60 con 0x7f1260154940 2023-10-11T06:11:52.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f1266bca640 1 -- 172.21.15.84:0/1557110036 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1260071e00 con 0x7f126006fee0 2023-10-11T06:11:52.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.025+0000 7f12663c9640 1 --2- 172.21.15.84:0/1557110036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f126015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.026+0000 7f1266bca640 1 --2- 172.21.15.84:0/1557110036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126006fee0 0x7f12600affd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1258009d90 tx=0x7f125802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f80746336c3579 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12653c7640 1 -- 172.21.15.84:0/1557110036 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125802cc60 con 0x7f126006fee0 2023-10-11T06:11:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12653c7640 1 -- 172.21.15.84:0/1557110036 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125802cdc0 con 0x7f126006fee0 2023-10-11T06:11:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12653c7640 1 -- 172.21.15.84:0/1557110036 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258034660 con 0x7f126006fee0 2023-10-11T06:11:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12673cb640 1 -- 172.21.15.84:0/1557110036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126006fee0 msgr2=0x7f12600affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12673cb640 1 --2- 172.21.15.84:0/1557110036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126006fee0 0x7f12600affd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1258009d90 tx=0x7f125802beb0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12673cb640 1 -- 172.21.15.84:0/1557110036 shutdown_connections 2023-10-11T06:11:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12673cb640 1 --2- 172.21.15.84:0/1557110036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126006fee0 0x7f12600affd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12673cb640 1 --2- 172.21.15.84:0/1557110036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f126015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.027+0000 7f12673cb640 1 -- 172.21.15.84:0/1557110036 >> 172.21.15.84:0/1557110036 conn(0x7f126006bfb0 msgr2=0x7f126006c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:52.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.028+0000 7f12673cb640 1 -- 172.21.15.84:0/1557110036 shutdown_connections 2023-10-11T06:11:52.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.028+0000 7f12673cb640 1 -- 172.21.15.84:0/1557110036 wait complete. 2023-10-11T06:11:52.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.029+0000 7f12673cb640 1 Processor -- start 2023-10-11T06:11:52.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.030+0000 7f12673cb640 1 -- start start 2023-10-11T06:11:52.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.030+0000 7f12673cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f126006fee0 0x7f12600b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.030+0000 7f12673cb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f12600b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f12673cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1260154940 0x7f12601f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f12673cb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12600b1c40 con 0x7f1260154940 2023-10-11T06:11:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f12673cb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12600b3950 con 0x7f126006fee0 2023-10-11T06:11:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f12673cb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12601f48b0 con 0x7f1260154570 2023-10-11T06:11:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1260154940 0x7f12601f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1260154940 0x7f12601f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56112/0 (socket says 172.21.15.84:56112) 2023-10-11T06:11:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 -- 172.21.15.84:0/1335800546 learned_addr learned my addr 172.21.15.84:0/1335800546 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 -- 172.21.15.84:0/1335800546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 msgr2=0x7f12600b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f12600b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 -- 172.21.15.84:0/1335800546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f126006fee0 msgr2=0x7f12600b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f126006fee0 0x7f12600b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.031+0000 7f1266bca640 1 -- 172.21.15.84:0/1335800546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258009a70 con 0x7f1260154940 2023-10-11T06:11:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.032+0000 7f1266bca640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1260154940 0x7f12601f4370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1258009d90 tx=0x7f1258004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.032+0000 7f1256ffd640 1 -- 172.21.15.84:0/1335800546 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258040650 con 0x7f1260154940 2023-10-11T06:11:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.032+0000 7f1256ffd640 1 -- 172.21.15.84:0/1335800546 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12580407d0 con 0x7f1260154940 2023-10-11T06:11:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.032+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12600b3060 con 0x7f1260154940 2023-10-11T06:11:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.032+0000 7f1256ffd640 1 -- 172.21.15.84:0/1335800546 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258040b80 con 0x7f1260154940 2023-10-11T06:11:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.032+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12600b3300 con 0x7f1260154940 2023-10-11T06:11:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.033+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126004e890 con 0x7f1260154940 2023-10-11T06:11:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.034+0000 7f1256ffd640 1 -- 172.21.15.84:0/1335800546 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1258040ce0 con 0x7f1260154940 2023-10-11T06:11:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.035+0000 7f1256ffd640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f124406a830 0x7f124406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.035+0000 7f1256ffd640 1 -- 172.21.15.84:0/1335800546 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f12580ac0d0 con 0x7f1260154940 2023-10-11T06:11:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.035+0000 7f12663c9640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f124406a830 0x7f124406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.036+0000 7f12663c9640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f124406a830 0x7f124406cce0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f125c006fd0 tx=0x7f125c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.039+0000 7f1256ffd640 1 -- 172.21.15.84:0/1335800546 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f12580771e0 con 0x7f1260154940 2023-10-11T06:11:52.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.232+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f12600b3a90 con 0x7f1260154940 2023-10-11T06:11:52.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.233+0000 7f1256ffd640 1 -- 172.21.15.84:0/1335800546 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f12600b3a90 con 0x7f1260154940 2023-10-11T06:11:52.234 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:52.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f124406a830 msgr2=0x7f124406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f124406a830 0x7f124406cce0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f125c006fd0 tx=0x7f125c008040 comp rx=0 tx=0).stop 2023-10-11T06:11:52.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1260154940 msgr2=0x7f12601f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1260154940 0x7f12601f4370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1258009d90 tx=0x7f1258004a40 comp rx=0 tx=0).stop 2023-10-11T06:11:52.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 shutdown_connections 2023-10-11T06:11:52.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f124406a830 0x7f124406cce0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1260154940 0x7f12601f4370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1260154570 0x7f12600b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 --2- 172.21.15.84:0/1335800546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f126006fee0 0x7f12600b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.235+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 >> 172.21.15.84:0/1335800546 conn(0x7f126006bfb0 msgr2=0x7f12600ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:52.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.236+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 shutdown_connections 2023-10-11T06:11:52.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.236+0000 7f12673cb640 1 -- 172.21.15.84:0/1335800546 wait complete. 2023-10-11T06:11:52.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:11:52.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.457+0000 7f6ffc1bb640 1 Processor -- start 2023-10-11T06:11:52.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.458+0000 7f6ffc1bb640 1 -- start start 2023-10-11T06:11:52.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ffc1bb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6ff40d2ac0 0x7f6ff40d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ffc1bb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40cf220 con 0x7f6ff40d0940 2023-10-11T06:11:52.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ffc1bb640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40cf500 con 0x7f6ff415f7e0 2023-10-11T06:11:52.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ffc1bb640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff40d3480 con 0x7f6ff40d2ac0 2023-10-11T06:11:52.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ffa758640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6ff40d0940 0x7f6ff40d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39422/0 (socket says 172.21.15.84:39422) 2023-10-11T06:11:52.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ffa758640 1 -- 172.21.15.84:0/201631439 learned_addr learned my addr 172.21.15.84:0/201631439 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:52.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ff9756640 1 --2- 172.21.15.84:0/201631439 >> v2:172.21.15.99:3300/0 conn(0x7f6ff40d2ac0 0x7f6ff40d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932261652 0 0) 0x7f6ff40cf220 con 0x7f6ff40d0940 2023-10-11T06:11:52.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.459+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe0003520 con 0x7f6ff40d0940 2023-10-11T06:11:52.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4057095506 0 0) 0x7f6fe0003520 con 0x7f6ff40d0940 2023-10-11T06:11:52.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 >> v2:172.21.15.99:3300/0 conn(0x7f6ff40d2ac0 msgr2=0x7f6ff40d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:52.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 --2- 172.21.15.84:0/201631439 >> v2:172.21.15.99:3300/0 conn(0x7f6ff40d2ac0 0x7f6ff40d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 >> v1:172.21.15.99:6789/0 conn(0x7f6ff415f7e0 legacy=0x7f6ff40cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40cf7e0 con 0x7f6ff40d0940 2023-10-11T06:11:52.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f6fec0019f0 con 0x7f6ff40d0940 2023-10-11T06:11:52.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 >> v1:172.21.15.84:6789/0 conn(0x7f6ff40d0940 legacy=0x7f6ff40d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6fe00039c0 0x7f6fe0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00047c0 0x7f6fe0004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fe0005170 0x7f6fe000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40d3480 con 0x7f6fe0005170 2023-10-11T06:11:52.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe000c050 con 0x7f6fe00047c0 2023-10-11T06:11:52.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe000c210 con 0x7f6fe00039c0 2023-10-11T06:11:52.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.460+0000 7f6ffa758640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fe0005170 0x7f6fe000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.461+0000 7f6ff9756640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00047c0 0x7f6fe0004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.461+0000 7f6ffa758640 1 -- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6fe00039c0 msgr2=0x7f6fe0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:52.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.461+0000 7f6ffa758640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6fe00039c0 0x7f6fe0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.461+0000 7f6ffa758640 1 -- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00047c0 msgr2=0x7f6fe0004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:52.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.461+0000 7f6ffa758640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00047c0 0x7f6fe0004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.461+0000 7f6ffa758640 1 -- 172.21.15.84:0/201631439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe0003520 con 0x7f6fe0005170 2023-10-11T06:11:52.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.461+0000 7f6ff9756640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00047c0 0x7f6fe0004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:52.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.462+0000 7f6ffa758640 1 -- 172.21.15.84:0/201631439 reap_dead start 2023-10-11T06:11:52.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.462+0000 7f6ffa758640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fe0005170 0x7f6fe000b980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6fec002230 tx=0x7f6fe00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5ba34c03108870 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.462+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec034630 con 0x7f6fe0005170 2023-10-11T06:11:52.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.462+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec02f070 con 0x7f6fe0005170 2023-10-11T06:11:52.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.462+0000 7f6ff8f55640 1 -- 172.21.15.84:0/201631439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0349d0 con 0x7f6fe0005170 2023-10-11T06:11:52.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.463+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/201631439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fe0005170 msgr2=0x7f6fe000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.463+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fe0005170 0x7f6fe000b980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6fec002230 tx=0x7f6fe00057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.463+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/201631439 shutdown_connections 2023-10-11T06:11:52.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.463+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/201631439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fe0005170 0x7f6fe000b980 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.463+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/201631439 >> 172.21.15.84:0/201631439 conn(0x7f6ff40713a0 msgr2=0x7f6ff40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:52.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.463+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/201631439 shutdown_connections 2023-10-11T06:11:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.463+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/201631439 wait complete. 2023-10-11T06:11:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.465+0000 7f6ffc1bb640 1 Processor -- start 2023-10-11T06:11:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.465+0000 7f6ffc1bb640 1 -- start start 2023-10-11T06:11:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffc1bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe0005170 0x7f6ff41511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffc1bb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ff40cc6a0 0x7f6ff40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffc1bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ff415ec90 0x7f6ff4158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffc1bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4151a20 con 0x7f6ff415ec90 2023-10-11T06:11:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffc1bb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40cb1d0 con 0x7f6fe0005170 2023-10-11T06:11:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffc1bb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40cb4b0 con 0x7f6ff40cc6a0 2023-10-11T06:11:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffa758640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ff415ec90 0x7f6ff4158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffa758640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ff415ec90 0x7f6ff4158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56128/0 (socket says 172.21.15.84:56128) 2023-10-11T06:11:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ffa758640 1 -- 172.21.15.84:0/3976642957 learned_addr learned my addr 172.21.15.84:0/3976642957 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ff9f57640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe0005170 0x7f6ff41511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.466+0000 7f6ff9756640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ff40cc6a0 0x7f6ff40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffa758640 1 -- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ff40cc6a0 msgr2=0x7f6ff40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffa758640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ff40cc6a0 0x7f6ff40cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffa758640 1 -- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe0005170 msgr2=0x7f6ff41511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffa758640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe0005170 0x7f6ff41511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffa758640 1 -- 172.21.15.84:0/3976642957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40cb790 con 0x7f6ff415ec90 2023-10-11T06:11:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffa758640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ff415ec90 0x7f6ff4158880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fec002230 tx=0x7f6fec001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6fea7fc640 1 -- 172.21.15.84:0/3976642957 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec004d60 con 0x7f6ff415ec90 2023-10-11T06:11:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6fea7fc640 1 -- 172.21.15.84:0/3976642957 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec03f470 con 0x7f6ff415ec90 2023-10-11T06:11:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6fea7fc640 1 -- 172.21.15.84:0/3976642957 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec03f820 con 0x7f6ff415ec90 2023-10-11T06:11:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff4151710 con 0x7f6ff415ec90 2023-10-11T06:11:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.467+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff40cc1f0 con 0x7f6ff415ec90 2023-10-11T06:11:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.468+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff4073ed0 con 0x7f6ff415ec90 2023-10-11T06:11:52.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.469+0000 7f6fea7fc640 1 -- 172.21.15.84:0/3976642957 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6fec031030 con 0x7f6ff415ec90 2023-10-11T06:11:52.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.470+0000 7f6fea7fc640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6fd806a860 0x7f6fd806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.470+0000 7f6fea7fc640 1 -- 172.21.15.84:0/3976642957 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6fec0aa5e0 con 0x7f6ff415ec90 2023-10-11T06:11:52.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.470+0000 7f6ff9f57640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6fd806a860 0x7f6fd806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.471+0000 7f6ff9f57640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6fd806a860 0x7f6fd806cd10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0001630 tx=0x7f6ff0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.475+0000 7f6fea7fc640 1 -- 172.21.15.84:0/3976642957 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f6fec0758b0 con 0x7f6ff415ec90 2023-10-11T06:11:52.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.661+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ff40b5030 con 0x7f6ff415ec90 2023-10-11T06:11:52.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.662+0000 7f6fea7fc640 1 -- 172.21.15.84:0/3976642957 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fec078ed0 con 0x7f6ff415ec90 2023-10-11T06:11:52.663 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:11:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6fd806a860 msgr2=0x7f6fd806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6fd806a860 0x7f6fd806cd10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0001630 tx=0x7f6ff0009040 comp rx=0 tx=0).stop 2023-10-11T06:11:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ff415ec90 msgr2=0x7f6ff4158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ff415ec90 0x7f6ff4158880 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fec002230 tx=0x7f6fec001cb0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 shutdown_connections 2023-10-11T06:11:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6fd806a860 0x7f6fd806cd10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ff415ec90 0x7f6ff4158880 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ff40cc6a0 0x7f6ff40cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 --2- 172.21.15.84:0/3976642957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe0005170 0x7f6ff41511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.664+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 >> 172.21.15.84:0/3976642957 conn(0x7f6ff40713a0 msgr2=0x7f6ff40cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:52.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.665+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 shutdown_connections 2023-10-11T06:11:52.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.665+0000 7f6ffc1bb640 1 -- 172.21.15.84:0/3976642957 wait complete. 2023-10-11T06:11:52.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:11:52.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.899+0000 7f4f1a567640 1 Processor -- start 2023-10-11T06:11:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.899+0000 7f4f1a567640 1 -- start start 2023-10-11T06:11:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.901+0000 7f4f1a567640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4f1415a250 0x7f4f140ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.901+0000 7f4f1a567640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f140ac820 con 0x7f4f14069c70 2023-10-11T06:11:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.901+0000 7f4f1a567640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1406a9c0 con 0x7f4f140d0100 2023-10-11T06:11:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.901+0000 7f4f1a567640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1415a620 con 0x7f4f1415a250 2023-10-11T06:11:52.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.901+0000 7f4f19d66640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4f14069c70 0x7f4f1406a070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39426/0 (socket says 172.21.15.84:39426) 2023-10-11T06:11:52.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.901+0000 7f4f19d66640 1 -- 172.21.15.84:0/551467653 learned_addr learned my addr 172.21.15.84:0/551467653 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:52.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f19565640 1 --2- 172.21.15.84:0/551467653 >> v2:172.21.15.99:3300/0 conn(0x7f4f1415a250 0x7f4f140ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249688463 0 0) 0x7f4f140ac820 con 0x7f4f14069c70 2023-10-11T06:11:52.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f00003520 con 0x7f4f14069c70 2023-10-11T06:11:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070321945 0 0) 0x7f4f00003520 con 0x7f4f14069c70 2023-10-11T06:11:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 >> v2:172.21.15.99:3300/0 conn(0x7f4f1415a250 msgr2=0x7f4f140ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 --2- 172.21.15.84:0/551467653 >> v2:172.21.15.99:3300/0 conn(0x7f4f1415a250 0x7f4f140ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 >> v1:172.21.15.99:6789/0 conn(0x7f4f140d0100 legacy=0x7f4f140caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140cb290 con 0x7f4f14069c70 2023-10-11T06:11:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f4f0c0019f0 con 0x7f4f14069c70 2023-10-11T06:11:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 >> v1:172.21.15.84:6789/0 conn(0x7f4f14069c70 legacy=0x7f4f1406a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f000039c0 0x7f4f00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f00004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f00005170 0x7f4f0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.902+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1415a620 con 0x7f4f000047c0 2023-10-11T06:11:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0000c030 con 0x7f4f000039c0 2023-10-11T06:11:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0000c1f0 con 0x7f4f00005170 2023-10-11T06:11:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f18d64640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f00004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f19565640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f000039c0 0x7f4f00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f19d66640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f00005170 0x7f4f0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f18d64640 1 -- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f00005170 msgr2=0x7f4f0000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f18d64640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f00005170 0x7f4f0000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f18d64640 1 -- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f000039c0 msgr2=0x7f4f00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f18d64640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f000039c0 0x7f4f00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f18d64640 1 -- 172.21.15.84:0/551467653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f00003520 con 0x7f4f000047c0 2023-10-11T06:11:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.903+0000 7f4f19d66640 1 -- 172.21.15.84:0/551467653 reap_dead start 2023-10-11T06:11:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.904+0000 7f4f18d64640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f00004c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f04001770 tx=0x7f4f000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad0e6d4e6837ecb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040052d0 con 0x7f4f000047c0 2023-10-11T06:11:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0402e050 con 0x7f4f000047c0 2023-10-11T06:11:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f0bfff640 1 -- 172.21.15.84:0/551467653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04003640 con 0x7f4f000047c0 2023-10-11T06:11:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f1a567640 1 -- 172.21.15.84:0/551467653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 msgr2=0x7f4f00004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f1a567640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f00004c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4f04001770 tx=0x7f4f000057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f1a567640 1 -- 172.21.15.84:0/551467653 shutdown_connections 2023-10-11T06:11:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f1a567640 1 --2- 172.21.15.84:0/551467653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f00004c30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f1a567640 1 -- 172.21.15.84:0/551467653 >> 172.21.15.84:0/551467653 conn(0x7f4f1406b6a0 msgr2=0x7f4f140cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f1a567640 1 -- 172.21.15.84:0/551467653 shutdown_connections 2023-10-11T06:11:52.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.905+0000 7f4f1a567640 1 -- 172.21.15.84:0/551467653 wait complete. 2023-10-11T06:11:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.907+0000 7f4f1a567640 1 Processor -- start 2023-10-11T06:11:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.907+0000 7f4f1a567640 1 -- start start 2023-10-11T06:11:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.908+0000 7f4f1a567640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f140accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.908+0000 7f4f1a567640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f140b27a0 0x7f4f14153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.908+0000 7f4f1a567640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f140c1f90 0x7f4f140b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.908+0000 7f4f1a567640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140ad230 con 0x7f4f000047c0 2023-10-11T06:11:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.908+0000 7f4f1a567640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f14154100 con 0x7f4f140c1f90 2023-10-11T06:11:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.908+0000 7f4f1a567640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140ad460 con 0x7f4f140b27a0 2023-10-11T06:11:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f140accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f140accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56144/0 (socket says 172.21.15.84:56144) 2023-10-11T06:11:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 -- 172.21.15.84:0/2344132705 learned_addr learned my addr 172.21.15.84:0/2344132705 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f18d64640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f140b27a0 0x7f4f14153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19d66640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f140c1f90 0x7f4f140b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 -- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f140b27a0 msgr2=0x7f4f14153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f140b27a0 0x7f4f14153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 -- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f140c1f90 msgr2=0x7f4f140b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f140c1f90 0x7f4f140b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 -- 172.21.15.84:0/2344132705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140cd4c0 con 0x7f4f000047c0 2023-10-11T06:11:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.909+0000 7f4f19565640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f140accf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4f100114e0 tx=0x7f4f100115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.910+0000 7f4f09ffb640 1 -- 172.21.15.84:0/2344132705 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10015660 con 0x7f4f000047c0 2023-10-11T06:11:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.910+0000 7f4f09ffb640 1 -- 172.21.15.84:0/2344132705 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f100157c0 con 0x7f4f000047c0 2023-10-11T06:11:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.910+0000 7f4f09ffb640 1 -- 172.21.15.84:0/2344132705 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10015b30 con 0x7f4f000047c0 2023-10-11T06:11:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.910+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f140cd7a0 con 0x7f4f000047c0 2023-10-11T06:11:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.910+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f140b0710 con 0x7f4f000047c0 2023-10-11T06:11:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.911+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1404e890 con 0x7f4f000047c0 2023-10-11T06:11:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.912+0000 7f4f09ffb640 1 -- 172.21.15.84:0/2344132705 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4f10015c90 con 0x7f4f000047c0 2023-10-11T06:11:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.913+0000 7f4f09ffb640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4ef806a860 0x7f4ef806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.913+0000 7f4f09ffb640 1 -- 172.21.15.84:0/2344132705 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4f10089080 con 0x7f4f000047c0 2023-10-11T06:11:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.913+0000 7f4f18d64640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4ef806a860 0x7f4ef806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.914+0000 7f4f18d64640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4ef806a860 0x7f4ef806cd10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4f04033000 tx=0x7f4f04009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:52.918+0000 7f4f09ffb640 1 -- 172.21.15.84:0/2344132705 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4f100542c0 con 0x7f4f000047c0 2023-10-11T06:11:53.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.104+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f1406a7a0 con 0x7f4f000047c0 2023-10-11T06:11:53.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.105+0000 7f4f09ffb640 1 -- 172.21.15.84:0/2344132705 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f100578e0 con 0x7f4f000047c0 2023-10-11T06:11:53.106 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:11:53.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4ef806a860 msgr2=0x7f4ef806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4ef806a860 0x7f4ef806cd10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4f04033000 tx=0x7f4f04009030 comp rx=0 tx=0).stop 2023-10-11T06:11:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 msgr2=0x7f4f140accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f140accf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4f100114e0 tx=0x7f4f100115d0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 shutdown_connections 2023-10-11T06:11:53.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4ef806a860 0x7f4ef806cd10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f140c1f90 0x7f4f140b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f140b27a0 0x7f4f14153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 --2- 172.21.15.84:0/2344132705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f000047c0 0x7f4f140accf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 >> 172.21.15.84:0/2344132705 conn(0x7f4f1406b6a0 msgr2=0x7f4f1406d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.107+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 shutdown_connections 2023-10-11T06:11:53.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.108+0000 7f4f1a567640 1 -- 172.21.15.84:0/2344132705 wait complete. 2023-10-11T06:11:53.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.340+0000 7ff78aa44640 1 Processor -- start 2023-10-11T06:11:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.341+0000 7ff78aa44640 1 -- start start 2023-10-11T06:11:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff78aa44640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff7840d0460 0x7ff78415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff78aa44640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 0x7ff7840b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff78aa44640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78415c910 con 0x7ff7840cf330 2023-10-11T06:11:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff78aa44640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff784160430 con 0x7ff7840d0460 2023-10-11T06:11:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff78aa44640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78415da20 con 0x7ff784160850 2023-10-11T06:11:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff783fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff7840cf330 0x7ff7840cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39440/0 (socket says 172.21.15.84:39440) 2023-10-11T06:11:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff783fff640 1 -- 172.21.15.84:0/1968081357 learned_addr learned my addr 172.21.15.84:0/1968081357 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:53.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.342+0000 7ff77bfff640 1 --2- 172.21.15.84:0/1968081357 >> v2:172.21.15.99:3300/0 conn(0x7ff7840d0460 0x7ff78415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989105654 0 0) 0x7ff774001110 con 0x7ff7840cf330 2023-10-11T06:11:53.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff788fe1640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 0x7ff7840b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff770003520 con 0x7ff7840cf330 2023-10-11T06:11:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2402620827 0 0) 0x7ff770003520 con 0x7ff7840cf330 2023-10-11T06:11:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 msgr2=0x7ff7840b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 0x7ff7840b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 >> v2:172.21.15.99:3300/0 conn(0x7ff7840d0460 msgr2=0x7ff78415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 --2- 172.21.15.84:0/1968081357 >> v2:172.21.15.99:3300/0 conn(0x7ff7840d0460 0x7ff78415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840d08b0 con 0x7ff7840cf330 2023-10-11T06:11:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7ff774001800 con 0x7ff7840cf330 2023-10-11T06:11:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff788fe1640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 0x7ff7840b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:11:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 >> v1:172.21.15.84:6789/0 conn(0x7ff7840cf330 legacy=0x7ff7840cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7700039a0 0x7ff770003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700047c0 0x7ff770004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 0x7ff770005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784160430 con 0x7ff7700039a0 2023-10-11T06:11:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78415da20 con 0x7ff7700047c0 2023-10-11T06:11:53.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000b980 con 0x7ff784160850 2023-10-11T06:11:53.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.343+0000 7ff783fff640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7700039a0 0x7ff770003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.344+0000 7ff783fff640 1 -- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 msgr2=0x7ff770005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:53.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.344+0000 7ff783fff640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff784160850 0x7ff770005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.344+0000 7ff783fff640 1 -- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700047c0 msgr2=0x7ff770004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.344+0000 7ff783fff640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700047c0 0x7ff770004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.344+0000 7ff783fff640 1 -- 172.21.15.84:0/1968081357 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770003520 con 0x7ff7700039a0 2023-10-11T06:11:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.345+0000 7ff783fff640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7700039a0 0x7ff770003db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff77400c270 tx=0x7ff7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5d6628aa4a55cd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.345+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774005ad0 con 0x7ff7700039a0 2023-10-11T06:11:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.345+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774005c50 con 0x7ff7700039a0 2023-10-11T06:11:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.345+0000 7ff7837fe640 1 -- 172.21.15.84:0/1968081357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77402c8a0 con 0x7ff7700039a0 2023-10-11T06:11:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.346+0000 7ff78aa44640 1 -- 172.21.15.84:0/1968081357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7700039a0 msgr2=0x7ff770003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.346+0000 7ff78aa44640 1 --2- 172.21.15.84:0/1968081357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7700039a0 0x7ff770003db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff77400c270 tx=0x7ff7700057f0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.346+0000 7ff788fe1640 1 -- 172.21.15.84:0/1968081357 reap_dead start 2023-10-11T06:11:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.346+0000 7ff78aa44640 1 -- 172.21.15.84:0/1968081357 shutdown_connections 2023-10-11T06:11:53.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.346+0000 7ff78aa44640 1 -- 172.21.15.84:0/1968081357 >> 172.21.15.84:0/1968081357 conn(0x7ff784071cc0 msgr2=0x7ff78409bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:53.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.346+0000 7ff78aa44640 1 -- 172.21.15.84:0/1968081357 shutdown_connections 2023-10-11T06:11:53.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.346+0000 7ff78aa44640 1 -- 172.21.15.84:0/1968081357 wait complete. 2023-10-11T06:11:53.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.347+0000 7ff78aa44640 1 Processor -- start 2023-10-11T06:11:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.348+0000 7ff78aa44640 1 -- start start 2023-10-11T06:11:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff78aa44640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700039a0 0x7ff7840b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff78aa44640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff78415e580 0x7ff78415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff78aa44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7841e6580 0x7ff7841e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff78aa44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784159750 con 0x7ff7841e6580 2023-10-11T06:11:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff78aa44640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840b87b0 con 0x7ff7700039a0 2023-10-11T06:11:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff78aa44640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7841e6f30 con 0x7ff78415e580 2023-10-11T06:11:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7841e6580 0x7ff7841e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7841e6580 0x7ff7841e69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56154/0 (socket says 172.21.15.84:56154) 2023-10-11T06:11:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 -- 172.21.15.84:0/3329382775 learned_addr learned my addr 172.21.15.84:0/3329382775 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:53.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff783fff640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700039a0 0x7ff7840b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff77bfff640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff78415e580 0x7ff78415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 -- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff78415e580 msgr2=0x7ff78415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff78415e580 0x7ff78415e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 -- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700039a0 msgr2=0x7ff7840b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700039a0 0x7ff7840b8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff788fe1640 1 -- 172.21.15.84:0/3329382775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff784159390 con 0x7ff7841e6580 2023-10-11T06:11:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.349+0000 7ff783fff640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700039a0 0x7ff7840b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.350+0000 7ff788fe1640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7841e6580 0x7ff7841e69f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff77c007c40 tx=0x7ff77c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:53.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.350+0000 7ff7817fa640 1 -- 172.21.15.84:0/3329382775 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c003b90 con 0x7ff7841e6580 2023-10-11T06:11:53.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.350+0000 7ff7817fa640 1 -- 172.21.15.84:0/3329382775 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00f040 con 0x7ff7841e6580 2023-10-11T06:11:53.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.350+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78415f000 con 0x7ff7841e6580 2023-10-11T06:11:53.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.350+0000 7ff7817fa640 1 -- 172.21.15.84:0/3329382775 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c004030 con 0x7ff7841e6580 2023-10-11T06:11:53.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.350+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7840b5750 con 0x7ff7841e6580 2023-10-11T06:11:53.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.351+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff784159890 con 0x7ff7841e6580 2023-10-11T06:11:53.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.352+0000 7ff7817fa640 1 -- 172.21.15.84:0/3329382775 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff77c004190 con 0x7ff7841e6580 2023-10-11T06:11:53.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.353+0000 7ff7817fa640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff76406a830 0x7ff76406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.353+0000 7ff7817fa640 1 -- 172.21.15.84:0/3329382775 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff77c088270 con 0x7ff7841e6580 2023-10-11T06:11:53.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.353+0000 7ff783fff640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff76406a830 0x7ff76406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.354+0000 7ff783fff640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff76406a830 0x7ff76406cce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff774034000 tx=0x7ff774009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:53.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.357+0000 7ff7817fa640 1 -- 172.21.15.84:0/3329382775 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff784159890 con 0x7ff7841e6580 2023-10-11T06:11:53.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.543+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff784159890 con 0x7ff7841e6580 2023-10-11T06:11:53.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.545+0000 7ff7817fa640 1 -- 172.21.15.84:0/3329382775 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7ff784159890 con 0x7ff7841e6580 2023-10-11T06:11:53.547 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:53.547 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:53.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff76406a830 msgr2=0x7ff76406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff76406a830 0x7ff76406cce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff774034000 tx=0x7ff774009030 comp rx=0 tx=0).stop 2023-10-11T06:11:53.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7841e6580 msgr2=0x7ff7841e69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7841e6580 0x7ff7841e69f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff77c007c40 tx=0x7ff77c007d30 comp rx=0 tx=0).stop 2023-10-11T06:11:53.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 shutdown_connections 2023-10-11T06:11:53.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff76406a830 0x7ff76406cce0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7841e6580 0x7ff7841e69f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff78415e580 0x7ff78415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 --2- 172.21.15.84:0/3329382775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7700039a0 0x7ff7840b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 >> 172.21.15.84:0/3329382775 conn(0x7ff784071cc0 msgr2=0x7ff7840b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:53.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 shutdown_connections 2023-10-11T06:11:53.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.548+0000 7ff78aa44640 1 -- 172.21.15.84:0/3329382775 wait complete. 2023-10-11T06:11:53.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:53.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:53.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.774+0000 7fd63425f640 1 Processor -- start 2023-10-11T06:11:53.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.774+0000 7fd63425f640 1 -- start start 2023-10-11T06:11:53.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.775+0000 7fd63425f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.775+0000 7fd63425f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd62c150300 0x7fd62c0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.775+0000 7fd63425f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c0c9b90 con 0x7fd62c0cc7c0 2023-10-11T06:11:53.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.775+0000 7fd63425f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c15da10 con 0x7fd62c150300 2023-10-11T06:11:53.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.775+0000 7fd63425f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c15ce00 con 0x7fd62c14ff30 2023-10-11T06:11:53.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.776+0000 7fd6317fa640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd62c150300 0x7fd62c0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.776+0000 7fd631ffb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.776+0000 7fd631ffb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:36216/0 (socket says 172.21.15.84:36216) 2023-10-11T06:11:53.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.776+0000 7fd631ffb640 1 -- 172.21.15.84:0/1572318529 learned_addr learned my addr 172.21.15.84:0/1572318529 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:53.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.777+0000 7fd631ffb640 1 -- 172.21.15.84:0/1572318529 >> v2:172.21.15.99:3300/0 conn(0x7fd62c150300 msgr2=0x7fd62c0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:53.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.777+0000 7fd631ffb640 1 --2- 172.21.15.84:0/1572318529 >> v2:172.21.15.99:3300/0 conn(0x7fd62c150300 0x7fd62c0cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.777+0000 7fd631ffb640 1 -- 172.21.15.84:0/1572318529 >> v1:172.21.15.99:6789/0 conn(0x7fd62c0cc7c0 legacy=0x7fd62c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.777+0000 7fd631ffb640 1 -- 172.21.15.84:0/1572318529 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c15d0e0 con 0x7fd62c14ff30 2023-10-11T06:11:53.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.778+0000 7fd631ffb640 1 --2- 172.21.15.84:0/1572318529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0c9650 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd62800bb40 tx=0x7fd62802e0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd39ee6ff7e7fc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:53.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd630ff9640 1 -- 172.21.15.84:0/1572318529 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62802ee70 con 0x7fd62c14ff30 2023-10-11T06:11:53.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd630ff9640 1 -- 172.21.15.84:0/1572318529 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628036440 con 0x7fd62c14ff30 2023-10-11T06:11:53.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd630ff9640 1 -- 172.21.15.84:0/1572318529 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280367d0 con 0x7fd62c14ff30 2023-10-11T06:11:53.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 -- 172.21.15.84:0/1572318529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 msgr2=0x7fd62c0c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 --2- 172.21.15.84:0/1572318529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0c9650 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd62800bb40 tx=0x7fd62802e0c0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 -- 172.21.15.84:0/1572318529 shutdown_connections 2023-10-11T06:11:53.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 --2- 172.21.15.84:0/1572318529 >> v2:172.21.15.99:3300/0 conn(0x7fd62c150300 0x7fd62c0cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 --2- 172.21.15.84:0/1572318529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0c9650 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 -- 172.21.15.84:0/1572318529 >> 172.21.15.84:0/1572318529 conn(0x7fd62c069c70 msgr2=0x7fd62c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:53.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 -- 172.21.15.84:0/1572318529 shutdown_connections 2023-10-11T06:11:53.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.779+0000 7fd63425f640 1 -- 172.21.15.84:0/1572318529 wait complete. 2023-10-11T06:11:53.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.781+0000 7fd63425f640 1 Processor -- start 2023-10-11T06:11:53.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.781+0000 7fd63425f640 1 -- start start 2023-10-11T06:11:53.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.782+0000 7fd63425f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.782+0000 7fd63425f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd62c150300 0x7fd62c14aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.782+0000 7fd63425f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd62c14bf60 0x7fd62c1f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.782+0000 7fd63425f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c15c810 con 0x7fd62c14bf60 2023-10-11T06:11:53.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.782+0000 7fd63425f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c15abb0 con 0x7fd62c150300 2023-10-11T06:11:53.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.782+0000 7fd63425f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c14b480 con 0x7fd62c14ff30 2023-10-11T06:11:53.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd62c14bf60 0x7fd62c1f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd62c14bf60 0x7fd62c1f0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56164/0 (socket says 172.21.15.84:56164) 2023-10-11T06:11:53.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 -- 172.21.15.84:0/926531237 learned_addr learned my addr 172.21.15.84:0/926531237 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:53.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd631ffb640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6317fa640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd62c150300 0x7fd62c14aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 -- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 msgr2=0x7fd62c0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 -- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd62c150300 msgr2=0x7fd62c14aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd62c150300 0x7fd62c14aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.783+0000 7fd6327fc640 1 -- 172.21.15.84:0/926531237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62800b770 con 0x7fd62c14bf60 2023-10-11T06:11:53.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.784+0000 7fd6327fc640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd62c14bf60 0x7fd62c1f0db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c6c0 tx=0x7fd62400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:53.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.784+0000 7fd6227fc640 1 -- 172.21.15.84:0/926531237 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624003b70 con 0x7fd62c14bf60 2023-10-11T06:11:53.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.784+0000 7fd6227fc640 1 -- 172.21.15.84:0/926531237 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd624003cd0 con 0x7fd62c14bf60 2023-10-11T06:11:53.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.784+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c14b760 con 0x7fd62c14bf60 2023-10-11T06:11:53.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.784+0000 7fd6227fc640 1 -- 172.21.15.84:0/926531237 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6240052f0 con 0x7fd62c14bf60 2023-10-11T06:11:53.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.784+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c15ae90 con 0x7fd62c14bf60 2023-10-11T06:11:53.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.785+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c14bc80 con 0x7fd62c14bf60 2023-10-11T06:11:53.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.786+0000 7fd6227fc640 1 -- 172.21.15.84:0/926531237 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd624005450 con 0x7fd62c14bf60 2023-10-11T06:11:53.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.787+0000 7fd6227fc640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd61006a830 0x7fd61006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:53.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.787+0000 7fd6227fc640 1 -- 172.21.15.84:0/926531237 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd624088e20 con 0x7fd62c14bf60 2023-10-11T06:11:53.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.787+0000 7fd631ffb640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd61006a830 0x7fd61006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:53.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.787+0000 7fd631ffb640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd61006a830 0x7fd61006cce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd628038000 tx=0x7fd628032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:53.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.791+0000 7fd6227fc640 1 -- 172.21.15.84:0/926531237 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd62c14bc80 con 0x7fd62c14bf60 2023-10-11T06:11:53.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.980+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd62c14bc80 con 0x7fd62c14bf60 2023-10-11T06:11:53.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.981+0000 7fd6227fc640 1 -- 172.21.15.84:0/926531237 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd62c14bc80 con 0x7fd62c14bf60 2023-10-11T06:11:53.982 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.983+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd61006a830 msgr2=0x7fd61006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.983+0000 7fd63425f640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd61006a830 0x7fd61006cce0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd628038000 tx=0x7fd628032000 comp rx=0 tx=0).stop 2023-10-11T06:11:53.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.983+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd62c14bf60 msgr2=0x7fd62c1f0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.983+0000 7fd63425f640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd62c14bf60 0x7fd62c1f0db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c6c0 tx=0x7fd62400cb80 comp rx=0 tx=0).stop 2023-10-11T06:11:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 shutdown_connections 2023-10-11T06:11:53.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd61006a830 0x7fd61006cce0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd62c14bf60 0x7fd62c1f0db0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd62c150300 0x7fd62c14aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 --2- 172.21.15.84:0/926531237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd62c14ff30 0x7fd62c0ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:53.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 >> 172.21.15.84:0/926531237 conn(0x7fd62c069c70 msgr2=0x7fd62c06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:53.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 shutdown_connections 2023-10-11T06:11:53.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:53.984+0000 7fd63425f640 1 -- 172.21.15.84:0/926531237 wait complete. 2023-10-11T06:11:54.004 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:54.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.211+0000 7f544f59e640 1 Processor -- start 2023-10-11T06:11:54.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.211+0000 7f544f59e640 1 -- start start 2023-10-11T06:11:54.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544f59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54500b1d50 0x7f54500d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54500d0b90 0x7f54500cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544f59e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54500ccf00 con 0x7f545006bc40 2023-10-11T06:11:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544f59e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54500b2120 con 0x7f54500d0b90 2023-10-11T06:11:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544f59e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54500d2e90 con 0x7f54500b1d50 2023-10-11T06:11:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54500d0b90 0x7f54500cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54500d0b90 0x7f54500cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56178/0 (socket says 172.21.15.84:56178) 2023-10-11T06:11:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544ed9d640 1 -- 172.21.15.84:0/2771552325 learned_addr learned my addr 172.21.15.84:0/2771552325 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544e59c640 1 --2- 172.21.15.84:0/2771552325 >> v2:172.21.15.99:3300/0 conn(0x7f54500b1d50 0x7f54500d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544ed9d640 1 -- 172.21.15.84:0/2771552325 >> v2:172.21.15.99:3300/0 conn(0x7f54500b1d50 msgr2=0x7f54500d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544ed9d640 1 --2- 172.21.15.84:0/2771552325 >> v2:172.21.15.99:3300/0 conn(0x7f54500b1d50 0x7f54500d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544ed9d640 1 -- 172.21.15.84:0/2771552325 >> v1:172.21.15.99:6789/0 conn(0x7f545006bc40 legacy=0x7f54500d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:54.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.213+0000 7f544ed9d640 1 -- 172.21.15.84:0/2771552325 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500d3170 con 0x7f54500d0b90 2023-10-11T06:11:54.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.215+0000 7f544ed9d640 1 --2- 172.21.15.84:0/2771552325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54500d0b90 0x7f54500cd950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f545008bbb0 tx=0x7f544802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbfefb34527486 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:54.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.215+0000 7f544d59a640 1 -- 172.21.15.84:0/2771552325 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448030860 con 0x7f54500d0b90 2023-10-11T06:11:54.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.215+0000 7f544d59a640 1 -- 172.21.15.84:0/2771552325 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54480309c0 con 0x7f54500d0b90 2023-10-11T06:11:54.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.215+0000 7f544d59a640 1 -- 172.21.15.84:0/2771552325 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448030d50 con 0x7f54500d0b90 2023-10-11T06:11:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.215+0000 7f544f59e640 1 -- 172.21.15.84:0/2771552325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54500d0b90 msgr2=0x7f54500cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.215+0000 7f544f59e640 1 --2- 172.21.15.84:0/2771552325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54500d0b90 0x7f54500cd950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f545008bbb0 tx=0x7f544802b9a0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.216+0000 7f544f59e640 1 -- 172.21.15.84:0/2771552325 shutdown_connections 2023-10-11T06:11:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.216+0000 7f544f59e640 1 --2- 172.21.15.84:0/2771552325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54500d0b90 0x7f54500cd950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.216+0000 7f544f59e640 1 --2- 172.21.15.84:0/2771552325 >> v2:172.21.15.99:3300/0 conn(0x7f54500b1d50 0x7f54500d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.216+0000 7f544f59e640 1 -- 172.21.15.84:0/2771552325 >> 172.21.15.84:0/2771552325 conn(0x7f5450089ff0 msgr2=0x7f545009bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:54.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.216+0000 7f544f59e640 1 -- 172.21.15.84:0/2771552325 shutdown_connections 2023-10-11T06:11:54.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.216+0000 7f544f59e640 1 -- 172.21.15.84:0/2771552325 wait complete. 2023-10-11T06:11:54.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.217+0000 7f544f59e640 1 Processor -- start 2023-10-11T06:11:54.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.217+0000 7f544f59e640 1 -- start start 2023-10-11T06:11:54.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.218+0000 7f544f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f545006bc40 0x7f54501f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.218+0000 7f544f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54500b1d50 0x7f54500b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.218+0000 7f544f59e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54500d0b90 0x7f54500cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.218+0000 7f544f59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54500b6a10 con 0x7f545006bc40 2023-10-11T06:11:54.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.218+0000 7f544f59e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54500b6cf0 con 0x7f54500b1d50 2023-10-11T06:11:54.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.218+0000 7f544f59e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54500b6fd0 con 0x7f54500d0b90 2023-10-11T06:11:54.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f545006bc40 0x7f54501f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f545006bc40 0x7f54501f1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56182/0 (socket says 172.21.15.84:56182) 2023-10-11T06:11:54.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 -- 172.21.15.84:0/1051401892 learned_addr learned my addr 172.21.15.84:0/1051401892 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:54.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 -- 172.21.15.84:0/1051401892 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54500d0b90 msgr2=0x7f54500cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54500d0b90 0x7f54500cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 -- 172.21.15.84:0/1051401892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54500b1d50 msgr2=0x7f54500b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54500b1d50 0x7f54500b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 -- 172.21.15.84:0/1051401892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480094d0 con 0x7f545006bc40 2023-10-11T06:11:54.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f544e59c640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f545006bc40 0x7f54501f1130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5440009fc0 tx=0x7f544000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:54.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.219+0000 7f5446ffd640 1 -- 172.21.15.84:0/1051401892 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440014620 con 0x7f545006bc40 2023-10-11T06:11:54.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.220+0000 7f5446ffd640 1 -- 172.21.15.84:0/1051401892 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54400147a0 con 0x7f545006bc40 2023-10-11T06:11:54.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.220+0000 7f5446ffd640 1 -- 172.21.15.84:0/1051401892 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440014b50 con 0x7f545006bc40 2023-10-11T06:11:54.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.220+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54500b72b0 con 0x7f545006bc40 2023-10-11T06:11:54.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.220+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54501f1670 con 0x7f545006bc40 2023-10-11T06:11:54.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.221+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54500c6f20 con 0x7f545006bc40 2023-10-11T06:11:54.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.222+0000 7f5446ffd640 1 -- 172.21.15.84:0/1051401892 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5440014cb0 con 0x7f545006bc40 2023-10-11T06:11:54.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.223+0000 7f5446ffd640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f543006a830 0x7f543006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.223+0000 7f5446ffd640 1 -- 172.21.15.84:0/1051401892 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5440089b20 con 0x7f545006bc40 2023-10-11T06:11:54.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.223+0000 7f544dd9b640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f543006a830 0x7f543006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.224+0000 7f544dd9b640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f543006a830 0x7f543006cce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54500ce9f0 tx=0x7f5438006ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:54.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.227+0000 7f5446ffd640 1 -- 172.21.15.84:0/1051401892 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f54500c6f20 con 0x7f545006bc40 2023-10-11T06:11:54.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.401+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f54500c6f20 con 0x7f545006bc40 2023-10-11T06:11:54.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.402+0000 7f5446ffd640 1 -- 172.21.15.84:0/1051401892 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f54500c6f20 con 0x7f545006bc40 2023-10-11T06:11:54.403 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:54.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f543006a830 msgr2=0x7f543006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f543006a830 0x7f543006cce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54500ce9f0 tx=0x7f5438006ba0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f545006bc40 msgr2=0x7f54501f1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f545006bc40 0x7f54501f1130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5440009fc0 tx=0x7f544000e570 comp rx=0 tx=0).stop 2023-10-11T06:11:54.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 shutdown_connections 2023-10-11T06:11:54.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f543006a830 0x7f543006cce0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54500d0b90 0x7f54500cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54500b1d50 0x7f54500b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 --2- 172.21.15.84:0/1051401892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f545006bc40 0x7f54501f1130 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 >> 172.21.15.84:0/1051401892 conn(0x7f5450089ff0 msgr2=0x7f54500b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:54.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 shutdown_connections 2023-10-11T06:11:54.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.405+0000 7f544f59e640 1 -- 172.21.15.84:0/1051401892 wait complete. 2023-10-11T06:11:54.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:11:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.635+0000 7fdff5300640 1 Processor -- start 2023-10-11T06:11:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.635+0000 7fdff5300640 1 -- start start 2023-10-11T06:11:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdff5300640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdff015a0b0 0x7fdff00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdff5300640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdff00c8d00 0x7fdff00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdff5300640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00ceb60 0x7fdff006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdff5300640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff006e0f0 con 0x7fdff00c8d00 2023-10-11T06:11:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdff5300640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00cbae0 con 0x7fdff015a0b0 2023-10-11T06:11:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdff5300640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00cefc0 con 0x7fdff00ceb60 2023-10-11T06:11:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdfe67fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdff00c8d00 0x7fdff00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdfe67fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdff00c8d00 0x7fdff00ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56188/0 (socket says 172.21.15.84:56188) 2023-10-11T06:11:54.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdfe67fc640 1 -- 172.21.15.84:0/1598971926 learned_addr learned my addr 172.21.15.84:0/1598971926 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:54.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdfef7fe640 1 --2- 172.21.15.84:0/1598971926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00ceb60 0x7fdff006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.637+0000 7fdfeeffd640 1 --2- 172.21.15.84:0/1598971926 >> v2:172.21.15.99:3300/0 conn(0x7fdff015a0b0 0x7fdff00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.638+0000 7fdfe67fc640 1 -- 172.21.15.84:0/1598971926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00ceb60 msgr2=0x7fdff006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.638+0000 7fdfe67fc640 1 --2- 172.21.15.84:0/1598971926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00ceb60 0x7fdff006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.638+0000 7fdfe67fc640 1 -- 172.21.15.84:0/1598971926 >> v2:172.21.15.99:3300/0 conn(0x7fdff015a0b0 msgr2=0x7fdff00cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.638+0000 7fdfe67fc640 1 --2- 172.21.15.84:0/1598971926 >> v2:172.21.15.99:3300/0 conn(0x7fdff015a0b0 0x7fdff00cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.638+0000 7fdfe67fc640 1 -- 172.21.15.84:0/1598971926 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff0071380 con 0x7fdff00c8d00 2023-10-11T06:11:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.638+0000 7fdfeeffd640 1 --2- 172.21.15.84:0/1598971926 >> v2:172.21.15.99:3300/0 conn(0x7fdff015a0b0 0x7fdff00cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.639+0000 7fdfe67fc640 1 --2- 172.21.15.84:0/1598971926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff00c8d00 0x7fdff00ce620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdff006ad30 tx=0x7fdfd802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee9aaff620e8a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.639+0000 7fdfee7fc640 1 -- 172.21.15.84:0/1598971926 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8031740 con 0x7fdff00c8d00 2023-10-11T06:11:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.639+0000 7fdfee7fc640 1 -- 172.21.15.84:0/1598971926 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd80318a0 con 0x7fdff00c8d00 2023-10-11T06:11:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.639+0000 7fdfee7fc640 1 -- 172.21.15.84:0/1598971926 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8031c30 con 0x7fdff00c8d00 2023-10-11T06:11:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 -- 172.21.15.84:0/1598971926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff00c8d00 msgr2=0x7fdff00ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 --2- 172.21.15.84:0/1598971926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff00c8d00 0x7fdff00ce620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdff006ad30 tx=0x7fdfd802b7e0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 -- 172.21.15.84:0/1598971926 shutdown_connections 2023-10-11T06:11:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 --2- 172.21.15.84:0/1598971926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff00c8d00 0x7fdff00ce620 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 --2- 172.21.15.84:0/1598971926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00ceb60 0x7fdff006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 --2- 172.21.15.84:0/1598971926 >> v2:172.21.15.99:3300/0 conn(0x7fdff015a0b0 0x7fdff00cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 -- 172.21.15.84:0/1598971926 >> 172.21.15.84:0/1598971926 conn(0x7fdff0069c70 msgr2=0x7fdff006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 -- 172.21.15.84:0/1598971926 shutdown_connections 2023-10-11T06:11:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.640+0000 7fdff5300640 1 -- 172.21.15.84:0/1598971926 wait complete. 2023-10-11T06:11:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.642+0000 7fdff5300640 1 Processor -- start 2023-10-11T06:11:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.642+0000 7fdff5300640 1 -- start start 2023-10-11T06:11:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdff5300640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00c8d00 0x7fdff01f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdff5300640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff00ceb60 0x7fdff00b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdff5300640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff015a0b0 0x7fdff00c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdff5300640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff01f1860 con 0x7fdff015a0b0 2023-10-11T06:11:54.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdff5300640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff0151930 con 0x7fdff00ceb60 2023-10-11T06:11:54.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdff5300640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b2800 con 0x7fdff00c8d00 2023-10-11T06:11:54.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff015a0b0 0x7fdff00c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff015a0b0 0x7fdff00c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56198/0 (socket says 172.21.15.84:56198) 2023-10-11T06:11:54.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 -- 172.21.15.84:0/2274672368 learned_addr learned my addr 172.21.15.84:0/2274672368 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:54.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfeeffd640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00c8d00 0x7fdff01f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfe67fc640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff00ceb60 0x7fdff00b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 -- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00c8d00 msgr2=0x7fdff01f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00c8d00 0x7fdff01f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 -- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff00ceb60 msgr2=0x7fdff00b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff00ceb60 0x7fdff00b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.643+0000 7fdfef7fe640 1 -- 172.21.15.84:0/2274672368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd80094d0 con 0x7fdff015a0b0 2023-10-11T06:11:54.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.644+0000 7fdfef7fe640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff015a0b0 0x7fdff00c72c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8007e60 tx=0x7fdfe800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:54.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.644+0000 7fdfe7fff640 1 -- 172.21.15.84:0/2274672368 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8014620 con 0x7fdff015a0b0 2023-10-11T06:11:54.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.644+0000 7fdfe7fff640 1 -- 172.21.15.84:0/2274672368 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe800ee90 con 0x7fdff015a0b0 2023-10-11T06:11:54.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.644+0000 7fdfe7fff640 1 -- 172.21.15.84:0/2274672368 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80149e0 con 0x7fdff015a0b0 2023-10-11T06:11:54.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.644+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff00b2ae0 con 0x7fdff015a0b0 2023-10-11T06:11:54.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.644+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff00b30a0 con 0x7fdff015a0b0 2023-10-11T06:11:54.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.645+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff0151700 con 0x7fdff015a0b0 2023-10-11T06:11:54.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.646+0000 7fdfe7fff640 1 -- 172.21.15.84:0/2274672368 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdfe8014b40 con 0x7fdff015a0b0 2023-10-11T06:11:54.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.647+0000 7fdfe7fff640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd006a830 0x7fdfd006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:54.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.647+0000 7fdfe7fff640 1 -- 172.21.15.84:0/2274672368 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdfe80898d0 con 0x7fdff015a0b0 2023-10-11T06:11:54.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.647+0000 7fdfeeffd640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd006a830 0x7fdfd006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:54.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.648+0000 7fdfeeffd640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd006a830 0x7fdfd006cce0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000a950 tx=0x7fdfe0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:54.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.651+0000 7fdfe7fff640 1 -- 172.21.15.84:0/2274672368 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdff0151700 con 0x7fdff015a0b0 2023-10-11T06:11:54.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.825+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdff0070da0 con 0x7fdff015a0b0 2023-10-11T06:11:54.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.825+0000 7fdfe7fff640 1 -- 172.21.15.84:0/2274672368 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdff0070da0 con 0x7fdff015a0b0 2023-10-11T06:11:54.827 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:11:54.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd006a830 msgr2=0x7fdfd006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd006a830 0x7fdfd006cce0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000a950 tx=0x7fdfe0008040 comp rx=0 tx=0).stop 2023-10-11T06:11:54.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff015a0b0 msgr2=0x7fdff00c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:54.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff015a0b0 0x7fdff00c72c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8007e60 tx=0x7fdfe800e4b0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 shutdown_connections 2023-10-11T06:11:54.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd006a830 0x7fdfd006cce0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff015a0b0 0x7fdff00c72c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff00ceb60 0x7fdff00b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 --2- 172.21.15.84:0/2274672368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff00c8d00 0x7fdff01f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:54.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 >> 172.21.15.84:0/2274672368 conn(0x7fdff0069c70 msgr2=0x7fdff00c35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:54.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 shutdown_connections 2023-10-11T06:11:54.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:54.828+0000 7fdff5300640 1 -- 172.21.15.84:0/2274672368 wait complete. 2023-10-11T06:11:54.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:11:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.047+0000 7fe4248a4640 1 Processor -- start 2023-10-11T06:11:55.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.047+0000 7fe4248a4640 1 -- start start 2023-10-11T06:11:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.049+0000 7fe4248a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d2ac0 0x7fe4200d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.049+0000 7fe4248a4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4200d0940 0x7fe4200d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.049+0000 7fe4248a4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4200d3450 con 0x7fe42015f7e0 2023-10-11T06:11:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.049+0000 7fe4248a4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4200cf220 con 0x7fe4200d0940 2023-10-11T06:11:55.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.049+0000 7fe4248a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200cf500 con 0x7fe4200d2ac0 2023-10-11T06:11:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41ed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4200d0940 0x7fe4200d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41ed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4200d0940 0x7fe4200d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:46572/0 (socket says 172.21.15.84:46572) 2023-10-11T06:11:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41ed9d640 1 -- 172.21.15.84:0/1995676083 learned_addr learned my addr 172.21.15.84:0/1995676083 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:55.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41dd9b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d2ac0 0x7fe4200d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41dd9b640 1 -- 172.21.15.84:0/1995676083 >> v2:172.21.15.99:3300/0 conn(0x7fe4200d0940 msgr2=0x7fe4200d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41dd9b640 1 --2- 172.21.15.84:0/1995676083 >> v2:172.21.15.99:3300/0 conn(0x7fe4200d0940 0x7fe4200d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41dd9b640 1 -- 172.21.15.84:0/1995676083 >> v1:172.21.15.99:6789/0 conn(0x7fe42015f7e0 legacy=0x7fe4200cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.050+0000 7fe41dd9b640 1 -- 172.21.15.84:0/1995676083 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200cf7e0 con 0x7fe4200d2ac0 2023-10-11T06:11:55.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.052+0000 7fe41dd9b640 1 --2- 172.21.15.84:0/1995676083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d2ac0 0x7fe4200d2f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4140097a0 tx=0x7fe41402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67e9c470a955080 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.052+0000 7fe41d59a640 1 -- 172.21.15.84:0/1995676083 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41402d040 con 0x7fe4200d2ac0 2023-10-11T06:11:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.052+0000 7fe41d59a640 1 -- 172.21.15.84:0/1995676083 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe414031810 con 0x7fe4200d2ac0 2023-10-11T06:11:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.052+0000 7fe41d59a640 1 -- 172.21.15.84:0/1995676083 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414031ba0 con 0x7fe4200d2ac0 2023-10-11T06:11:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 -- 172.21.15.84:0/1995676083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d2ac0 msgr2=0x7fe4200d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 --2- 172.21.15.84:0/1995676083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d2ac0 0x7fe4200d2f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4140097a0 tx=0x7fe41402b8b0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 -- 172.21.15.84:0/1995676083 shutdown_connections 2023-10-11T06:11:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 --2- 172.21.15.84:0/1995676083 >> v2:172.21.15.99:3300/0 conn(0x7fe4200d0940 0x7fe4200d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 --2- 172.21.15.84:0/1995676083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d2ac0 0x7fe4200d2f10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 -- 172.21.15.84:0/1995676083 >> 172.21.15.84:0/1995676083 conn(0x7fe4200713a0 msgr2=0x7fe4200717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 -- 172.21.15.84:0/1995676083 shutdown_connections 2023-10-11T06:11:55.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.053+0000 7fe4248a4640 1 -- 172.21.15.84:0/1995676083 wait complete. 2023-10-11T06:11:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.055+0000 7fe4248a4640 1 Processor -- start 2023-10-11T06:11:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.055+0000 7fe4248a4640 1 -- start start 2023-10-11T06:11:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe4248a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d0940 0x7fe4201f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe4248a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4200d2ac0 0x7fe4200c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe4248a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe42015f7e0 0x7fe4200b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe4248a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42014c060 con 0x7fe4200d2ac0 2023-10-11T06:11:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe4248a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200b5960 con 0x7fe42015f7e0 2023-10-11T06:11:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe4248a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201f59d0 con 0x7fe4200d0940 2023-10-11T06:11:55.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4200d2ac0 0x7fe4200c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4200d2ac0 0x7fe4200c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56210/0 (socket says 172.21.15.84:56210) 2023-10-11T06:11:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 -- 172.21.15.84:0/817070998 learned_addr learned my addr 172.21.15.84:0/817070998 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41ed9d640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe42015f7e0 0x7fe4200b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41e59c640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d0940 0x7fe4201f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 -- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d0940 msgr2=0x7fe4201f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d0940 0x7fe4201f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 -- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe42015f7e0 msgr2=0x7fe4200b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe42015f7e0 0x7fe4200b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41dd9b640 1 -- 172.21.15.84:0/817070998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414009400 con 0x7fe4200d2ac0 2023-10-11T06:11:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.056+0000 7fe41ed9d640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe42015f7e0 0x7fe4200b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:55.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.057+0000 7fe41dd9b640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4200d2ac0 0x7fe4200c8aa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098c0 tx=0x7fe4140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.057+0000 7fe40effd640 1 -- 172.21.15.84:0/817070998 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414006e10 con 0x7fe4200d2ac0 2023-10-11T06:11:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.057+0000 7fe40effd640 1 -- 172.21.15.84:0/817070998 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41403ad60 con 0x7fe4200d2ac0 2023-10-11T06:11:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.057+0000 7fe40effd640 1 -- 172.21.15.84:0/817070998 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414044620 con 0x7fe4200d2ac0 2023-10-11T06:11:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.057+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4201f5cb0 con 0x7fe4200d2ac0 2023-10-11T06:11:55.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.057+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4200b5c00 con 0x7fe4200d2ac0 2023-10-11T06:11:55.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.058+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4200b5df0 con 0x7fe4200d2ac0 2023-10-11T06:11:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.059+0000 7fe40effd640 1 -- 172.21.15.84:0/817070998 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe41404d440 con 0x7fe4200d2ac0 2023-10-11T06:11:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.060+0000 7fe40effd640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe40006a830 0x7fe40006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.060+0000 7fe40effd640 1 -- 172.21.15.84:0/817070998 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe4140ab430 con 0x7fe4200d2ac0 2023-10-11T06:11:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.060+0000 7fe41e59c640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe40006a830 0x7fe40006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.061+0000 7fe41e59c640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe40006a830 0x7fe40006cce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe410000f60 tx=0x7fe410009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.064+0000 7fe40effd640 1 -- 172.21.15.84:0/817070998 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe4140765d0 con 0x7fe4200d2ac0 2023-10-11T06:11:55.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.252+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe420073ed0 con 0x7fe4200d2ac0 2023-10-11T06:11:55.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.253+0000 7fe40effd640 1 -- 172.21.15.84:0/817070998 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe414079bf0 con 0x7fe4200d2ac0 2023-10-11T06:11:55.254 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:11:55.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe40006a830 msgr2=0x7fe40006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe40006a830 0x7fe40006cce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe410000f60 tx=0x7fe410009040 comp rx=0 tx=0).stop 2023-10-11T06:11:55.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4200d2ac0 msgr2=0x7fe4200c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4200d2ac0 0x7fe4200c8aa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4140098c0 tx=0x7fe4140046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 shutdown_connections 2023-10-11T06:11:55.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe40006a830 0x7fe40006cce0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe42015f7e0 0x7fe4200b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4200d2ac0 0x7fe4200c8aa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 --2- 172.21.15.84:0/817070998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4200d0940 0x7fe4201f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.255+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 >> 172.21.15.84:0/817070998 conn(0x7fe4200713a0 msgr2=0x7fe420159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:55.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.256+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 shutdown_connections 2023-10-11T06:11:55.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.256+0000 7fe4248a4640 1 -- 172.21.15.84:0/817070998 wait complete. 2023-10-11T06:11:55.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.486+0000 7f8fbd3bf640 1 Processor -- start 2023-10-11T06:11:55.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.486+0000 7f8fbd3bf640 1 -- start start 2023-10-11T06:11:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.487+0000 7f8fbd3bf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8fb80c9330 0x7f8fb8158730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.487+0000 7f8fbd3bf640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb806c3d0 0x7f8fb8158c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.487+0000 7f8fbd3bf640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb80adc40 con 0x7f8fb806c000 2023-10-11T06:11:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.487+0000 7f8fbd3bf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb81591b0 con 0x7f8fb80c9330 2023-10-11T06:11:55.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.487+0000 7f8fbd3bf640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80c97e0 con 0x7f8fb806c3d0 2023-10-11T06:11:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8faffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8fb80c9330 0x7f8fb8158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8faffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8fb80c9330 0x7f8fb8158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56220/0 (socket says 172.21.15.84:56220) 2023-10-11T06:11:55.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8faffff640 1 -- 172.21.15.84:0/222101059 learned_addr learned my addr 172.21.15.84:0/222101059 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8fbcbbe640 1 --2- 172.21.15.84:0/222101059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb806c3d0 0x7f8fb8158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8faffff640 1 -- 172.21.15.84:0/222101059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb806c3d0 msgr2=0x7f8fb8158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8faffff640 1 --2- 172.21.15.84:0/222101059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb806c3d0 0x7f8fb8158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8faffff640 1 -- 172.21.15.84:0/222101059 >> v1:172.21.15.99:6789/0 conn(0x7f8fb806c000 legacy=0x7f8fb80c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.488+0000 7f8faffff640 1 -- 172.21.15.84:0/222101059 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80c9ac0 con 0x7f8fb80c9330 2023-10-11T06:11:55.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.489+0000 7f8faffff640 1 --2- 172.21.15.84:0/222101059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb80c9330 0x7f8fb8158730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009870 tx=0x7f8fa002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ed2525e68f861a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.490+0000 7f8fb77fe640 1 -- 172.21.15.84:0/222101059 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0030840 con 0x7f8fb80c9330 2023-10-11T06:11:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.490+0000 7f8fb77fe640 1 -- 172.21.15.84:0/222101059 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa00309a0 con 0x7f8fb80c9330 2023-10-11T06:11:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.490+0000 7f8fb77fe640 1 -- 172.21.15.84:0/222101059 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0030d30 con 0x7f8fb80c9330 2023-10-11T06:11:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.490+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/222101059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb80c9330 msgr2=0x7f8fb8158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.490+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/222101059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb80c9330 0x7f8fb8158730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009870 tx=0x7f8fa002b980 comp rx=0 tx=0).stop 2023-10-11T06:11:55.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.491+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/222101059 shutdown_connections 2023-10-11T06:11:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.491+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/222101059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb806c3d0 0x7f8fb8158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.491+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/222101059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb80c9330 0x7f8fb8158730 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.491+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/222101059 >> 172.21.15.84:0/222101059 conn(0x7f8fb8069c70 msgr2=0x7f8fb806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.491+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/222101059 shutdown_connections 2023-10-11T06:11:55.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.491+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/222101059 wait complete. 2023-10-11T06:11:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.493+0000 7f8fbd3bf640 1 Processor -- start 2023-10-11T06:11:55.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.493+0000 7f8fbd3bf640 1 -- start start 2023-10-11T06:11:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.494+0000 7f8fbd3bf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb806c000 0x7f8fb814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fbd3bf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb806c3d0 0x7f8fb80c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fbd3bf640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb80c9330 0x7f8fb80c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fbd3bf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb814d3e0 con 0x7f8fb806c000 2023-10-11T06:11:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fbd3bf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb814e8a0 con 0x7f8fb806c3d0 2023-10-11T06:11:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fbd3bf640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8163ca0 con 0x7f8fb80c9330 2023-10-11T06:11:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb806c000 0x7f8fb814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb806c000 0x7f8fb814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56228/0 (socket says 172.21.15.84:56228) 2023-10-11T06:11:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 -- 172.21.15.84:0/3103339756 learned_addr learned my addr 172.21.15.84:0/3103339756 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8faffff640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb806c3d0 0x7f8fb80c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 -- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb80c9330 msgr2=0x7f8fb80c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb80c9330 0x7f8fb80c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 -- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb806c3d0 msgr2=0x7f8fb80c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb806c3d0 0x7f8fb80c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8fb7fff640 1 -- 172.21.15.84:0/3103339756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa00094d0 con 0x7f8fb806c000 2023-10-11T06:11:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.495+0000 7f8faffff640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb806c3d0 0x7f8fb80c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.496+0000 7f8fb7fff640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb806c000 0x7f8fb814cea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c430 tx=0x7f8fa800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.496+0000 7f8fb57fa640 1 -- 172.21.15.84:0/3103339756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80052d0 con 0x7f8fb806c000 2023-10-11T06:11:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.496+0000 7f8fb57fa640 1 -- 172.21.15.84:0/3103339756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8005450 con 0x7f8fb806c000 2023-10-11T06:11:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.496+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb80ac820 con 0x7f8fb806c000 2023-10-11T06:11:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.496+0000 7f8fb57fa640 1 -- 172.21.15.84:0/3103339756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8007260 con 0x7f8fb806c000 2023-10-11T06:11:55.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.496+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb80acde0 con 0x7f8fb806c000 2023-10-11T06:11:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.497+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb804e890 con 0x7f8fb806c000 2023-10-11T06:11:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.498+0000 7f8fb57fa640 1 -- 172.21.15.84:0/3103339756 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8fa80073c0 con 0x7f8fb806c000 2023-10-11T06:11:55.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.499+0000 7f8fb57fa640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f9806a830 0x7f8f9806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.499+0000 7f8fb57fa640 1 -- 172.21.15.84:0/3103339756 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8fa8088410 con 0x7f8fb806c000 2023-10-11T06:11:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.499+0000 7f8faffff640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f9806a830 0x7f8f9806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.500+0000 7f8faffff640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f9806a830 0x7f8f9806cce0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0036000 tx=0x7f8fa003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.504+0000 7f8fb57fa640 1 -- 172.21.15.84:0/3103339756 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8fa8053650 con 0x7f8fb806c000 2023-10-11T06:11:55.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.687+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fb814f010 con 0x7f8fb806c000 2023-10-11T06:11:55.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.690+0000 7f8fb57fa640 1 -- 172.21.15.84:0/3103339756 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f8fa8056c70 con 0x7f8fb806c000 2023-10-11T06:11:55.691 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:11:55.692 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:11:55.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f9806a830 msgr2=0x7f8f9806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f9806a830 0x7f8f9806cce0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0036000 tx=0x7f8fa003b040 comp rx=0 tx=0).stop 2023-10-11T06:11:55.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb806c000 msgr2=0x7f8fb814cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb806c000 0x7f8fb814cea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c430 tx=0x7f8fa800c8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 shutdown_connections 2023-10-11T06:11:55.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f9806a830 0x7f8f9806cce0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fb80c9330 0x7f8fb80c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb806c3d0 0x7f8fb80c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 --2- 172.21.15.84:0/3103339756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb806c000 0x7f8fb814cea0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.692+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 >> 172.21.15.84:0/3103339756 conn(0x7f8fb8069c70 msgr2=0x7f8fb814fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.693+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 shutdown_connections 2023-10-11T06:11:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.693+0000 7f8fbd3bf640 1 -- 172.21.15.84:0/3103339756 wait complete. 2023-10-11T06:11:55.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:11:55.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:11:55.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.922+0000 7eff66f05640 1 Processor -- start 2023-10-11T06:11:55.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.922+0000 7eff66f05640 1 -- start start 2023-10-11T06:11:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff66f05640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1b10 0x7eff600cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff66f05640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7eff6006c530 0x7eff600cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff66f05640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6015cba0 con 0x7eff600b1ee0 2023-10-11T06:11:55.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff66f05640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6009c220 con 0x7eff6006c530 2023-10-11T06:11:55.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff66f05640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600d1360 con 0x7eff600b1b10 2023-10-11T06:11:55.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff65702640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7eff600b1ee0 0x7eff600ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39450/0 (socket says 172.21.15.84:39450) 2023-10-11T06:11:55.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff65702640 1 -- 172.21.15.84:0/409329773 learned_addr learned my addr 172.21.15.84:0/409329773 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:55.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff66704640 1 --2- 172.21.15.84:0/409329773 >> v2:172.21.15.84:3300/0 conn(0x7eff6006c530 0x7eff600cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.924+0000 7eff65f03640 1 --2- 172.21.15.84:0/409329773 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1b10 0x7eff600cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.925+0000 7eff66704640 1 -- 172.21.15.84:0/409329773 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1b10 msgr2=0x7eff600cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:55.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.925+0000 7eff66704640 1 --2- 172.21.15.84:0/409329773 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1b10 0x7eff600cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.925+0000 7eff66704640 1 -- 172.21.15.84:0/409329773 >> v1:172.21.15.84:6789/0 conn(0x7eff600b1ee0 legacy=0x7eff600ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.925+0000 7eff66704640 1 -- 172.21.15.84:0/409329773 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600d0ac0 con 0x7eff6006c530 2023-10-11T06:11:55.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.925+0000 7eff64f01640 1 -- 172.21.15.84:0/409329773 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23180271 0 0) 0x7eff6015cba0 con 0x7eff600b1ee0 2023-10-11T06:11:55.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.926+0000 7eff66704640 1 --2- 172.21.15.84:0/409329773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff600cf740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7eff58009d90 tx=0x7eff5802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3e08716843f7b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.926+0000 7eff64f01640 1 -- 172.21.15.84:0/409329773 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5802cce0 con 0x7eff6006c530 2023-10-11T06:11:55.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.926+0000 7eff64f01640 1 -- 172.21.15.84:0/409329773 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5802ce40 con 0x7eff6006c530 2023-10-11T06:11:55.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.926+0000 7eff64f01640 1 -- 172.21.15.84:0/409329773 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58034660 con 0x7eff6006c530 2023-10-11T06:11:55.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 -- 172.21.15.84:0/409329773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 msgr2=0x7eff600cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 --2- 172.21.15.84:0/409329773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff600cf740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7eff58009d90 tx=0x7eff5802bf30 comp rx=0 tx=0).stop 2023-10-11T06:11:55.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 -- 172.21.15.84:0/409329773 shutdown_connections 2023-10-11T06:11:55.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 --2- 172.21.15.84:0/409329773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff600cf740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 --2- 172.21.15.84:0/409329773 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1b10 0x7eff600cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 -- 172.21.15.84:0/409329773 >> 172.21.15.84:0/409329773 conn(0x7eff6009bc20 msgr2=0x7eff600d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:55.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 -- 172.21.15.84:0/409329773 shutdown_connections 2023-10-11T06:11:55.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.927+0000 7eff66f05640 1 -- 172.21.15.84:0/409329773 wait complete. 2023-10-11T06:11:55.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.928+0000 7eff66f05640 1 Processor -- start 2023-10-11T06:11:55.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.929+0000 7eff66f05640 1 -- start start 2023-10-11T06:11:55.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.929+0000 7eff66f05640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff6015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff66f05640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff600b1b10 0x7eff6015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff66f05640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1ee0 0x7eff600b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff66f05640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6015a930 con 0x7eff6006c530 2023-10-11T06:11:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff66f05640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600b6660 con 0x7eff600b1b10 2023-10-11T06:11:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff66f05640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600b6940 con 0x7eff600b1ee0 2023-10-11T06:11:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff6015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff6015aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56234/0 (socket says 172.21.15.84:56234) 2023-10-11T06:11:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 -- 172.21.15.84:0/3125945051 learned_addr learned my addr 172.21.15.84:0/3125945051 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 -- 172.21.15.84:0/3125945051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1ee0 msgr2=0x7eff600b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1ee0 0x7eff600b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 -- 172.21.15.84:0/3125945051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff600b1b10 msgr2=0x7eff6015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:55.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff600b1b10 0x7eff6015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:55.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 -- 172.21.15.84:0/3125945051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58009a70 con 0x7eff6006c530 2023-10-11T06:11:55.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.930+0000 7eff65f03640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff6015aaf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e900 tx=0x7eff5c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.931+0000 7eff567fc640 1 -- 172.21.15.84:0/3125945051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c009660 con 0x7eff6006c530 2023-10-11T06:11:55.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.931+0000 7eff567fc640 1 -- 172.21.15.84:0/3125945051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c00ca00 con 0x7eff6006c530 2023-10-11T06:11:55.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.931+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff600b6c20 con 0x7eff6006c530 2023-10-11T06:11:55.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.931+0000 7eff567fc640 1 -- 172.21.15.84:0/3125945051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00cd70 con 0x7eff6006c530 2023-10-11T06:11:55.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.931+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff600b71e0 con 0x7eff6006c530 2023-10-11T06:11:55.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.932+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff600b7370 con 0x7eff6006c530 2023-10-11T06:11:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.933+0000 7eff567fc640 1 -- 172.21.15.84:0/3125945051 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7eff5c013470 con 0x7eff6006c530 2023-10-11T06:11:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.934+0000 7eff567fc640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7eff4406a830 0x7eff4406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.934+0000 7eff567fc640 1 -- 172.21.15.84:0/3125945051 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7eff5c089690 con 0x7eff6006c530 2023-10-11T06:11:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.934+0000 7eff65702640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7eff4406a830 0x7eff4406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.935+0000 7eff65702640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7eff4406a830 0x7eff4406cce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff50007fd0 tx=0x7eff50007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:55.939+0000 7eff567fc640 1 -- 172.21.15.84:0/3125945051 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7eff5c054830 con 0x7eff6006c530 2023-10-11T06:11:56.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.110+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff6004e890 con 0x7eff6006c530 2023-10-11T06:11:56.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.111+0000 7eff567fc640 1 -- 172.21.15.84:0/3125945051 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff5c017020 con 0x7eff6006c530 2023-10-11T06:11:56.112 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:11:56.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7eff4406a830 msgr2=0x7eff4406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7eff4406a830 0x7eff4406cce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff50007fd0 tx=0x7eff50007960 comp rx=0 tx=0).stop 2023-10-11T06:11:56.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 msgr2=0x7eff6015aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff6015aaf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e900 tx=0x7eff5c00edc0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 shutdown_connections 2023-10-11T06:11:56.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7eff4406a830 0x7eff4406cce0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff600b1ee0 0x7eff600b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff600b1b10 0x7eff6015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 --2- 172.21.15.84:0/3125945051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff6006c530 0x7eff6015aaf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 >> 172.21.15.84:0/3125945051 conn(0x7eff6009bc20 msgr2=0x7eff6015bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:56.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 shutdown_connections 2023-10-11T06:11:56.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.113+0000 7eff66f05640 1 -- 172.21.15.84:0/3125945051 wait complete. 2023-10-11T06:11:56.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:11:56.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.348+0000 7fc106651640 1 Processor -- start 2023-10-11T06:11:56.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.348+0000 7fc106651640 1 -- start start 2023-10-11T06:11:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc106651640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1000cd2b0 0x7fc1000cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc106651640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1000cccd0 0x7fc1000cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc106651640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10006d6e0 con 0x7fc1001591b0 2023-10-11T06:11:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc106651640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc100159580 con 0x7fc1000cccd0 2023-10-11T06:11:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc106651640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10006b030 con 0x7fc1000cd2b0 2023-10-11T06:11:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc104bee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1000cccd0 0x7fc1000cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc104bee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1000cccd0 0x7fc1000cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56238/0 (socket says 172.21.15.84:56238) 2023-10-11T06:11:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc104bee640 1 -- 172.21.15.84:0/17635521 learned_addr learned my addr 172.21.15.84:0/17635521 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc0f77fe640 1 --2- 172.21.15.84:0/17635521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1000cd2b0 0x7fc1000cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc104bee640 1 -- 172.21.15.84:0/17635521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1000cd2b0 msgr2=0x7fc1000cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc104bee640 1 --2- 172.21.15.84:0/17635521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1000cd2b0 0x7fc1000cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc104bee640 1 -- 172.21.15.84:0/17635521 >> v1:172.21.15.99:6789/0 conn(0x7fc1001591b0 legacy=0x7fc10015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:56.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.350+0000 7fc104bee640 1 -- 172.21.15.84:0/17635521 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10015ccc0 con 0x7fc1000cccd0 2023-10-11T06:11:56.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.351+0000 7fc104bee640 1 --2- 172.21.15.84:0/17635521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cccd0 0x7fc1000cddc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e50 tx=0x7fc0f802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7159489d73f9528a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:56.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc0ff7fe640 1 -- 172.21.15.84:0/17635521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f802ca70 con 0x7fc1000cccd0 2023-10-11T06:11:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc0ff7fe640 1 -- 172.21.15.84:0/17635521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f802cbd0 con 0x7fc1000cccd0 2023-10-11T06:11:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc0ff7fe640 1 -- 172.21.15.84:0/17635521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80343f0 con 0x7fc1000cccd0 2023-10-11T06:11:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc106651640 1 -- 172.21.15.84:0/17635521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cccd0 msgr2=0x7fc1000cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc106651640 1 --2- 172.21.15.84:0/17635521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cccd0 0x7fc1000cddc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e50 tx=0x7fc0f802bed0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc106651640 1 -- 172.21.15.84:0/17635521 shutdown_connections 2023-10-11T06:11:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc106651640 1 --2- 172.21.15.84:0/17635521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cccd0 0x7fc1000cddc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc106651640 1 --2- 172.21.15.84:0/17635521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1000cd2b0 0x7fc1000cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.352+0000 7fc106651640 1 -- 172.21.15.84:0/17635521 >> 172.21.15.84:0/17635521 conn(0x7fc1000b16c0 msgr2=0x7fc1000ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.353+0000 7fc106651640 1 -- 172.21.15.84:0/17635521 shutdown_connections 2023-10-11T06:11:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.353+0000 7fc106651640 1 -- 172.21.15.84:0/17635521 wait complete. 2023-10-11T06:11:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.354+0000 7fc106651640 1 Processor -- start 2023-10-11T06:11:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.354+0000 7fc106651640 1 -- start start 2023-10-11T06:11:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.355+0000 7fc106651640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1000cccd0 0x7fc1001596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.355+0000 7fc106651640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cd2b0 0x7fc1001584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.355+0000 7fc106651640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1001591b0 0x7fc1000af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.355+0000 7fc106651640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1001521a0 con 0x7fc1000cd2b0 2023-10-11T06:11:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.355+0000 7fc106651640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc100153610 con 0x7fc1000cccd0 2023-10-11T06:11:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.355+0000 7fc106651640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10014f740 con 0x7fc1001591b0 2023-10-11T06:11:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cd2b0 0x7fc1001584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cd2b0 0x7fc1001584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56246/0 (socket says 172.21.15.84:56246) 2023-10-11T06:11:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 -- 172.21.15.84:0/3708348589 learned_addr learned my addr 172.21.15.84:0/3708348589 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0fffff640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1000cccd0 0x7fc1001596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc104bee640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1001591b0 0x7fc1000af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 -- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1001591b0 msgr2=0x7fc1000af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1001591b0 0x7fc1000af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 -- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1000cccd0 msgr2=0x7fc1001596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1000cccd0 0x7fc1001596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 -- 172.21.15.84:0/3708348589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009b30 con 0x7fc1000cd2b0 2023-10-11T06:11:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0f77fe640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cd2b0 0x7fc1001584b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c7f0 tx=0x7fc0e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.356+0000 7fc0fd7fa640 1 -- 172.21.15.84:0/3708348589 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8004220 con 0x7fc1000cd2b0 2023-10-11T06:11:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.357+0000 7fc0fd7fa640 1 -- 172.21.15.84:0/3708348589 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e80043a0 con 0x7fc1000cd2b0 2023-10-11T06:11:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.357+0000 7fc0fd7fa640 1 -- 172.21.15.84:0/3708348589 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8007860 con 0x7fc1000cd2b0 2023-10-11T06:11:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.357+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc100153880 con 0x7fc1000cd2b0 2023-10-11T06:11:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.357+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc100159c00 con 0x7fc1000cd2b0 2023-10-11T06:11:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.358+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc100153da0 con 0x7fc1000cd2b0 2023-10-11T06:11:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.359+0000 7fc0fd7fa640 1 -- 172.21.15.84:0/3708348589 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc0e8007bd0 con 0x7fc1000cd2b0 2023-10-11T06:11:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.360+0000 7fc0fd7fa640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0e006a830 0x7fc0e006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.360+0000 7fc0fd7fa640 1 -- 172.21.15.84:0/3708348589 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc0e8089070 con 0x7fc1000cd2b0 2023-10-11T06:11:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.360+0000 7fc0fffff640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0e006a830 0x7fc0e006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.361+0000 7fc0fffff640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0e006a830 0x7fc0e006cce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000a750 tx=0x7fc0f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:56.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.365+0000 7fc0fd7fa640 1 -- 172.21.15.84:0/3708348589 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc100153da0 con 0x7fc1000cd2b0 2023-10-11T06:11:56.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:11:56.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:11:56.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.555+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc100153da0 con 0x7fc1000cd2b0 2023-10-11T06:11:56.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.556+0000 7fc0fd7fa640 1 -- 172.21.15.84:0/3708348589 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc100153da0 con 0x7fc1000cd2b0 2023-10-11T06:11:56.557 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:11:56.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0e006a830 msgr2=0x7fc0e006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0e006a830 0x7fc0e006cce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000a750 tx=0x7fc0f0008040 comp rx=0 tx=0).stop 2023-10-11T06:11:56.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cd2b0 msgr2=0x7fc1001584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cd2b0 0x7fc1001584b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c7f0 tx=0x7fc0e800ccb0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 shutdown_connections 2023-10-11T06:11:56.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0e006a830 0x7fc0e006cce0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1001591b0 0x7fc1000af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1000cd2b0 0x7fc1001584b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 --2- 172.21.15.84:0/3708348589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1000cccd0 0x7fc1001596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 >> 172.21.15.84:0/3708348589 conn(0x7fc1000b16c0 msgr2=0x7fc100069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:56.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 shutdown_connections 2023-10-11T06:11:56.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.558+0000 7fc106651640 1 -- 172.21.15.84:0/3708348589 wait complete. 2023-10-11T06:11:56.576 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:11:56.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.840+0000 7ff494e84640 1 Processor -- start 2023-10-11T06:11:56.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.840+0000 7ff494e84640 1 -- start start 2023-10-11T06:11:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff494e84640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 0x7ff4900c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff494e84640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff4900ce510 0x7ff4900ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff494e84640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900c9b90 con 0x7ff49014ff30 2023-10-11T06:11:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff494e84640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49015da10 con 0x7ff4900ce510 2023-10-11T06:11:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff494e84640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900ceef0 con 0x7ff490150300 2023-10-11T06:11:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff48ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff4900ce510 0x7ff4900ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff48ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff4900ce510 0x7ff4900ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56254/0 (socket says 172.21.15.84:56254) 2023-10-11T06:11:56.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff48ed9d640 1 -- 172.21.15.84:0/58327536 learned_addr learned my addr 172.21.15.84:0/58327536 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:56.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.842+0000 7ff48dd9b640 1 --2- 172.21.15.84:0/58327536 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 0x7ff4900c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.843+0000 7ff48ed9d640 1 -- 172.21.15.84:0/58327536 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 msgr2=0x7ff4900c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:56.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.843+0000 7ff48ed9d640 1 --2- 172.21.15.84:0/58327536 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 0x7ff4900c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.843+0000 7ff48ed9d640 1 -- 172.21.15.84:0/58327536 >> v1:172.21.15.99:6789/0 conn(0x7ff49014ff30 legacy=0x7ff4900cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:56.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.843+0000 7ff48ed9d640 1 -- 172.21.15.84:0/58327536 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900cc990 con 0x7ff4900ce510 2023-10-11T06:11:56.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.844+0000 7ff48ed9d640 1 --2- 172.21.15.84:0/58327536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff4900ce9b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff488009870 tx=0x7ff48802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd2cb90891b801d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:56.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.844+0000 7ff48d59a640 1 -- 172.21.15.84:0/58327536 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880317c0 con 0x7ff4900ce510 2023-10-11T06:11:56.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.844+0000 7ff48d59a640 1 -- 172.21.15.84:0/58327536 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff488031920 con 0x7ff4900ce510 2023-10-11T06:11:56.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.844+0000 7ff48d59a640 1 -- 172.21.15.84:0/58327536 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488031cb0 con 0x7ff4900ce510 2023-10-11T06:11:56.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 -- 172.21.15.84:0/58327536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 msgr2=0x7ff4900ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 --2- 172.21.15.84:0/58327536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff4900ce9b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff488009870 tx=0x7ff48802b860 comp rx=0 tx=0).stop 2023-10-11T06:11:56.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 -- 172.21.15.84:0/58327536 shutdown_connections 2023-10-11T06:11:56.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 --2- 172.21.15.84:0/58327536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff4900ce9b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 --2- 172.21.15.84:0/58327536 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 0x7ff4900c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 -- 172.21.15.84:0/58327536 >> 172.21.15.84:0/58327536 conn(0x7ff490069c70 msgr2=0x7ff49006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:56.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 -- 172.21.15.84:0/58327536 shutdown_connections 2023-10-11T06:11:56.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.845+0000 7ff494e84640 1 -- 172.21.15.84:0/58327536 wait complete. 2023-10-11T06:11:56.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.847+0000 7ff494e84640 1 Processor -- start 2023-10-11T06:11:56.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.847+0000 7ff494e84640 1 -- start start 2023-10-11T06:11:56.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff494e84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff490142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff494e84640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49014ff30 0x7ff490158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff494e84640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 0x7ff490138790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff494e84640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490157330 con 0x7ff4900ce510 2023-10-11T06:11:56.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff494e84640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901575d0 con 0x7ff49014ff30 2023-10-11T06:11:56.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff494e84640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901578b0 con 0x7ff490150300 2023-10-11T06:11:56.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff490142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff490142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56262/0 (socket says 172.21.15.84:56262) 2023-10-11T06:11:56.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 -- 172.21.15.84:0/4029789752 learned_addr learned my addr 172.21.15.84:0/4029789752 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:56.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 -- 172.21.15.84:0/4029789752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 msgr2=0x7ff490138790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 0x7ff490138790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48dd9b640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49014ff30 0x7ff490158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 -- 172.21.15.84:0/4029789752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49014ff30 msgr2=0x7ff490158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:56.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49014ff30 0x7ff490158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:56.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.849+0000 7ff48e59c640 1 -- 172.21.15.84:0/4029789752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880094d0 con 0x7ff4900ce510 2023-10-11T06:11:56.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.850+0000 7ff48e59c640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff490142650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c7c0 tx=0x7ff48000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:56.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.850+0000 7ff486ffd640 1 -- 172.21.15.84:0/4029789752 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000aa50 con 0x7ff4900ce510 2023-10-11T06:11:56.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.850+0000 7ff486ffd640 1 -- 172.21.15.84:0/4029789752 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48000abb0 con 0x7ff4900ce510 2023-10-11T06:11:56.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.850+0000 7ff486ffd640 1 -- 172.21.15.84:0/4029789752 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480012420 con 0x7ff4900ce510 2023-10-11T06:11:56.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.850+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff490157b90 con 0x7ff4900ce510 2023-10-11T06:11:56.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.850+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff490148420 con 0x7ff4900ce510 2023-10-11T06:11:56.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.851+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4901580b0 con 0x7ff4900ce510 2023-10-11T06:11:56.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.852+0000 7ff486ffd640 1 -- 172.21.15.84:0/4029789752 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff480012580 con 0x7ff4900ce510 2023-10-11T06:11:56.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.853+0000 7ff486ffd640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff47006a830 0x7ff47006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:56.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.853+0000 7ff486ffd640 1 -- 172.21.15.84:0/4029789752 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff480088f50 con 0x7ff4900ce510 2023-10-11T06:11:56.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.853+0000 7ff48dd9b640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff47006a830 0x7ff47006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:56.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.854+0000 7ff48dd9b640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff47006a830 0x7ff47006cce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff478006fd0 tx=0x7ff478008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:56.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:56.857+0000 7ff486ffd640 1 -- 172.21.15.84:0/4029789752 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff4901580b0 con 0x7ff4900ce510 2023-10-11T06:11:57.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.051+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4901580b0 con 0x7ff4900ce510 2023-10-11T06:11:57.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.051+0000 7ff486ffd640 1 -- 172.21.15.84:0/4029789752 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff480016020 con 0x7ff4900ce510 2023-10-11T06:11:57.053 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:11:57.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff47006a830 msgr2=0x7ff47006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff47006a830 0x7ff47006cce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff478006fd0 tx=0x7ff478008040 comp rx=0 tx=0).stop 2023-10-11T06:11:57.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 msgr2=0x7ff490142650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff490142650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c7c0 tx=0x7ff48000cc80 comp rx=0 tx=0).stop 2023-10-11T06:11:57.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 shutdown_connections 2023-10-11T06:11:57.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff47006a830 0x7ff47006cce0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff490150300 0x7ff490138790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff49014ff30 0x7ff490158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 --2- 172.21.15.84:0/4029789752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4900ce510 0x7ff490142650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.054+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 >> 172.21.15.84:0/4029789752 conn(0x7ff490069c70 msgr2=0x7ff49014f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:57.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.055+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 shutdown_connections 2023-10-11T06:11:57.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.055+0000 7ff494e84640 1 -- 172.21.15.84:0/4029789752 wait complete. 2023-10-11T06:11:57.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:11:57.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.290+0000 7fc548d73640 1 Processor -- start 2023-10-11T06:11:57.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.290+0000 7fc548d73640 1 -- start start 2023-10-11T06:11:57.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc548d73640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc544160830 0x7fc5440ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc548d73640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5440c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc548d73640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5440cc9e0 con 0x7fc54406dd00 2023-10-11T06:11:57.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc548d73640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54406d720 con 0x7fc544160830 2023-10-11T06:11:57.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc548d73640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54406d180 con 0x7fc5440cf270 2023-10-11T06:11:57.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc54259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc544160830 0x7fc5440ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc542d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc54406dd00 0x7fc544164400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39458/0 (socket says 172.21.15.84:39458) 2023-10-11T06:11:57.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.292+0000 7fc542d9d640 1 -- 172.21.15.84:0/3415923890 learned_addr learned my addr 172.21.15.84:0/3415923890 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:57.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.293+0000 7fc53bfff640 1 --2- 172.21.15.84:0/3415923890 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5440c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.293+0000 7fc54259c640 1 -- 172.21.15.84:0/3415923890 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 msgr2=0x7fc5440c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:57.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.293+0000 7fc54259c640 1 --2- 172.21.15.84:0/3415923890 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5440c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.293+0000 7fc54259c640 1 -- 172.21.15.84:0/3415923890 >> v1:172.21.15.84:6789/0 conn(0x7fc54406dd00 legacy=0x7fc544164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.293+0000 7fc54259c640 1 -- 172.21.15.84:0/3415923890 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54406d440 con 0x7fc544160830 2023-10-11T06:11:57.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.293+0000 7fc53bfff640 1 --2- 172.21.15.84:0/3415923890 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5440c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:57.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.293+0000 7fc541d9b640 1 -- 172.21.15.84:0/3415923890 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793152358 0 0) 0x7fc5440cc9e0 con 0x7fc54406dd00 2023-10-11T06:11:57.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.294+0000 7fc54259c640 1 --2- 172.21.15.84:0/3415923890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc544160830 0x7fc5440ced30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097a0 tx=0x7fc53402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ab5415b19002c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:57.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.294+0000 7fc541d9b640 1 -- 172.21.15.84:0/3415923890 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53402d040 con 0x7fc544160830 2023-10-11T06:11:57.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.294+0000 7fc541d9b640 1 -- 172.21.15.84:0/3415923890 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc534031810 con 0x7fc544160830 2023-10-11T06:11:57.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.294+0000 7fc541d9b640 1 -- 172.21.15.84:0/3415923890 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534031ba0 con 0x7fc544160830 2023-10-11T06:11:57.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 -- 172.21.15.84:0/3415923890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc544160830 msgr2=0x7fc5440ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 --2- 172.21.15.84:0/3415923890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc544160830 0x7fc5440ced30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097a0 tx=0x7fc53402b8b0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 -- 172.21.15.84:0/3415923890 shutdown_connections 2023-10-11T06:11:57.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 --2- 172.21.15.84:0/3415923890 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5440c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 --2- 172.21.15.84:0/3415923890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc544160830 0x7fc5440ced30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 -- 172.21.15.84:0/3415923890 >> 172.21.15.84:0/3415923890 conn(0x7fc544069c70 msgr2=0x7fc54406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:57.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 -- 172.21.15.84:0/3415923890 shutdown_connections 2023-10-11T06:11:57.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.295+0000 7fc548d73640 1 -- 172.21.15.84:0/3415923890 wait complete. 2023-10-11T06:11:57.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.297+0000 7fc548d73640 1 Processor -- start 2023-10-11T06:11:57.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.297+0000 7fc548d73640 1 -- start start 2023-10-11T06:11:57.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc548d73640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc54406dd00 0x7fc5440c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc548d73640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5441f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc548d73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc544160830 0x7fc544161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc548d73640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440b1c60 con 0x7fc54406dd00 2023-10-11T06:11:57.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc548d73640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5441f9b30 con 0x7fc544160830 2023-10-11T06:11:57.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc548d73640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5441f9e10 con 0x7fc5440cf270 2023-10-11T06:11:57.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc54406dd00 0x7fc5440c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc54406dd00 0x7fc5440c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56294/0 (socket says 172.21.15.84:56294) 2023-10-11T06:11:57.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 -- 172.21.15.84:0/494662718 learned_addr learned my addr 172.21.15.84:0/494662718 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:57.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 -- 172.21.15.84:0/494662718 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 msgr2=0x7fc5441f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5441f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 -- 172.21.15.84:0/494662718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc544160830 msgr2=0x7fc544161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc544160830 0x7fc544161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.298+0000 7fc54259c640 1 -- 172.21.15.84:0/494662718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc534009400 con 0x7fc54406dd00 2023-10-11T06:11:57.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.299+0000 7fc54259c640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc54406dd00 0x7fc5440c80c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc5340098c0 tx=0x7fc5340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:57.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.299+0000 7fc53affd640 1 -- 172.21.15.84:0/494662718 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53403ad60 con 0x7fc54406dd00 2023-10-11T06:11:57.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.299+0000 7fc53affd640 1 -- 172.21.15.84:0/494662718 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc534044400 con 0x7fc54406dd00 2023-10-11T06:11:57.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.299+0000 7fc53affd640 1 -- 172.21.15.84:0/494662718 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340447b0 con 0x7fc54406dd00 2023-10-11T06:11:57.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.299+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5441fa0f0 con 0x7fc54406dd00 2023-10-11T06:11:57.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.299+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5441619d0 con 0x7fc54406dd00 2023-10-11T06:11:57.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.300+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc544073ed0 con 0x7fc54406dd00 2023-10-11T06:11:57.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.301+0000 7fc53affd640 1 -- 172.21.15.84:0/494662718 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc53404d440 con 0x7fc54406dd00 2023-10-11T06:11:57.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.302+0000 7fc53affd640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc52406a830 0x7fc52406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.302+0000 7fc53affd640 1 -- 172.21.15.84:0/494662718 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc5340ab5d0 con 0x7fc54406dd00 2023-10-11T06:11:57.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.302+0000 7fc53bfff640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc52406a830 0x7fc52406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.303+0000 7fc53bfff640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc52406a830 0x7fc52406cce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc52c006fd0 tx=0x7fc52c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:57.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.306+0000 7fc53affd640 1 -- 172.21.15.84:0/494662718 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc534076770 con 0x7fc54406dd00 2023-10-11T06:11:57.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.494+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5440708f0 con 0x7fc54406dd00 2023-10-11T06:11:57.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.494+0000 7fc53affd640 1 -- 172.21.15.84:0/494662718 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc534079d90 con 0x7fc54406dd00 2023-10-11T06:11:57.496 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:11:57.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.496+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc52406a830 msgr2=0x7fc52406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.496+0000 7fc548d73640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc52406a830 0x7fc52406cce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc52c006fd0 tx=0x7fc52c008040 comp rx=0 tx=0).stop 2023-10-11T06:11:57.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.496+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc54406dd00 msgr2=0x7fc5440c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.496+0000 7fc548d73640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc54406dd00 0x7fc5440c80c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc5340098c0 tx=0x7fc5340046c0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 shutdown_connections 2023-10-11T06:11:57.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc52406a830 0x7fc52406cce0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc544160830 0x7fc544161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5440cf270 0x7fc5441f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 --2- 172.21.15.84:0/494662718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc54406dd00 0x7fc5440c80c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 >> 172.21.15.84:0/494662718 conn(0x7fc544069c70 msgr2=0x7fc54415cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:57.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 shutdown_connections 2023-10-11T06:11:57.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.497+0000 7fc548d73640 1 -- 172.21.15.84:0/494662718 wait complete. 2023-10-11T06:11:57.523 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2023-10-11T06:11:57.524 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-11T06:11:57.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-10-11T06:11:57.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.744+0000 7f19fceda640 1 Processor -- start 2023-10-11T06:11:57.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.744+0000 7f19fceda640 1 -- start start 2023-10-11T06:11:57.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19fceda640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19f80afe10 0x7f19f80b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19fceda640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0b00 0x7f19f80cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19fceda640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f815c700 con 0x7f19f815c330 2023-10-11T06:11:57.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19fceda640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f80b0f20 con 0x7f19f80afe10 2023-10-11T06:11:57.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19fceda640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d1100 con 0x7f19f80d0b00 2023-10-11T06:11:57.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19f6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19f80afe10 0x7f19f80b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19f77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f19f815c330 0x7f19f80d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:39468/0 (socket says 172.21.15.84:39468) 2023-10-11T06:11:57.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.745+0000 7f19f77fe640 1 -- 172.21.15.84:0/206154254 learned_addr learned my addr 172.21.15.84:0/206154254 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:11:57.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.746+0000 7f19f7fff640 1 --2- 172.21.15.84:0/206154254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0b00 0x7f19f80cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.746+0000 7f19f6ffd640 1 -- 172.21.15.84:0/206154254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0b00 msgr2=0x7f19f80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:57.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.746+0000 7f19f6ffd640 1 --2- 172.21.15.84:0/206154254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0b00 0x7f19f80cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.746+0000 7f19f6ffd640 1 -- 172.21.15.84:0/206154254 >> v1:172.21.15.84:6789/0 conn(0x7f19f815c330 legacy=0x7f19f80d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.746+0000 7f19f67fc640 1 -- 172.21.15.84:0/206154254 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764951685 0 0) 0x7f19e8001610 con 0x7f19f815c330 2023-10-11T06:11:57.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.746+0000 7f19f6ffd640 1 -- 172.21.15.84:0/206154254 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80b02a0 con 0x7f19f80afe10 2023-10-11T06:11:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.746+0000 7f19f7fff640 1 --2- 172.21.15.84:0/206154254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0b00 0x7f19f80cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.747+0000 7f19f6ffd640 1 --2- 172.21.15.84:0/206154254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f80afe10 0x7f19f80b1340 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7279c0836824bb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:57.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.747+0000 7f19f67fc640 1 -- 172.21.15.84:0/206154254 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00316f0 con 0x7f19f80afe10 2023-10-11T06:11:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.747+0000 7f19f67fc640 1 -- 172.21.15.84:0/206154254 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e0031850 con 0x7f19f80afe10 2023-10-11T06:11:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.747+0000 7f19f67fc640 1 -- 172.21.15.84:0/206154254 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0031be0 con 0x7f19f80afe10 2023-10-11T06:11:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 -- 172.21.15.84:0/206154254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f80afe10 msgr2=0x7f19f80b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 --2- 172.21.15.84:0/206154254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f80afe10 0x7f19f80b1340 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002b790 comp rx=0 tx=0).stop 2023-10-11T06:11:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 -- 172.21.15.84:0/206154254 shutdown_connections 2023-10-11T06:11:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 --2- 172.21.15.84:0/206154254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0b00 0x7f19f80cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 --2- 172.21.15.84:0/206154254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f80afe10 0x7f19f80b1340 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 -- 172.21.15.84:0/206154254 >> 172.21.15.84:0/206154254 conn(0x7f19f809bb10 msgr2=0x7f19f80d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:57.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 -- 172.21.15.84:0/206154254 shutdown_connections 2023-10-11T06:11:57.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.748+0000 7f19fceda640 1 -- 172.21.15.84:0/206154254 wait complete. 2023-10-11T06:11:57.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.750+0000 7f19fceda640 1 Processor -- start 2023-10-11T06:11:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.750+0000 7f19fceda640 1 -- start start 2023-10-11T06:11:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19fceda640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80afe10 0x7f19f81e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19fceda640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f80d0b00 0x7f19f80ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19fceda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f815c330 0x7f19f8153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19fceda640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f81e64d0 con 0x7f19f815c330 2023-10-11T06:11:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19fceda640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80c9ba0 con 0x7f19f80d0b00 2023-10-11T06:11:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19fceda640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f81e5430 con 0x7f19f80afe10 2023-10-11T06:11:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f815c330 0x7f19f8153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f815c330 0x7f19f8153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56316/0 (socket says 172.21.15.84:56316) 2023-10-11T06:11:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19f7fff640 1 -- 172.21.15.84:0/1987722282 learned_addr learned my addr 172.21.15.84:0/1987722282 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.751+0000 7f19f77fe640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80afe10 0x7f19f81e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f6ffd640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f80d0b00 0x7f19f80ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f7fff640 1 -- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80afe10 msgr2=0x7f19f81e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f7fff640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80afe10 0x7f19f81e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f7fff640 1 -- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f80d0b00 msgr2=0x7f19f80ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:57.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f7fff640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f80d0b00 0x7f19f80ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:57.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f7fff640 1 -- 172.21.15.84:0/1987722282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009400 con 0x7f19f815c330 2023-10-11T06:11:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f77fe640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80afe10 0x7f19f81e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f6ffd640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f80d0b00 0x7f19f80ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:57.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19f7fff640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f815c330 0x7f19f8153b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f19f000c7f0 tx=0x7f19f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:57.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19effff640 1 -- 172.21.15.84:0/1987722282 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0004220 con 0x7f19f815c330 2023-10-11T06:11:57.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19effff640 1 -- 172.21.15.84:0/1987722282 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f00043a0 con 0x7f19f815c330 2023-10-11T06:11:57.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f81e5710 con 0x7f19f815c330 2023-10-11T06:11:57.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19effff640 1 -- 172.21.15.84:0/1987722282 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0007860 con 0x7f19f815c330 2023-10-11T06:11:57.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.752+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f81e5cd0 con 0x7f19f815c330 2023-10-11T06:11:57.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.754+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f804e890 con 0x7f19f815c330 2023-10-11T06:11:57.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.754+0000 7f19effff640 1 -- 172.21.15.84:0/1987722282 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f19f0007bd0 con 0x7f19f815c330 2023-10-11T06:11:57.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.755+0000 7f19effff640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19d806a830 0x7f19d806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:57.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.755+0000 7f19effff640 1 -- 172.21.15.84:0/1987722282 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f19f0088f40 con 0x7f19f815c330 2023-10-11T06:11:57.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.756+0000 7f19f77fe640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19d806a830 0x7f19d806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:57.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.756+0000 7f19f77fe640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19d806a830 0x7f19d806cce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f80d3570 tx=0x7f19e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:57.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:57.760+0000 7f19effff640 1 -- 172.21.15.84:0/1987722282 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f19f00542d0 con 0x7f19f815c330 2023-10-11T06:11:58.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.017+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f19f80ce640 con 0x7f19f815c330 2023-10-11T06:11:58.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.932+0000 7f19effff640 1 -- 172.21.15.84:0/1987722282 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v64) v1 ==== 150+0+0 (secure 0 0 0) 0x7f19f001e0d0 con 0x7f19f815c330 2023-10-11T06:11:58.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19d806a830 msgr2=0x7f19d806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:58.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19d806a830 0x7f19d806cce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f80d3570 tx=0x7f19e8009040 comp rx=0 tx=0).stop 2023-10-11T06:11:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f815c330 msgr2=0x7f19f8153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f815c330 0x7f19f8153b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f19f000c7f0 tx=0x7f19f000ccb0 comp rx=0 tx=0).stop 2023-10-11T06:11:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 shutdown_connections 2023-10-11T06:11:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19d806a830 0x7f19d806cce0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f815c330 0x7f19f8153b30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f80d0b00 0x7f19f80ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 --2- 172.21.15.84:0/1987722282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80afe10 0x7f19f81e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 >> 172.21.15.84:0/1987722282 conn(0x7f19f809bb10 msgr2=0x7f19f80ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 shutdown_connections 2023-10-11T06:11:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:58.935+0000 7f19fceda640 1 -- 172.21.15.84:0/1987722282 wait complete. 2023-10-11T06:11:58.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-11T06:11:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.173+0000 7ff973b18640 1 Processor -- start 2023-10-11T06:11:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.174+0000 7ff973b18640 1 -- start start 2023-10-11T06:11:59.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.175+0000 7ff973b18640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff96c15f7e0 0x7ff96c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.175+0000 7ff973b18640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c0cd6c0 0x7ff96c0d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.175+0000 7ff973b18640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0d3460 con 0x7ff96c0d3000 2023-10-11T06:11:59.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.175+0000 7ff973b18640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0cf220 con 0x7ff96c15f7e0 2023-10-11T06:11:59.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.175+0000 7ff973b18640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0cf500 con 0x7ff96c0cd6c0 2023-10-11T06:11:59.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.175+0000 7ff9718b4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff96c15f7e0 0x7ff96c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9718b4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff96c15f7e0 0x7ff96c0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59188/0 (socket says 172.21.15.84:59188) 2023-10-11T06:11:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9718b4640 1 -- 172.21.15.84:0/2419437948 learned_addr learned my addr 172.21.15.84:0/2419437948 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9710b3640 1 --2- 172.21.15.84:0/2419437948 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c0cd6c0 0x7ff96c0d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9718b4640 1 -- 172.21.15.84:0/2419437948 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c0cd6c0 msgr2=0x7ff96c0d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9718b4640 1 --2- 172.21.15.84:0/2419437948 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c0cd6c0 0x7ff96c0d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9718b4640 1 -- 172.21.15.84:0/2419437948 >> v1:172.21.15.99:6789/0 conn(0x7ff96c0d3000 legacy=0x7ff96c15fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9718b4640 1 -- 172.21.15.84:0/2419437948 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0cf7e0 con 0x7ff96c15f7e0 2023-10-11T06:11:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.176+0000 7ff9710b3640 1 --2- 172.21.15.84:0/2419437948 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c0cd6c0 0x7ff96c0d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.177+0000 7ff9718b4640 1 --2- 172.21.15.84:0/2419437948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c15f7e0 0x7ff96c0cd180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29f50ac88584b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.177+0000 7ff9708b2640 1 -- 172.21.15.84:0/2419437948 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c031640 con 0x7ff96c15f7e0 2023-10-11T06:11:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff9708b2640 1 -- 172.21.15.84:0/2419437948 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c0317a0 con 0x7ff96c15f7e0 2023-10-11T06:11:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff9708b2640 1 -- 172.21.15.84:0/2419437948 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c031b30 con 0x7ff96c15f7e0 2023-10-11T06:11:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 -- 172.21.15.84:0/2419437948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c15f7e0 msgr2=0x7ff96c0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 --2- 172.21.15.84:0/2419437948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c15f7e0 0x7ff96c0cd180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c02b8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 -- 172.21.15.84:0/2419437948 shutdown_connections 2023-10-11T06:11:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 --2- 172.21.15.84:0/2419437948 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c0cd6c0 0x7ff96c0d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 --2- 172.21.15.84:0/2419437948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c15f7e0 0x7ff96c0cd180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 -- 172.21.15.84:0/2419437948 >> 172.21.15.84:0/2419437948 conn(0x7ff96c0713a0 msgr2=0x7ff96c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 -- 172.21.15.84:0/2419437948 shutdown_connections 2023-10-11T06:11:59.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.178+0000 7ff973b18640 1 -- 172.21.15.84:0/2419437948 wait complete. 2023-10-11T06:11:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.180+0000 7ff973b18640 1 Processor -- start 2023-10-11T06:11:59.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.180+0000 7ff973b18640 1 -- start start 2023-10-11T06:11:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.181+0000 7ff973b18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c0cd6c0 0x7ff96c0c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.181+0000 7ff973b18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c0d3000 0x7ff96c0c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.181+0000 7ff973b18640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c15f7e0 0x7ff96c0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.181+0000 7ff973b18640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0ccf40 con 0x7ff96c0d3000 2023-10-11T06:11:59.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.181+0000 7ff973b18640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c14afc0 con 0x7ff96c0cd6c0 2023-10-11T06:11:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.181+0000 7ff973b18640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0ccc60 con 0x7ff96c15f7e0 2023-10-11T06:11:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c0d3000 0x7ff96c0c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c0d3000 0x7ff96c0c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59200/0 (socket says 172.21.15.84:59200) 2023-10-11T06:11:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 -- 172.21.15.84:0/4070233606 learned_addr learned my addr 172.21.15.84:0/4070233606 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:59.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 -- 172.21.15.84:0/4070233606 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c15f7e0 msgr2=0x7ff96c0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c15f7e0 0x7ff96c0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 -- 172.21.15.84:0/4070233606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c0cd6c0 msgr2=0x7ff96c0c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c0cd6c0 0x7ff96c0c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 -- 172.21.15.84:0/4070233606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0094d0 con 0x7ff96c0d3000 2023-10-11T06:11:59.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.182+0000 7ff9710b3640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c0d3000 0x7ff96c0c6660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c7f0 tx=0x7ff96000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:59.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.183+0000 7ff96a7fc640 1 -- 172.21.15.84:0/4070233606 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960007a30 con 0x7ff96c0d3000 2023-10-11T06:11:59.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.183+0000 7ff96a7fc640 1 -- 172.21.15.84:0/4070233606 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff960007b90 con 0x7ff96c0d3000 2023-10-11T06:11:59.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.183+0000 7ff96a7fc640 1 -- 172.21.15.84:0/4070233606 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ce40 con 0x7ff96c0d3000 2023-10-11T06:11:59.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.183+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c14a900 con 0x7ff96c0d3000 2023-10-11T06:11:59.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.183+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c0cc8e0 con 0x7ff96c0d3000 2023-10-11T06:11:59.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.184+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c073ed0 con 0x7ff96c0d3000 2023-10-11T06:11:59.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.185+0000 7ff96a7fc640 1 -- 172.21.15.84:0/4070233606 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff9600113f0 con 0x7ff96c0d3000 2023-10-11T06:11:59.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.186+0000 7ff96a7fc640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff95806a830 0x7ff95806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.186+0000 7ff96a7fc640 1 -- 172.21.15.84:0/4070233606 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff9600891c0 con 0x7ff96c0d3000 2023-10-11T06:11:59.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.186+0000 7ff9718b4640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff95806a830 0x7ff95806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.187+0000 7ff9718b4640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff95806a830 0x7ff95806cce0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff95c036000 tx=0x7ff95c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:59.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.191+0000 7ff96a7fc640 1 -- 172.21.15.84:0/4070233606 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff9600543f0 con 0x7ff96c0d3000 2023-10-11T06:11:59.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.382+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff96c0b5030 con 0x7ff96c0d3000 2023-10-11T06:11:59.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.391+0000 7ff96a7fc640 1 -- 172.21.15.84:0/4070233606 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff960057a10 con 0x7ff96c0d3000 2023-10-11T06:11:59.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff95806a830 msgr2=0x7ff95806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff95806a830 0x7ff95806cce0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff95c036000 tx=0x7ff95c030000 comp rx=0 tx=0).stop 2023-10-11T06:11:59.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c0d3000 msgr2=0x7ff96c0c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c0d3000 0x7ff96c0c6660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c7f0 tx=0x7ff96000cbf0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 shutdown_connections 2023-10-11T06:11:59.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff95806a830 0x7ff95806cce0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff96c15f7e0 0x7ff96c0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff96c0d3000 0x7ff96c0c6660 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 --2- 172.21.15.84:0/4070233606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c0cd6c0 0x7ff96c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 >> 172.21.15.84:0/4070233606 conn(0x7ff96c0713a0 msgr2=0x7ff96c071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:59.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 shutdown_connections 2023-10-11T06:11:59.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.394+0000 7ff973b18640 1 -- 172.21.15.84:0/4070233606 wait complete. 2023-10-11T06:11:59.415 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T06:11:59.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:11:59.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:11:59.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T06:11:59.657 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:11:59.658 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:11:59.719 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:11:59.719 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:11:59.774 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:11:59.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.981+0000 7f79b6e1a640 1 Processor -- start 2023-10-11T06:11:59.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.981+0000 7f79b6e1a640 1 -- start start 2023-10-11T06:11:59.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.983+0000 7f79b6e1a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b00af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.983+0000 7f79b6e1a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79b00d25e0 0x7f79b00d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.983+0000 7f79b6e1a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00d0eb0 con 0x7f79b00d0a50 2023-10-11T06:11:59.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.983+0000 7f79b6e1a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00d2a30 con 0x7f79b00d25e0 2023-10-11T06:11:59.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.983+0000 7f79b6e1a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b006cd10 con 0x7f79b015c290 2023-10-11T06:11:59.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.984+0000 7f79affff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79b00d25e0 0x7f79b00d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.984+0000 7f79b4bb6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b00af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.984+0000 7f79affff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79b00d25e0 0x7f79b00d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:46216/0 (socket says 172.21.15.84:46216) 2023-10-11T06:11:59.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.984+0000 7f79affff640 1 -- 172.21.15.84:0/1689468012 learned_addr learned my addr 172.21.15.84:0/1689468012 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:59.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.985+0000 7f79affff640 1 -- 172.21.15.84:0/1689468012 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 msgr2=0x7f79b00af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:11:59.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.985+0000 7f79affff640 1 --2- 172.21.15.84:0/1689468012 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b00af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.985+0000 7f79affff640 1 -- 172.21.15.84:0/1689468012 >> v1:172.21.15.99:6789/0 conn(0x7f79b00d0a50 legacy=0x7f79b006b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.985+0000 7f79affff640 1 -- 172.21.15.84:0/1689468012 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00afb60 con 0x7f79b00d25e0 2023-10-11T06:11:59.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.986+0000 7f79affff640 1 --2- 172.21.15.84:0/1689468012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b00d0510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f799c009870 tx=0x7f799c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ed5b41464ee455c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:59.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.986+0000 7f79af7fe640 1 -- 172.21.15.84:0/1689468012 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c031640 con 0x7f79b00d25e0 2023-10-11T06:11:59.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.986+0000 7f79af7fe640 1 -- 172.21.15.84:0/1689468012 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c0317a0 con 0x7f79b00d25e0 2023-10-11T06:11:59.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.986+0000 7f79af7fe640 1 -- 172.21.15.84:0/1689468012 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c031b30 con 0x7f79b00d25e0 2023-10-11T06:11:59.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 -- 172.21.15.84:0/1689468012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 msgr2=0x7f79b00d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/1689468012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b00d0510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f799c009870 tx=0x7f799c02b8f0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 -- 172.21.15.84:0/1689468012 shutdown_connections 2023-10-11T06:11:59.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/1689468012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b00d0510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/1689468012 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b00af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 -- 172.21.15.84:0/1689468012 >> 172.21.15.84:0/1689468012 conn(0x7f79b0071cc0 msgr2=0x7f79b009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:11:59.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 -- 172.21.15.84:0/1689468012 shutdown_connections 2023-10-11T06:11:59.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.987+0000 7f79b6e1a640 1 -- 172.21.15.84:0/1689468012 wait complete. 2023-10-11T06:11:59.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.989+0000 7f79b6e1a640 1 Processor -- start 2023-10-11T06:11:59.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.989+0000 7f79b6e1a640 1 -- start start 2023-10-11T06:11:59.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.990+0000 7f79b6e1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79b00d0a50 0x7f79b00cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.990+0000 7f79b6e1a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b009f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.990+0000 7f79b6e1a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b0157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:11:59.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.990+0000 7f79b6e1a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00ca550 con 0x7f79b00d0a50 2023-10-11T06:11:59.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.990+0000 7f79b6e1a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00ca7f0 con 0x7f79b00d25e0 2023-10-11T06:11:59.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.990+0000 7f79b6e1a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00caad0 con 0x7f79b015c290 2023-10-11T06:11:59.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.990+0000 7f79b4bb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79b00d0a50 0x7f79b00cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79b00d0a50 0x7f79b00cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59216/0 (socket says 172.21.15.84:59216) 2023-10-11T06:11:59.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 -- 172.21.15.84:0/2124739737 learned_addr learned my addr 172.21.15.84:0/2124739737 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:11:59.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b53b7640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b0157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79affff640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b009f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:11:59.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 -- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 msgr2=0x7f79b0157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b0157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 -- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 msgr2=0x7f79b009f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:11:59.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b009f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:11:59.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 -- 172.21.15.84:0/2124739737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c0094d0 con 0x7f79b00d0a50 2023-10-11T06:11:59.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79affff640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b009f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:11:59.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.991+0000 7f79b4bb6640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79b00d0a50 0x7f79b00cb5b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79a400e870 tx=0x7f79a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:11:59.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.992+0000 7f79ad7fa640 1 -- 172.21.15.84:0/2124739737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40094f0 con 0x7f79b00d0a50 2023-10-11T06:11:59.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.992+0000 7f79ad7fa640 1 -- 172.21.15.84:0/2124739737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a400c940 con 0x7f79b00d0a50 2023-10-11T06:11:59.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.992+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b00cadb0 con 0x7f79b00d0a50 2023-10-11T06:11:59.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.992+0000 7f79ad7fa640 1 -- 172.21.15.84:0/2124739737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400ccf0 con 0x7f79b00d0a50 2023-10-11T06:11:59.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.992+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b00cbaf0 con 0x7f79b00d0a50 2023-10-11T06:11:59.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.993+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b0073ed0 con 0x7f79b00d0a50 2023-10-11T06:12:00.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.994+0000 7f79ad7fa640 1 -- 172.21.15.84:0/2124739737 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f79a4013470 con 0x7f79b00d0a50 2023-10-11T06:12:00.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.995+0000 7f79ad7fa640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f799406a830 0x7f799406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.995+0000 7f79ad7fa640 1 -- 172.21.15.84:0/2124739737 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f79a4089280 con 0x7f79b00d0a50 2023-10-11T06:12:00.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.995+0000 7f79affff640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f799406a830 0x7f799406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:11:59.996+0000 7f79affff640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f799406a830 0x7f799406cce0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f799c036000 tx=0x7f799c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:00.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.000+0000 7f79ad7fa640 1 -- 172.21.15.84:0/2124739737 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f79a4054310 con 0x7f79b00d0a50 2023-10-11T06:12:00.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.194+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79b006c720 con 0x7f79b00d0a50 2023-10-11T06:12:00.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.195+0000 7f79ad7fa640 1 -- 172.21.15.84:0/2124739737 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f79a4057930 con 0x7f79b00d0a50 2023-10-11T06:12:00.198 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:00.199 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:00.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f799406a830 msgr2=0x7f799406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f799406a830 0x7f799406cce0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f799c036000 tx=0x7f799c03c040 comp rx=0 tx=0).stop 2023-10-11T06:12:00.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79b00d0a50 msgr2=0x7f79b00cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79b00d0a50 0x7f79b00cb5b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79a400e870 tx=0x7f79a400ed30 comp rx=0 tx=0).stop 2023-10-11T06:12:00.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 shutdown_connections 2023-10-11T06:12:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f799406a830 0x7f799406cce0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79b015c290 0x7f79b0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79b00d25e0 0x7f79b009f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 --2- 172.21.15.84:0/2124739737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79b00d0a50 0x7f79b00cb5b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.198+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 >> 172.21.15.84:0/2124739737 conn(0x7f79b0071cc0 msgr2=0x7f79b01596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.199+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 shutdown_connections 2023-10-11T06:12:00.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.199+0000 7f79b6e1a640 1 -- 172.21.15.84:0/2124739737 wait complete. 2023-10-11T06:12:00.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:00.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:00.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.435+0000 7f8e4c918640 1 Processor -- start 2023-10-11T06:12:00.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.435+0000 7f8e4c918640 1 -- start start 2023-10-11T06:12:00.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.437+0000 7f8e4c918640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e480d2f40 0x7f8e480d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.437+0000 7f8e4c918640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e480cf220 con 0x7f8e480d2ac0 2023-10-11T06:12:00.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.437+0000 7f8e4c918640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e480cf500 con 0x7f8e4815f7e0 2023-10-11T06:12:00.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.437+0000 7f8e4c918640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e480cf7e0 con 0x7f8e480d2f40 2023-10-11T06:12:00.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.437+0000 7f8e3ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8e480d2ac0 0x7f8e480d0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44766/0 (socket says 172.21.15.84:44766) 2023-10-11T06:12:00.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.437+0000 7f8e3ffff640 1 -- 172.21.15.84:0/1985099767 learned_addr learned my addr 172.21.15.84:0/1985099767 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:00.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e46d9d640 1 --2- 172.21.15.84:0/1985099767 >> v2:172.21.15.99:3300/0 conn(0x7f8e480d2f40 0x7f8e480d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741612898 0 0) 0x7f8e480cf220 con 0x7f8e480d2ac0 2023-10-11T06:12:00.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e34003520 con 0x7f8e480d2ac0 2023-10-11T06:12:00.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2861092784 0 0) 0x7f8e34003520 con 0x7f8e480d2ac0 2023-10-11T06:12:00.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 >> v2:172.21.15.99:3300/0 conn(0x7f8e480d2f40 msgr2=0x7f8e480d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:00.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 --2- 172.21.15.84:0/1985099767 >> v2:172.21.15.99:3300/0 conn(0x7f8e480d2f40 0x7f8e480d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 >> v1:172.21.15.99:6789/0 conn(0x7f8e4815f7e0 legacy=0x7f8e480cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480cfac0 con 0x7f8e480d2ac0 2023-10-11T06:12:00.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f8e30001670 con 0x7f8e480d2ac0 2023-10-11T06:12:00.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 >> v1:172.21.15.84:6789/0 conn(0x7f8e480d2ac0 legacy=0x7f8e480d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.438+0000 7f8e45d9b640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e34003970 0x7f8e34003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e45d9b640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e34004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e45d9b640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34005170 0x7f8e3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480cf7e0 con 0x7f8e340047c0 2023-10-11T06:12:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3400bff0 con 0x7f8e34003970 2023-10-11T06:12:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3400c1b0 con 0x7f8e34005170 2023-10-11T06:12:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e3ffff640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e34004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e46d9d640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34005170 0x7f8e3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e3ffff640 1 -- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34005170 msgr2=0x7f8e3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e4659c640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e34003970 0x7f8e34003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e3ffff640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34005170 0x7f8e3400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e3ffff640 1 -- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e34003970 msgr2=0x7f8e34003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e3ffff640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e34003970 0x7f8e34003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e3ffff640 1 -- 172.21.15.84:0/1985099767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e34003520 con 0x7f8e340047c0 2023-10-11T06:12:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e46d9d640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e34005170 0x7f8e3400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.439+0000 7f8e46d9d640 1 -- 172.21.15.84:0/1985099767 reap_dead start 2023-10-11T06:12:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.440+0000 7f8e3ffff640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e34004c30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c240 tx=0x7f8e340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f53b23aa2a54e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30005ad0 con 0x7f8e340047c0 2023-10-11T06:12:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30005c30 con 0x7f8e340047c0 2023-10-11T06:12:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e45d9b640 1 -- 172.21.15.84:0/1985099767 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3002ca00 con 0x7f8e340047c0 2023-10-11T06:12:00.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e4c918640 1 -- 172.21.15.84:0/1985099767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 msgr2=0x7f8e34004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e4c918640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e34004c30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c240 tx=0x7f8e340057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e4c918640 1 -- 172.21.15.84:0/1985099767 shutdown_connections 2023-10-11T06:12:00.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e4c918640 1 --2- 172.21.15.84:0/1985099767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e34004c30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e4c918640 1 -- 172.21.15.84:0/1985099767 >> 172.21.15.84:0/1985099767 conn(0x7f8e480713a0 msgr2=0x7f8e480717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:00.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e4c918640 1 -- 172.21.15.84:0/1985099767 shutdown_connections 2023-10-11T06:12:00.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.441+0000 7f8e4c918640 1 -- 172.21.15.84:0/1985099767 wait complete. 2023-10-11T06:12:00.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.443+0000 7f8e4c918640 1 Processor -- start 2023-10-11T06:12:00.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.443+0000 7f8e4c918640 1 -- start start 2023-10-11T06:12:00.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.444+0000 7f8e4c918640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e480cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.444+0000 7f8e4c918640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e480cc260 0x7f8e480c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.444+0000 7f8e4c918640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e480cc6d0 0x7f8e480c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.444+0000 7f8e4c918640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480cac20 con 0x7f8e340047c0 2023-10-11T06:12:00.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.444+0000 7f8e4c918640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480c9fa0 con 0x7f8e480cc260 2023-10-11T06:12:00.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.444+0000 7f8e4c918640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480ca280 con 0x7f8e480cc6d0 2023-10-11T06:12:00.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e480cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e480cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59232/0 (socket says 172.21.15.84:59232) 2023-10-11T06:12:00.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 -- 172.21.15.84:0/3793539368 learned_addr learned my addr 172.21.15.84:0/3793539368 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:00.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 -- 172.21.15.84:0/3793539368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e480cc6d0 msgr2=0x7f8e480c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e480cc6d0 0x7f8e480c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 -- 172.21.15.84:0/3793539368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e480cc260 msgr2=0x7f8e480c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e480cc260 0x7f8e480c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 -- 172.21.15.84:0/3793539368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480ca560 con 0x7f8e340047c0 2023-10-11T06:12:00.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e4659c640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e480cb040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800d570 tx=0x7f8e3800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:00.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.445+0000 7f8e3effd640 1 -- 172.21.15.84:0/3793539368 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003610 con 0x7f8e340047c0 2023-10-11T06:12:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.446+0000 7f8e3effd640 1 -- 172.21.15.84:0/3793539368 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38003770 con 0x7f8e340047c0 2023-10-11T06:12:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.446+0000 7f8e3effd640 1 -- 172.21.15.84:0/3793539368 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003b20 con 0x7f8e340047c0 2023-10-11T06:12:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.446+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e480ca840 con 0x7f8e340047c0 2023-10-11T06:12:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.446+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e480cb580 con 0x7f8e340047c0 2023-10-11T06:12:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.447+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e480cae90 con 0x7f8e340047c0 2023-10-11T06:12:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.448+0000 7f8e3effd640 1 -- 172.21.15.84:0/3793539368 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8e38003c80 con 0x7f8e340047c0 2023-10-11T06:12:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.448+0000 7f8e3effd640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2806a830 0x7f8e2806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.449+0000 7f8e3effd640 1 -- 172.21.15.84:0/3793539368 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8e38088ed0 con 0x7f8e340047c0 2023-10-11T06:12:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.449+0000 7f8e3ffff640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2806a830 0x7f8e2806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.450+0000 7f8e3ffff640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2806a830 0x7f8e2806cce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e30034000 tx=0x7f8e30039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.455+0000 7f8e3effd640 1 -- 172.21.15.84:0/3793539368 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8e380541c0 con 0x7f8e340047c0 2023-10-11T06:12:00.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.642+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e48073ed0 con 0x7f8e340047c0 2023-10-11T06:12:00.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.643+0000 7f8e3effd640 1 -- 172.21.15.84:0/3793539368 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e380577e0 con 0x7f8e340047c0 2023-10-11T06:12:00.645 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:00.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2806a830 msgr2=0x7f8e2806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2806a830 0x7f8e2806cce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e30034000 tx=0x7f8e30039040 comp rx=0 tx=0).stop 2023-10-11T06:12:00.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 msgr2=0x7f8e480cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e480cb040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800d570 tx=0x7f8e3800da30 comp rx=0 tx=0).stop 2023-10-11T06:12:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 shutdown_connections 2023-10-11T06:12:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e2806a830 0x7f8e2806cce0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e480cc6d0 0x7f8e480c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e480cc260 0x7f8e480c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 --2- 172.21.15.84:0/3793539368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e340047c0 0x7f8e480cb040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.645+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 >> 172.21.15.84:0/3793539368 conn(0x7f8e480713a0 msgr2=0x7f8e48071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.646+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 shutdown_connections 2023-10-11T06:12:00.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.646+0000 7f8e4c918640 1 -- 172.21.15.84:0/3793539368 wait complete. 2023-10-11T06:12:00.668 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:00.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.882+0000 7f9fa7d73640 1 Processor -- start 2023-10-11T06:12:00.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.883+0000 7f9fa7d73640 1 -- start start 2023-10-11T06:12:00.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.884+0000 7f9fa7d73640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fa00b0040 0x7f9fa00b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.884+0000 7f9fa7d73640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa00c8e50 0x7f9fa00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.884+0000 7f9fa7d73640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa006c7b0 con 0x7f9fa0158fc0 2023-10-11T06:12:00.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.884+0000 7f9fa7d73640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa00b0a00 con 0x7f9fa00b0040 2023-10-11T06:12:00.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.884+0000 7f9fa7d73640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa0159390 con 0x7f9fa00c8e50 2023-10-11T06:12:00.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.885+0000 7f9fa6310640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa00c8e50 0x7f9fa00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.885+0000 7f9fa530e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fa00b0040 0x7f9fa00b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.885+0000 7f9fa6310640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa00c8e50 0x7f9fa00c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:50700/0 (socket says 172.21.15.84:50700) 2023-10-11T06:12:00.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.885+0000 7f9fa6310640 1 -- 172.21.15.84:0/240727202 learned_addr learned my addr 172.21.15.84:0/240727202 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:00.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.886+0000 7f9fa4b0d640 1 -- 172.21.15.84:0/240727202 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878201717 0 0) 0x7f9fa006c7b0 con 0x7f9fa0158fc0 2023-10-11T06:12:00.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.886+0000 7f9fa4b0d640 1 -- 172.21.15.84:0/240727202 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c003520 con 0x7f9fa0158fc0 2023-10-11T06:12:00.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.886+0000 7f9fa6310640 1 -- 172.21.15.84:0/240727202 >> v2:172.21.15.99:3300/0 conn(0x7f9fa00b0040 msgr2=0x7f9fa00b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:00.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.886+0000 7f9fa6310640 1 --2- 172.21.15.84:0/240727202 >> v2:172.21.15.99:3300/0 conn(0x7f9fa00b0040 0x7f9fa00b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.886+0000 7f9fa6310640 1 -- 172.21.15.84:0/240727202 >> v1:172.21.15.99:6789/0 conn(0x7f9fa0158fc0 legacy=0x7f9fa00ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.886+0000 7f9fa6310640 1 -- 172.21.15.84:0/240727202 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00cd9a0 con 0x7f9fa00c8e50 2023-10-11T06:12:00.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.887+0000 7f9fa6310640 1 --2- 172.21.15.84:0/240727202 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa00c8e50 0x7f9fa00c92d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800d150 tx=0x7f9f9802e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbade94cfeab4915 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:00.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa4b0d640 1 -- 172.21.15.84:0/240727202 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800b0d0 con 0x7f9fa00c8e50 2023-10-11T06:12:00.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa4b0d640 1 -- 172.21.15.84:0/240727202 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9800b230 con 0x7f9fa00c8e50 2023-10-11T06:12:00.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa4b0d640 1 -- 172.21.15.84:0/240727202 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98009600 con 0x7f9fa00c8e50 2023-10-11T06:12:00.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 -- 172.21.15.84:0/240727202 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa00c8e50 msgr2=0x7f9fa00c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/240727202 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa00c8e50 0x7f9fa00c92d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800d150 tx=0x7f9f9802e160 comp rx=0 tx=0).stop 2023-10-11T06:12:00.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 -- 172.21.15.84:0/240727202 shutdown_connections 2023-10-11T06:12:00.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/240727202 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa00c8e50 0x7f9fa00c92d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/240727202 >> v2:172.21.15.99:3300/0 conn(0x7f9fa00b0040 0x7f9fa00b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 -- 172.21.15.84:0/240727202 >> 172.21.15.84:0/240727202 conn(0x7f9fa006a8b0 msgr2=0x7f9fa006bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:00.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 -- 172.21.15.84:0/240727202 shutdown_connections 2023-10-11T06:12:00.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.888+0000 7f9fa7d73640 1 -- 172.21.15.84:0/240727202 wait complete. 2023-10-11T06:12:00.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.890+0000 7f9fa7d73640 1 Processor -- start 2023-10-11T06:12:00.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.890+0000 7f9fa7d73640 1 -- start start 2023-10-11T06:12:00.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.891+0000 7f9fa7d73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fa00b0040 0x7f9fa014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.891+0000 7f9fa7d73640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa00c8e50 0x7f9fa00c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.891+0000 7f9fa7d73640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa0158fc0 0x7f9fa00c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.891+0000 7f9fa7d73640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00ac570 con 0x7f9fa00c8e50 2023-10-11T06:12:00.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.891+0000 7f9fa7d73640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00c6760 con 0x7f9fa00b0040 2023-10-11T06:12:00.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.891+0000 7f9fa7d73640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00c6a40 con 0x7f9fa0158fc0 2023-10-11T06:12:00.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.891+0000 7f9fa530e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa00c8e50 0x7f9fa00c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa00c8e50 0x7f9fa00c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59236/0 (socket says 172.21.15.84:59236) 2023-10-11T06:12:00.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 -- 172.21.15.84:0/2684982143 learned_addr learned my addr 172.21.15.84:0/2684982143 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:00.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa5b0f640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fa00b0040 0x7f9fa014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa6310640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa0158fc0 0x7f9fa00c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 -- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa0158fc0 msgr2=0x7f9fa00c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa0158fc0 0x7f9fa00c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 -- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fa00b0040 msgr2=0x7f9fa014ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:00.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fa00b0040 0x7f9fa014ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:00.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 -- 172.21.15.84:0/2684982143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9800bd40 con 0x7f9fa00c8e50 2023-10-11T06:12:00.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa5b0f640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fa00b0040 0x7f9fa014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:00.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9fa530e640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa00c8e50 0x7f9fa00c5df0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e9b0 tx=0x7f9f9000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:00.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.892+0000 7f9f967fc640 1 -- 172.21.15.84:0/2684982143 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90009670 con 0x7f9fa00c8e50 2023-10-11T06:12:00.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.893+0000 7f9f967fc640 1 -- 172.21.15.84:0/2684982143 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9000c9a0 con 0x7f9fa00c8e50 2023-10-11T06:12:00.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.893+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa00c6d20 con 0x7f9fa00c8e50 2023-10-11T06:12:00.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.893+0000 7f9f967fc640 1 -- 172.21.15.84:0/2684982143 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000cd50 con 0x7f9fa00c8e50 2023-10-11T06:12:00.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.893+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa014f2c0 con 0x7f9fa00c8e50 2023-10-11T06:12:00.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.894+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa00c7240 con 0x7f9fa00c8e50 2023-10-11T06:12:00.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.895+0000 7f9f967fc640 1 -- 172.21.15.84:0/2684982143 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9f90013470 con 0x7f9fa00c8e50 2023-10-11T06:12:00.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.895+0000 7f9f967fc640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f8406a830 0x7f9f8406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:00.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.896+0000 7f9f967fc640 1 -- 172.21.15.84:0/2684982143 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9f90089370 con 0x7f9fa00c8e50 2023-10-11T06:12:00.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.896+0000 7f9fa5b0f640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f8406a830 0x7f9f8406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:00.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.896+0000 7f9fa5b0f640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f8406a830 0x7f9f8406cce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001610 tx=0x7f9f9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:00.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:00.900+0000 7f9f967fc640 1 -- 172.21.15.84:0/2684982143 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9f90054520 con 0x7f9fa00c8e50 2023-10-11T06:12:01.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.095+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9fa0073ed0 con 0x7f9fa00c8e50 2023-10-11T06:12:01.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.096+0000 7f9f967fc640 1 -- 172.21.15.84:0/2684982143 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9f90057b40 con 0x7f9fa00c8e50 2023-10-11T06:12:01.098 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:01.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.098+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f8406a830 msgr2=0x7f9f8406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.098+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f8406a830 0x7f9f8406cce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001610 tx=0x7f9f9c009040 comp rx=0 tx=0).stop 2023-10-11T06:12:01.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa00c8e50 msgr2=0x7f9fa00c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa00c8e50 0x7f9fa00c5df0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e9b0 tx=0x7f9f9000ee70 comp rx=0 tx=0).stop 2023-10-11T06:12:01.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 shutdown_connections 2023-10-11T06:12:01.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f8406a830 0x7f9f8406cce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9fa0158fc0 0x7f9fa00c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa00c8e50 0x7f9fa00c5df0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 --2- 172.21.15.84:0/2684982143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fa00b0040 0x7f9fa014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 >> 172.21.15.84:0/2684982143 conn(0x7f9fa006a8b0 msgr2=0x7f9fa0157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:01.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 shutdown_connections 2023-10-11T06:12:01.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.099+0000 7f9fa7d73640 1 -- 172.21.15.84:0/2684982143 wait complete. 2023-10-11T06:12:01.119 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:01.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.326+0000 7f5633fff640 1 Processor -- start 2023-10-11T06:12:01.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.327+0000 7f5633fff640 1 -- start start 2023-10-11T06:12:01.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.328+0000 7f5633fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56340afdf0 0x7f56340ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.328+0000 7f5633fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f56340cc6b0 0x7f56340cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.328+0000 7f5633fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340cbb30 con 0x7f56340b01c0 2023-10-11T06:12:01.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.328+0000 7f5633fff640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56340c9820 con 0x7f56340cc6b0 2023-10-11T06:12:01.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.328+0000 7f5633fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56340c9b00 con 0x7f56340afdf0 2023-10-11T06:12:01.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56337fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f56340b01c0 0x7f56340b11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44770/0 (socket says 172.21.15.84:44770) 2023-10-11T06:12:01.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56337fe640 1 -- 172.21.15.84:0/522839253 learned_addr learned my addr 172.21.15.84:0/522839253 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:01.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f5632ffd640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56340afdf0 0x7f56340ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f562bfff640 1 --2- 172.21.15.84:0/522839253 >> v2:172.21.15.99:3300/0 conn(0x7f56340cc6b0 0x7f56340cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372875458 0 0) 0x7f56340cbb30 con 0x7f56340b01c0 2023-10-11T06:12:01.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5620003520 con 0x7f56340b01c0 2023-10-11T06:12:01.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2112624675 0 0) 0x7f5620003520 con 0x7f56340b01c0 2023-10-11T06:12:01.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56340afdf0 msgr2=0x7f56340ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:01.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56340afdf0 0x7f56340ca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 >> v2:172.21.15.99:3300/0 conn(0x7f56340cc6b0 msgr2=0x7f56340cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:01.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 --2- 172.21.15.84:0/522839253 >> v2:172.21.15.99:3300/0 conn(0x7f56340cc6b0 0x7f56340cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340c9de0 con 0x7f56340b01c0 2023-10-11T06:12:01.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.329+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f562c0019f0 con 0x7f56340b01c0 2023-10-11T06:12:01.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 >> v1:172.21.15.84:6789/0 conn(0x7f56340b01c0 legacy=0x7f56340b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56327fc640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56340afdf0 0x7f5620003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56327fc640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f56200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56327fc640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004d00 0x7f5620005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56340c9820 con 0x7f5620003ef0 2023-10-11T06:12:01.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56340c9b00 con 0x7f5620004d00 2023-10-11T06:12:01.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562000b980 con 0x7f56340afdf0 2023-10-11T06:12:01.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f562bfff640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f56200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f562bfff640 1 -- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56340afdf0 msgr2=0x7f5620003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:01.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f562bfff640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56340afdf0 0x7f5620003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f56337fe640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004d00 0x7f5620005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f562bfff640 1 -- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004d00 msgr2=0x7f5620005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:01.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f562bfff640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004d00 0x7f5620005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.330+0000 7f562bfff640 1 -- 172.21.15.84:0/522839253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620003520 con 0x7f5620003ef0 2023-10-11T06:12:01.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.331+0000 7f562bfff640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f56200047c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f561c009ac0 tx=0x7f56200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d7c4d332178401 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:01.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c02c710 con 0x7f5620003ef0 2023-10-11T06:12:01.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c02c870 con 0x7f5620003ef0 2023-10-11T06:12:01.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f56327fc640 1 -- 172.21.15.84:0/522839253 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c02cc00 con 0x7f5620003ef0 2023-10-11T06:12:01.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f5633fff640 1 -- 172.21.15.84:0/522839253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 msgr2=0x7f56200047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f5633fff640 1 --2- 172.21.15.84:0/522839253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f56200047c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f561c009ac0 tx=0x7f56200057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f56337fe640 1 -- 172.21.15.84:0/522839253 reap_dead start 2023-10-11T06:12:01.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f5633fff640 1 -- 172.21.15.84:0/522839253 shutdown_connections 2023-10-11T06:12:01.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f5633fff640 1 -- 172.21.15.84:0/522839253 >> 172.21.15.84:0/522839253 conn(0x7f56340cadf0 msgr2=0x7f56340cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:01.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f5633fff640 1 -- 172.21.15.84:0/522839253 shutdown_connections 2023-10-11T06:12:01.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.332+0000 7f5633fff640 1 -- 172.21.15.84:0/522839253 wait complete. 2023-10-11T06:12:01.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.334+0000 7f5633fff640 1 Processor -- start 2023-10-11T06:12:01.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.334+0000 7f5633fff640 1 -- start start 2023-10-11T06:12:01.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.335+0000 7f5633fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f563415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.335+0000 7f5633fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56340c4530 0x7f56340c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.335+0000 7f5633fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f563415c160 0x7f563415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.335+0000 7f5633fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56340ac090 con 0x7f5620003ef0 2023-10-11T06:12:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.335+0000 7f5633fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563415cb10 con 0x7f56340c4530 2023-10-11T06:12:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.335+0000 7f5633fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56340ab060 con 0x7f563415c160 2023-10-11T06:12:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f563415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f563415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59256/0 (socket says 172.21.15.84:59256) 2023-10-11T06:12:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 -- 172.21.15.84:0/3571742635 learned_addr learned my addr 172.21.15.84:0/3571742635 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f562bfff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56340c4530 0x7f56340c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 -- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f563415c160 msgr2=0x7f563415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f56337fe640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f563415c160 0x7f563415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f563415c160 0x7f563415c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 -- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56340c4530 msgr2=0x7f56340c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56340c4530 0x7f56340c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 -- 172.21.15.84:0/3571742635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340ab2d0 con 0x7f5620003ef0 2023-10-11T06:12:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f562bfff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56340c4530 0x7f56340c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.336+0000 7f5632ffd640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f563415a030 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f562400e820 tx=0x7f562400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.337+0000 7f562b7fe640 1 -- 172.21.15.84:0/3571742635 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400c940 con 0x7f5620003ef0 2023-10-11T06:12:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.337+0000 7f562b7fe640 1 -- 172.21.15.84:0/3571742635 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562400caa0 con 0x7f5620003ef0 2023-10-11T06:12:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.337+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56340ab5b0 con 0x7f5620003ef0 2023-10-11T06:12:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.337+0000 7f562b7fe640 1 -- 172.21.15.84:0/3571742635 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400ce30 con 0x7f5620003ef0 2023-10-11T06:12:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.337+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56340abb70 con 0x7f5620003ef0 2023-10-11T06:12:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.338+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563404e890 con 0x7f5620003ef0 2023-10-11T06:12:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.339+0000 7f562b7fe640 1 -- 172.21.15.84:0/3571742635 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5624013470 con 0x7f5620003ef0 2023-10-11T06:12:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.340+0000 7f562b7fe640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.340+0000 7f562b7fe640 1 -- 172.21.15.84:0/3571742635 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f56240890a0 con 0x7f5620003ef0 2023-10-11T06:12:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.341+0000 7f562bfff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.341+0000 7f562bfff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f561c036000 tx=0x7f561c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.345+0000 7f562b7fe640 1 -- 172.21.15.84:0/3571742635 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5624054240 con 0x7f5620003ef0 2023-10-11T06:12:01.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.531+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56340b0ca0 con 0x7f5620003ef0 2023-10-11T06:12:01.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.532+0000 7f562b7fe640 1 -- 172.21.15.84:0/3571742635 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5624057860 con 0x7f5620003ef0 2023-10-11T06:12:01.535 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:01.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 msgr2=0x7f561406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f561c036000 tx=0x7f561c030000 comp rx=0 tx=0).stop 2023-10-11T06:12:01.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 msgr2=0x7f563415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f563415a030 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f562400e820 tx=0x7f562400ece0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 shutdown_connections 2023-10-11T06:12:01.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561406a830 0x7f561406cce0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f563415c160 0x7f563415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56340c4530 0x7f56340c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 --2- 172.21.15.84:0/3571742635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5620003ef0 0x7f563415a030 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 >> 172.21.15.84:0/3571742635 conn(0x7f56340cadf0 msgr2=0x7f5634160250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:01.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 shutdown_connections 2023-10-11T06:12:01.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.535+0000 7f5633fff640 1 -- 172.21.15.84:0/3571742635 wait complete. 2023-10-11T06:12:01.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:01.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.778+0000 7f34e5f67640 1 Processor -- start 2023-10-11T06:12:01.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.778+0000 7f34e5f67640 1 -- start start 2023-10-11T06:12:01.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.780+0000 7f34e5f67640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 0x7f34e00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.780+0000 7f34e5f67640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e0071380 con 0x7f34e00c8d00 2023-10-11T06:12:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.780+0000 7f34e5f67640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e006de10 con 0x7f34e015a0b0 2023-10-11T06:12:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.780+0000 7f34e5f67640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e006e0f0 con 0x7f34e00ced30 2023-10-11T06:12:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.780+0000 7f34dffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f34e00c8d00 0x7f34e006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44774/0 (socket says 172.21.15.84:44774) 2023-10-11T06:12:01.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.780+0000 7f34dffff640 1 -- 172.21.15.84:0/2634751525 learned_addr learned my addr 172.21.15.84:0/2634751525 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:01.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34deffd640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 0x7f34e00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370641826 0 0) 0x7f34e0071380 con 0x7f34e00c8d00 2023-10-11T06:12:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c8003520 con 0x7f34e00c8d00 2023-10-11T06:12:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263189549 0 0) 0x7f34c8003520 con 0x7f34e00c8d00 2023-10-11T06:12:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 msgr2=0x7f34e00cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 0x7f34e00cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 >> v1:172.21.15.99:6789/0 conn(0x7f34e015a0b0 legacy=0x7f34e00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0070dc0 con 0x7f34e00c8d00 2023-10-11T06:12:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f34d80019f0 con 0x7f34e00c8d00 2023-10-11T06:12:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 >> v1:172.21.15.84:6789/0 conn(0x7f34e00c8d00 legacy=0x7f34e006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34c80039a0 0x7f34c8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34c80047c0 0x7f34c8004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 0x7f34c8005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e006e0f0 con 0x7f34c80047c0 2023-10-11T06:12:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c800bb40 con 0x7f34c80039a0 2023-10-11T06:12:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.781+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c800bd00 con 0x7f34e00ced30 2023-10-11T06:12:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34deffd640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34c80047c0 0x7f34c8004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34df7fe640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34c80039a0 0x7f34c8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34dffff640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 0x7f34c8005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34deffd640 1 -- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 msgr2=0x7f34c8005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34deffd640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e00ced30 0x7f34c8005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34deffd640 1 -- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34c80039a0 msgr2=0x7f34c8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34deffd640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34c80039a0 0x7f34c8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.782+0000 7f34deffd640 1 -- 172.21.15.84:0/2634751525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8003520 con 0x7f34c80047c0 2023-10-11T06:12:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.783+0000 7f34deffd640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34c80047c0 0x7f34c8004c30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34e00cf120 tx=0x7f34c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9799fc419e9ad67b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d402c630 con 0x7f34c80047c0 2023-10-11T06:12:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d402c790 con 0x7f34c80047c0 2023-10-11T06:12:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34de7fc640 1 -- 172.21.15.84:0/2634751525 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d402cb20 con 0x7f34c80047c0 2023-10-11T06:12:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34e5f67640 1 -- 172.21.15.84:0/2634751525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34c80047c0 msgr2=0x7f34c8004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34e5f67640 1 --2- 172.21.15.84:0/2634751525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34c80047c0 0x7f34c8004c30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34e00cf120 tx=0x7f34c80057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34dffff640 1 -- 172.21.15.84:0/2634751525 reap_dead start 2023-10-11T06:12:01.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34e5f67640 1 -- 172.21.15.84:0/2634751525 shutdown_connections 2023-10-11T06:12:01.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34e5f67640 1 -- 172.21.15.84:0/2634751525 >> 172.21.15.84:0/2634751525 conn(0x7f34e0069c70 msgr2=0x7f34e006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:01.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34e5f67640 1 -- 172.21.15.84:0/2634751525 shutdown_connections 2023-10-11T06:12:01.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.784+0000 7f34e5f67640 1 -- 172.21.15.84:0/2634751525 wait complete. 2023-10-11T06:12:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.786+0000 7f34e5f67640 1 Processor -- start 2023-10-11T06:12:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.786+0000 7f34e5f67640 1 -- start start 2023-10-11T06:12:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.787+0000 7f34e5f67640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c80047c0 0x7f34e00b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.787+0000 7f34e5f67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e014e090 0x7f34e01460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.787+0000 7f34e5f67640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e00b26c0 0x7f34e006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.787+0000 7f34e5f67640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00b3c40 con 0x7f34e014e090 2023-10-11T06:12:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.787+0000 7f34e5f67640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0146600 con 0x7f34e00b26c0 2023-10-11T06:12:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.787+0000 7f34e5f67640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00b3030 con 0x7f34c80047c0 2023-10-11T06:12:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e014e090 0x7f34e01460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e014e090 0x7f34e01460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59270/0 (socket says 172.21.15.84:59270) 2023-10-11T06:12:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 -- 172.21.15.84:0/1422155785 learned_addr learned my addr 172.21.15.84:0/1422155785 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34df7fe640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c80047c0 0x7f34e00b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 -- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c80047c0 msgr2=0x7f34e00b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c80047c0 0x7f34e00b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 -- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e00b26c0 msgr2=0x7f34e006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34dffff640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e00b26c0 0x7f34e006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e00b26c0 0x7f34e006cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 -- 172.21.15.84:0/1422155785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00b3310 con 0x7f34e014e090 2023-10-11T06:12:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34df7fe640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c80047c0 0x7f34e00b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.788+0000 7f34deffd640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e014e090 0x7f34e01460c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f34d4037040 tx=0x7f34d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.789+0000 7f34c7fff640 1 -- 172.21.15.84:0/1422155785 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4003810 con 0x7f34e014e090 2023-10-11T06:12:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.789+0000 7f34c7fff640 1 -- 172.21.15.84:0/1422155785 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4003970 con 0x7f34e014e090 2023-10-11T06:12:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.789+0000 7f34c7fff640 1 -- 172.21.15.84:0/1422155785 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4003d00 con 0x7f34e014e090 2023-10-11T06:12:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.789+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e00b35f0 con 0x7f34e014e090 2023-10-11T06:12:01.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.789+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e00b2b40 con 0x7f34e014e090 2023-10-11T06:12:01.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.790+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e00b3df0 con 0x7f34e014e090 2023-10-11T06:12:01.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.791+0000 7f34c7fff640 1 -- 172.21.15.84:0/1422155785 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f34d4040450 con 0x7f34e014e090 2023-10-11T06:12:01.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.792+0000 7f34c7fff640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f34cc06a860 0x7f34cc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:01.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.792+0000 7f34c7fff640 1 -- 172.21.15.84:0/1422155785 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f34d40ab2b0 con 0x7f34e014e090 2023-10-11T06:12:01.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.792+0000 7f34df7fe640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f34cc06a860 0x7f34cc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:01.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.792+0000 7f34df7fe640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f34cc06a860 0x7f34cc06cd10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f34e00733d0 tx=0x7f34d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:01.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.796+0000 7f34c7fff640 1 -- 172.21.15.84:0/1422155785 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f34e00b3df0 con 0x7f34e014e090 2023-10-11T06:12:01.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.985+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34e00b3df0 con 0x7f34e014e090 2023-10-11T06:12:01.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.986+0000 7f34c7fff640 1 -- 172.21.15.84:0/1422155785 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34e00b3df0 con 0x7f34e014e090 2023-10-11T06:12:01.988 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.988+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f34cc06a860 msgr2=0x7f34cc06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.988+0000 7f34e5f67640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f34cc06a860 0x7f34cc06cd10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f34e00733d0 tx=0x7f34d0009040 comp rx=0 tx=0).stop 2023-10-11T06:12:01.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.988+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e014e090 msgr2=0x7f34e01460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.988+0000 7f34e5f67640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e014e090 0x7f34e01460c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f34d4037040 tx=0x7f34d4004730 comp rx=0 tx=0).stop 2023-10-11T06:12:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 shutdown_connections 2023-10-11T06:12:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f34cc06a860 0x7f34cc06cd10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e00b26c0 0x7f34e006cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e014e090 0x7f34e01460c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 --2- 172.21.15.84:0/1422155785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c80047c0 0x7f34e00b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:01.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 >> 172.21.15.84:0/1422155785 conn(0x7f34e0069c70 msgr2=0x7f34e00c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:01.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 shutdown_connections 2023-10-11T06:12:01.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:01.989+0000 7f34e5f67640 1 -- 172.21.15.84:0/1422155785 wait complete. 2023-10-11T06:12:02.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:02.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.219+0000 7f9d6fbae640 1 Processor -- start 2023-10-11T06:12:02.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.219+0000 7f9d6fbae640 1 -- start start 2023-10-11T06:12:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.220+0000 7f9d6fbae640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d680ce0d0 0x7f9d680c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6fbae640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 0x7f9d6815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6fbae640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d681593f0 con 0x7f9d680adb90 2023-10-11T06:12:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6fbae640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6815a340 con 0x7f9d680ce0d0 2023-10-11T06:12:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6fbae640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680ce4a0 con 0x7f9d680cd300 2023-10-11T06:12:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6d149640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9d680adb90 0x7f9d680caee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44788/0 (socket says 172.21.15.84:44788) 2023-10-11T06:12:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6d149640 1 -- 172.21.15.84:0/4170294464 learned_addr learned my addr 172.21.15.84:0/4170294464 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6e14b640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 0x7f9d6815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6d94a640 1 --2- 172.21.15.84:0/4170294464 >> v2:172.21.15.99:3300/0 conn(0x7f9d680ce0d0 0x7f9d680c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650004208 0 0) 0x7f9d681593f0 con 0x7f9d680adb90 2023-10-11T06:12:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.221+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d54003520 con 0x7f9d680adb90 2023-10-11T06:12:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1174018385 0 0) 0x7f9d54003520 con 0x7f9d680adb90 2023-10-11T06:12:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 msgr2=0x7f9d6815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 0x7f9d6815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 >> v2:172.21.15.99:3300/0 conn(0x7f9d680ce0d0 msgr2=0x7f9d680c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 --2- 172.21.15.84:0/4170294464 >> v2:172.21.15.99:3300/0 conn(0x7f9d680ce0d0 0x7f9d680c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d681596c0 con 0x7f9d680adb90 2023-10-11T06:12:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f9d58001690 con 0x7f9d680adb90 2023-10-11T06:12:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6d94a640 1 --2- 172.21.15.84:0/4170294464 >> v2:172.21.15.99:3300/0 conn(0x7f9d680ce0d0 0x7f9d680c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 >> v1:172.21.15.84:6789/0 conn(0x7f9d680adb90 legacy=0x7f9d680caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 0x7f9d540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d54004d00 0x7f9d54005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6815a340 con 0x7f9d54003f70 2023-10-11T06:12:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680ce4a0 con 0x7f9d54004d00 2023-10-11T06:12:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5400ba30 con 0x7f9d680cd300 2023-10-11T06:12:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.222+0000 7f9d6d149640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6e14b640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d54004d00 0x7f9d54005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6d94a640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 0x7f9d540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6d149640 1 -- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 msgr2=0x7f9d540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6d149640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 0x7f9d540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6d149640 1 -- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d54004d00 msgr2=0x7f9d54005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6d149640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d54004d00 0x7f9d54005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6d149640 1 -- 172.21.15.84:0/4170294464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d54003520 con 0x7f9d54003f70 2023-10-11T06:12:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6e14b640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d54004d00 0x7f9d54005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.223+0000 7f9d6d94a640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680cd300 0x7f9d540039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.224+0000 7f9d6d149640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d540047c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c160 tx=0x7f9d540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4e71b43cdeb8fd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.224+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58005ad0 con 0x7f9d54003f70 2023-10-11T06:12:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.224+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d58005c30 con 0x7f9d54003f70 2023-10-11T06:12:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.224+0000 7f9d6c948640 1 -- 172.21.15.84:0/4170294464 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5802c7c0 con 0x7f9d54003f70 2023-10-11T06:12:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.225+0000 7f9d6fbae640 1 -- 172.21.15.84:0/4170294464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 msgr2=0x7f9d540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.225+0000 7f9d6fbae640 1 --2- 172.21.15.84:0/4170294464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d540047c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c160 tx=0x7f9d540057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.225+0000 7f9d6e14b640 1 -- 172.21.15.84:0/4170294464 reap_dead start 2023-10-11T06:12:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.225+0000 7f9d6fbae640 1 -- 172.21.15.84:0/4170294464 shutdown_connections 2023-10-11T06:12:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.225+0000 7f9d6fbae640 1 -- 172.21.15.84:0/4170294464 >> 172.21.15.84:0/4170294464 conn(0x7f9d68069c70 msgr2=0x7f9d6806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.225+0000 7f9d6fbae640 1 -- 172.21.15.84:0/4170294464 shutdown_connections 2023-10-11T06:12:02.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.225+0000 7f9d6fbae640 1 -- 172.21.15.84:0/4170294464 wait complete. 2023-10-11T06:12:02.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.227+0000 7f9d6fbae640 1 Processor -- start 2023-10-11T06:12:02.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.227+0000 7f9d6fbae640 1 -- start start 2023-10-11T06:12:02.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6fbae640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d680c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6fbae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d680c2ae0 0x7f9d680ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6fbae640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680c8150 0x7f9d680ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6fbae640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6814c3b0 con 0x7f9d54003f70 2023-10-11T06:12:02.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6fbae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d681f2a90 con 0x7f9d680c2ae0 2023-10-11T06:12:02.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6fbae640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d681f1e80 con 0x7f9d680c8150 2023-10-11T06:12:02.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d94a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d680c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d94a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d680c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59296/0 (socket says 172.21.15.84:59296) 2023-10-11T06:12:02.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d94a640 1 -- 172.21.15.84:0/1829800875 learned_addr learned my addr 172.21.15.84:0/1829800875 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:02.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d94a640 1 -- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680c8150 msgr2=0x7f9d680ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6e14b640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680c8150 0x7f9d680ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d149640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d680c2ae0 0x7f9d680ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d94a640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680c8150 0x7f9d680ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d94a640 1 -- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d680c2ae0 msgr2=0x7f9d680ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.228+0000 7f9d6d94a640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d680c2ae0 0x7f9d680ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.229+0000 7f9d6d94a640 1 -- 172.21.15.84:0/1829800875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d681f2160 con 0x7f9d54003f70 2023-10-11T06:12:02.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.229+0000 7f9d6d94a640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d680c25a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400f650 tx=0x7f9d6400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:02.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.229+0000 7f9d5e7fc640 1 -- 172.21.15.84:0/1829800875 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d640098e0 con 0x7f9d54003f70 2023-10-11T06:12:02.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.229+0000 7f9d5e7fc640 1 -- 172.21.15.84:0/1829800875 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6400c030 con 0x7f9d54003f70 2023-10-11T06:12:02.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.229+0000 7f9d5e7fc640 1 -- 172.21.15.84:0/1829800875 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64009c80 con 0x7f9d54003f70 2023-10-11T06:12:02.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.229+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d681f2440 con 0x7f9d54003f70 2023-10-11T06:12:02.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.229+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d681f1b90 con 0x7f9d54003f70 2023-10-11T06:12:02.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.230+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d68073ed0 con 0x7f9d54003f70 2023-10-11T06:12:02.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.231+0000 7f9d5e7fc640 1 -- 172.21.15.84:0/1829800875 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9d64014480 con 0x7f9d54003f70 2023-10-11T06:12:02.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.232+0000 7f9d5e7fc640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9d4c06a830 0x7f9d4c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.232+0000 7f9d5e7fc640 1 -- 172.21.15.84:0/1829800875 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9d64089400 con 0x7f9d54003f70 2023-10-11T06:12:02.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.232+0000 7f9d6d149640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9d4c06a830 0x7f9d4c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.233+0000 7f9d6d149640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9d4c06a830 0x7f9d4c06cce0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d58034000 tx=0x7f9d58037040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:02.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.237+0000 7f9d5e7fc640 1 -- 172.21.15.84:0/1829800875 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9d64054630 con 0x7f9d54003f70 2023-10-11T06:12:02.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.440+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d680ccb90 con 0x7f9d54003f70 2023-10-11T06:12:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.441+0000 7f9d5e7fc640 1 -- 172.21.15.84:0/1829800875 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9d6405ce60 con 0x7f9d54003f70 2023-10-11T06:12:02.443 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:02.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9d4c06a830 msgr2=0x7f9d4c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9d4c06a830 0x7f9d4c06cce0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d58034000 tx=0x7f9d58037040 comp rx=0 tx=0).stop 2023-10-11T06:12:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 msgr2=0x7f9d680c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d680c25a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400f650 tx=0x7f9d6400fb10 comp rx=0 tx=0).stop 2023-10-11T06:12:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 shutdown_connections 2023-10-11T06:12:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9d4c06a830 0x7f9d4c06cce0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d680c8150 0x7f9d680ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d680c2ae0 0x7f9d680ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 --2- 172.21.15.84:0/1829800875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d54003f70 0x7f9d680c25a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 >> 172.21.15.84:0/1829800875 conn(0x7f9d68069c70 msgr2=0x7f9d6815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 shutdown_connections 2023-10-11T06:12:02.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.444+0000 7f9d6fbae640 1 -- 172.21.15.84:0/1829800875 wait complete. 2023-10-11T06:12:02.464 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:12:02.465 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:02.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:02.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.728+0000 7f8bb8afb640 1 Processor -- start 2023-10-11T06:12:02.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.729+0000 7f8bb8afb640 1 -- start start 2023-10-11T06:12:02.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.730+0000 7f8bb8afb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8bb415c290 0x7f8bb40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.730+0000 7f8bb8afb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb40d2a60 con 0x7f8bb40d25e0 2023-10-11T06:12:02.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.730+0000 7f8bb8afb640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb406cd10 con 0x7f8bb40d0c20 2023-10-11T06:12:02.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.730+0000 7f8bb8afb640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40afb60 con 0x7f8bb415c290 2023-10-11T06:12:02.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.730+0000 7f8bb1d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8bb40d25e0 0x7f8bb40d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44802/0 (socket says 172.21.15.84:44802) 2023-10-11T06:12:02.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb1d9b640 1 -- 172.21.15.84:0/2732833046 learned_addr learned my addr 172.21.15.84:0/2732833046 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:02.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367826212 0 0) 0x7f8bb40d2a60 con 0x7f8bb40d25e0 2023-10-11T06:12:02.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba0003520 con 0x7f8bb40d25e0 2023-10-11T06:12:02.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb259c640 1 --2- 172.21.15.84:0/2732833046 >> v2:172.21.15.99:3300/0 conn(0x7f8bb415c290 0x7f8bb40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3829508703 0 0) 0x7f8ba0003520 con 0x7f8bb40d25e0 2023-10-11T06:12:02.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 >> v2:172.21.15.99:3300/0 conn(0x7f8bb415c290 msgr2=0x7f8bb40af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 --2- 172.21.15.84:0/2732833046 >> v2:172.21.15.99:3300/0 conn(0x7f8bb415c290 0x7f8bb40af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 >> v1:172.21.15.99:6789/0 conn(0x7f8bb40d0c20 legacy=0x7f8bb40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40afe40 con 0x7f8bb40d25e0 2023-10-11T06:12:02.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f8b9c001670 con 0x7f8bb40d25e0 2023-10-11T06:12:02.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.731+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 >> v1:172.21.15.84:6789/0 conn(0x7f8bb40d25e0 legacy=0x7f8bb40d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb159a640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0003970 0x7f8ba0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb159a640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb159a640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba0005170 0x7f8ba000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40afb60 con 0x7f8ba0005170 2023-10-11T06:12:02.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000c080 con 0x7f8ba00047c0 2023-10-11T06:12:02.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000c240 con 0x7f8ba0003970 2023-10-11T06:12:02.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb2d9d640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba0005170 0x7f8ba000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb259c640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0003970 0x7f8ba0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb1d9b640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0003970 msgr2=0x7f8ba0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb2d9d640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0003970 0x7f8ba0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba00047c0 msgr2=0x7f8ba0004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:02.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb2d9d640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.732+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/2732833046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0003520 con 0x7f8ba0005170 2023-10-11T06:12:02.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.733+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/2732833046 reap_dead start 2023-10-11T06:12:02.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.734+0000 7f8bb2d9d640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba0005170 0x7f8ba000b980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40d1010 tx=0x7f8ba00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b43b9b569a34c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:02.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.734+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac007220 con 0x7f8ba0005170 2023-10-11T06:12:02.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.734+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac02e050 con 0x7f8ba0005170 2023-10-11T06:12:02.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.734+0000 7f8bb159a640 1 -- 172.21.15.84:0/2732833046 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0075e0 con 0x7f8ba0005170 2023-10-11T06:12:02.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.734+0000 7f8bb8afb640 1 -- 172.21.15.84:0/2732833046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba0005170 msgr2=0x7f8ba000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.734+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba0005170 0x7f8ba000b980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40d1010 tx=0x7f8ba00057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.735+0000 7f8bb8afb640 1 -- 172.21.15.84:0/2732833046 shutdown_connections 2023-10-11T06:12:02.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.735+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/2732833046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba0005170 0x7f8ba000b980 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.735+0000 7f8bb8afb640 1 -- 172.21.15.84:0/2732833046 >> 172.21.15.84:0/2732833046 conn(0x7f8bb4071cc0 msgr2=0x7f8bb409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:02.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.735+0000 7f8bb8afb640 1 -- 172.21.15.84:0/2732833046 shutdown_connections 2023-10-11T06:12:02.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.735+0000 7f8bb8afb640 1 -- 172.21.15.84:0/2732833046 wait complete. 2023-10-11T06:12:02.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.736+0000 7f8bb8afb640 1 Processor -- start 2023-10-11T06:12:02.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.737+0000 7f8bb8afb640 1 -- start start 2023-10-11T06:12:02.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb8afb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0005170 0x7f8bb41524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb8afb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb41506f0 0x7f8bb4153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb8afb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4153870 0x7f8bb40c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb8afb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40c7690 con 0x7f8bb4153870 2023-10-11T06:12:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb8afb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40c78e0 con 0x7f8bb41506f0 2023-10-11T06:12:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb8afb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40c7bc0 con 0x7f8ba0005170 2023-10-11T06:12:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4153870 0x7f8bb40c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4153870 0x7f8bb40c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59318/0 (socket says 172.21.15.84:59318) 2023-10-11T06:12:02.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/1491281647 learned_addr learned my addr 172.21.15.84:0/1491281647 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:02.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb259c640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0005170 0x7f8bb41524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb1d9b640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb41506f0 0x7f8bb4153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0005170 msgr2=0x7f8bb41524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0005170 0x7f8bb41524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb41506f0 msgr2=0x7f8bb4153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb41506f0 0x7f8bb4153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.738+0000 7f8bb2d9d640 1 -- 172.21.15.84:0/1491281647 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40c7ea0 con 0x7f8bb4153870 2023-10-11T06:12:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.739+0000 7f8bb2d9d640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4153870 0x7f8bb40c86a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bac003e70 tx=0x7f8bac0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.739+0000 7f8baaffd640 1 -- 172.21.15.84:0/1491281647 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac007430 con 0x7f8bb4153870 2023-10-11T06:12:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.739+0000 7f8baaffd640 1 -- 172.21.15.84:0/1491281647 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac007590 con 0x7f8bb4153870 2023-10-11T06:12:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.739+0000 7f8baaffd640 1 -- 172.21.15.84:0/1491281647 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac03e6b0 con 0x7f8bb4153870 2023-10-11T06:12:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.739+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb40c8180 con 0x7f8bb4153870 2023-10-11T06:12:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.739+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb414ec50 con 0x7f8bb4153870 2023-10-11T06:12:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.740+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb414ee10 con 0x7f8bb4153870 2023-10-11T06:12:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.741+0000 7f8baaffd640 1 -- 172.21.15.84:0/1491281647 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8bac03e810 con 0x7f8bb4153870 2023-10-11T06:12:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.742+0000 7f8baaffd640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8b9406a860 0x7f8b9406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.742+0000 7f8baaffd640 1 -- 172.21.15.84:0/1491281647 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8bac0aab50 con 0x7f8bb4153870 2023-10-11T06:12:02.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.743+0000 7f8bb259c640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8b9406a860 0x7f8b9406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.743+0000 7f8bb259c640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8b9406a860 0x7f8b9406cd10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009fd0 tx=0x7f8ba400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.747+0000 7f8baaffd640 1 -- 172.21.15.84:0/1491281647 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8bac075cf0 con 0x7f8bb4153870 2023-10-11T06:12:02.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.938+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bb40c8370 con 0x7f8bb4153870 2023-10-11T06:12:02.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.940+0000 7f8baaffd640 1 -- 172.21.15.84:0/1491281647 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f8bac079310 con 0x7f8bb4153870 2023-10-11T06:12:02.943 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:02.943 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:02.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8b9406a860 msgr2=0x7f8b9406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8b9406a860 0x7f8b9406cd10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009fd0 tx=0x7f8ba400b040 comp rx=0 tx=0).stop 2023-10-11T06:12:02.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4153870 msgr2=0x7f8bb40c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:02.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4153870 0x7f8bb40c86a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bac003e70 tx=0x7f8bac0036f0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 shutdown_connections 2023-10-11T06:12:02.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8b9406a860 0x7f8b9406cd10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bb4153870 0x7f8bb40c86a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb41506f0 0x7f8bb4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 --2- 172.21.15.84:0/1491281647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ba0005170 0x7f8bb41524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:02.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.943+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 >> 172.21.15.84:0/1491281647 conn(0x7f8bb4071cc0 msgr2=0x7f8bb40b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:02.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.944+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 shutdown_connections 2023-10-11T06:12:02.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:02.944+0000 7f8bb8afb640 1 -- 172.21.15.84:0/1491281647 wait complete. 2023-10-11T06:12:02.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:02.965 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:03.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.181+0000 7fdf1fb7e640 1 Processor -- start 2023-10-11T06:12:03.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.181+0000 7fdf1fb7e640 1 -- start start 2023-10-11T06:12:03.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.183+0000 7fdf1fb7e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf18150300 0x7fdf180c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.183+0000 7fdf1fb7e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180c9b90 con 0x7fdf1814ff30 2023-10-11T06:12:03.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.183+0000 7fdf1fb7e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1815da10 con 0x7fdf180ce510 2023-10-11T06:12:03.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.183+0000 7fdf1fb7e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf180cc990 con 0x7fdf18150300 2023-10-11T06:12:03.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.183+0000 7fdf1d91a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdf1814ff30 0x7fdf180cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44804/0 (socket says 172.21.15.84:44804) 2023-10-11T06:12:03.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.183+0000 7fdf1d91a640 1 -- 172.21.15.84:0/1230035870 learned_addr learned my addr 172.21.15.84:0/1230035870 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:03.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.183+0000 7fdf1d119640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf18150300 0x7fdf180c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:03.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621596672 0 0) 0x7fdf180c9b90 con 0x7fdf1814ff30 2023-10-11T06:12:03.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf04003520 con 0x7fdf1814ff30 2023-10-11T06:12:03.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1485696473 0 0) 0x7fdf04003520 con 0x7fdf1814ff30 2023-10-11T06:12:03.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf18150300 msgr2=0x7fdf180c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:03.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf18150300 0x7fdf180c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 >> v1:172.21.15.99:6789/0 conn(0x7fdf180ce510 legacy=0x7fdf180ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1815ce00 con 0x7fdf1814ff30 2023-10-11T06:12:03.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fdf14001670 con 0x7fdf1814ff30 2023-10-11T06:12:03.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 >> v1:172.21.15.84:6789/0 conn(0x7fdf1814ff30 legacy=0x7fdf180cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf18150300 0x7fdf040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf04003f70 0x7fdf040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf04005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf180cc990 con 0x7fdf04004d00 2023-10-11T06:12:03.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0400bb40 con 0x7fdf04003f70 2023-10-11T06:12:03.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0400df10 con 0x7fdf18150300 2023-10-11T06:12:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.184+0000 7fdf1e11b640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf04005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.185+0000 7fdf1e11b640 1 -- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf18150300 msgr2=0x7fdf040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.185+0000 7fdf1e11b640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf18150300 0x7fdf040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.185+0000 7fdf1e11b640 1 -- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf04003f70 msgr2=0x7fdf040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:03.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.185+0000 7fdf1e11b640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf04003f70 0x7fdf040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.185+0000 7fdf1e11b640 1 -- 172.21.15.84:0/1230035870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04003520 con 0x7fdf04004d00 2023-10-11T06:12:03.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.186+0000 7fdf1e11b640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf04005170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdf10001e50 tx=0x7fdf040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25126af3a40e2e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:03.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.186+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10003b70 con 0x7fdf04004d00 2023-10-11T06:12:03.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.186+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf10003cd0 con 0x7fdf04004d00 2023-10-11T06:12:03.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.186+0000 7fdf1c918640 1 -- 172.21.15.84:0/1230035870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10005310 con 0x7fdf04004d00 2023-10-11T06:12:03.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.187+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/1230035870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 msgr2=0x7fdf04005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.187+0000 7fdf1fb7e640 1 --2- 172.21.15.84:0/1230035870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf04005170 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdf10001e50 tx=0x7fdf040057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.187+0000 7fdf1e11b640 1 -- 172.21.15.84:0/1230035870 reap_dead start 2023-10-11T06:12:03.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.187+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/1230035870 shutdown_connections 2023-10-11T06:12:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.187+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/1230035870 >> 172.21.15.84:0/1230035870 conn(0x7fdf18069c70 msgr2=0x7fdf1806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.187+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/1230035870 shutdown_connections 2023-10-11T06:12:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.187+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/1230035870 wait complete. 2023-10-11T06:12:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.189+0000 7fdf1fb7e640 1 Processor -- start 2023-10-11T06:12:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.189+0000 7fdf1fb7e640 1 -- start start 2023-10-11T06:12:03.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1fb7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf1815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1fb7e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf181f1990 0x7fdf1814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1fb7e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf18155dd0 0x7fdf1814d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1fb7e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1815b120 con 0x7fdf04004d00 2023-10-11T06:12:03.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1fb7e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1814e600 con 0x7fdf18155dd0 2023-10-11T06:12:03.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1fb7e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1815a7e0 con 0x7fdf181f1990 2023-10-11T06:12:03.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1d91a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf1815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:03.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1d91a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf1815a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59336/0 (socket says 172.21.15.84:59336) 2023-10-11T06:12:03.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.190+0000 7fdf1d91a640 1 -- 172.21.15.84:0/610580068 learned_addr learned my addr 172.21.15.84:0/610580068 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:03.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1d91a640 1 -- 172.21.15.84:0/610580068 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf181f1990 msgr2=0x7fdf1814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1d91a640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf181f1990 0x7fdf1814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1d91a640 1 -- 172.21.15.84:0/610580068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf18155dd0 msgr2=0x7fdf1814d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1d91a640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf18155dd0 0x7fdf1814d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1d91a640 1 -- 172.21.15.84:0/610580068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf181f1ee0 con 0x7fdf04004d00 2023-10-11T06:12:03.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1d91a640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf1815a2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400c730 tx=0x7fdf1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:03.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf0e7fc640 1 -- 172.21.15.84:0/610580068 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14007350 con 0x7fdf04004d00 2023-10-11T06:12:03.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf0e7fc640 1 -- 172.21.15.84:0/610580068 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf140074b0 con 0x7fdf04004d00 2023-10-11T06:12:03.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf0e7fc640 1 -- 172.21.15.84:0/610580068 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004a20 con 0x7fdf04004d00 2023-10-11T06:12:03.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf181f21c0 con 0x7fdf04004d00 2023-10-11T06:12:03.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.191+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf181f2780 con 0x7fdf04004d00 2023-10-11T06:12:03.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.193+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1815b290 con 0x7fdf04004d00 2023-10-11T06:12:03.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.193+0000 7fdf0e7fc640 1 -- 172.21.15.84:0/610580068 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdf1400a5e0 con 0x7fdf04004d00 2023-10-11T06:12:03.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.194+0000 7fdf0e7fc640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdefc06a830 0x7fdefc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.194+0000 7fdf0e7fc640 1 -- 172.21.15.84:0/610580068 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdf14088530 con 0x7fdf04004d00 2023-10-11T06:12:03.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.195+0000 7fdf1d119640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdefc06a830 0x7fdefc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:03.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.195+0000 7fdf1d119640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdefc06a830 0x7fdefc06cce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf181f4ee0 tx=0x7fdf080094e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:03.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.199+0000 7fdf0e7fc640 1 -- 172.21.15.84:0/610580068 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdf1815b290 con 0x7fdf04004d00 2023-10-11T06:12:03.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.386+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf181f29a0 con 0x7fdf04004d00 2023-10-11T06:12:03.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.387+0000 7fdf0e7fc640 1 -- 172.21.15.84:0/610580068 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf14053800 con 0x7fdf04004d00 2023-10-11T06:12:03.390 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdefc06a830 msgr2=0x7fdefc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdefc06a830 0x7fdefc06cce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdf181f4ee0 tx=0x7fdf080094e0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 msgr2=0x7fdf1815a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf1815a2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400c730 tx=0x7fdf1400cbf0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 shutdown_connections 2023-10-11T06:12:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdefc06a830 0x7fdefc06cce0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf18155dd0 0x7fdf1814d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf181f1990 0x7fdf1814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 --2- 172.21.15.84:0/610580068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf04004d00 0x7fdf1815a2a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 >> 172.21.15.84:0/610580068 conn(0x7fdf18069c70 msgr2=0x7fdf1806ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 shutdown_connections 2023-10-11T06:12:03.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.390+0000 7fdf1fb7e640 1 -- 172.21.15.84:0/610580068 wait complete. 2023-10-11T06:12:03.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:03.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.623+0000 7fce3df77640 1 Processor -- start 2023-10-11T06:12:03.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.624+0000 7fce3df77640 1 -- start start 2023-10-11T06:12:03.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.625+0000 7fce3df77640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce38156810 0x7fce3806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.625+0000 7fce3df77640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce380cde80 con 0x7fce38156be0 2023-10-11T06:12:03.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.625+0000 7fce3df77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3806df80 con 0x7fce3806c880 2023-10-11T06:12:03.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.625+0000 7fce3df77640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce380ce140 con 0x7fce38156810 2023-10-11T06:12:03.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce377fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce38156810 0x7fce3806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:03.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce377fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce38156810 0x7fce3806d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59338/0 (socket says 172.21.15.84:59338) 2023-10-11T06:12:03.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce2ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fce38156be0 0x7fce3806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44806/0 (socket says 172.21.15.84:44806) 2023-10-11T06:12:03.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce377fe640 1 -- 172.21.15.84:0/648486328 learned_addr learned my addr 172.21.15.84:0/648486328 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:03.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce377fe640 1 -- 172.21.15.84:0/648486328 >> v1:172.21.15.99:6789/0 conn(0x7fce3806c880 legacy=0x7fce380ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:03.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce377fe640 1 -- 172.21.15.84:0/648486328 >> v1:172.21.15.84:6789/0 conn(0x7fce38156be0 legacy=0x7fce3806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce377fe640 1 -- 172.21.15.84:0/648486328 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce380cc3b0 con 0x7fce38156810 2023-10-11T06:12:03.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.626+0000 7fce36ffd640 1 -- 172.21.15.84:0/648486328 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084854164 0 0) 0x7fce380cde80 con 0x7fce38156be0 2023-10-11T06:12:03.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.627+0000 7fce377fe640 1 --2- 172.21.15.84:0/648486328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156810 0x7fce3806d8e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce28009870 tx=0x7fce2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb25dc8718f1152 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:03.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce36ffd640 1 -- 172.21.15.84:0/648486328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28031870 con 0x7fce38156810 2023-10-11T06:12:03.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce36ffd640 1 -- 172.21.15.84:0/648486328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce280319d0 con 0x7fce38156810 2023-10-11T06:12:03.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce36ffd640 1 -- 172.21.15.84:0/648486328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28031d60 con 0x7fce38156810 2023-10-11T06:12:03.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce3df77640 1 -- 172.21.15.84:0/648486328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156810 msgr2=0x7fce3806d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce3df77640 1 --2- 172.21.15.84:0/648486328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156810 0x7fce3806d8e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce28009870 tx=0x7fce2802b910 comp rx=0 tx=0).stop 2023-10-11T06:12:03.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce3df77640 1 -- 172.21.15.84:0/648486328 shutdown_connections 2023-10-11T06:12:03.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce3df77640 1 --2- 172.21.15.84:0/648486328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156810 0x7fce3806d8e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce3df77640 1 -- 172.21.15.84:0/648486328 >> 172.21.15.84:0/648486328 conn(0x7fce38069c70 msgr2=0x7fce3806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:03.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce3df77640 1 -- 172.21.15.84:0/648486328 shutdown_connections 2023-10-11T06:12:03.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.628+0000 7fce3df77640 1 -- 172.21.15.84:0/648486328 wait complete. 2023-10-11T06:12:03.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.630+0000 7fce3df77640 1 Processor -- start 2023-10-11T06:12:03.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.630+0000 7fce3df77640 1 -- start start 2023-10-11T06:12:03.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce3df77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce3806c880 0x7fce380af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce3df77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38156810 0x7fce380ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce3df77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156be0 0x7fce381f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce3df77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce380ada80 con 0x7fce38156be0 2023-10-11T06:12:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce3df77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce381f3070 con 0x7fce38156810 2023-10-11T06:12:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce3df77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce381f3350 con 0x7fce3806c880 2023-10-11T06:12:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156be0 0x7fce381f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:03.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156be0 0x7fce381f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59346/0 (socket says 172.21.15.84:59346) 2023-10-11T06:12:03.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.631+0000 7fce37fff640 1 -- 172.21.15.84:0/2783790259 learned_addr learned my addr 172.21.15.84:0/2783790259 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:03.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce37fff640 1 -- 172.21.15.84:0/2783790259 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce3806c880 msgr2=0x7fce380af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce37fff640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce3806c880 0x7fce380af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce37fff640 1 -- 172.21.15.84:0/2783790259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38156810 msgr2=0x7fce380ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce37fff640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38156810 0x7fce380ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce37fff640 1 -- 172.21.15.84:0/2783790259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce280094d0 con 0x7fce38156be0 2023-10-11T06:12:03.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce37fff640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156be0 0x7fce381f2530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce3000c700 tx=0x7fce3000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:03.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce34ff9640 1 -- 172.21.15.84:0/2783790259 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000aa50 con 0x7fce38156be0 2023-10-11T06:12:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce34ff9640 1 -- 172.21.15.84:0/2783790259 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3000abb0 con 0x7fce38156be0 2023-10-11T06:12:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce381f3630 con 0x7fce38156be0 2023-10-11T06:12:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.632+0000 7fce34ff9640 1 -- 172.21.15.84:0/2783790259 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3000ce10 con 0x7fce38156be0 2023-10-11T06:12:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.633+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce381f3bf0 con 0x7fce38156be0 2023-10-11T06:12:03.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.634+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce380cbd30 con 0x7fce38156be0 2023-10-11T06:12:03.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.635+0000 7fce34ff9640 1 -- 172.21.15.84:0/2783790259 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fce30012420 con 0x7fce38156be0 2023-10-11T06:12:03.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.636+0000 7fce34ff9640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce1806a830 0x7fce1806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:03.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.636+0000 7fce34ff9640 1 -- 172.21.15.84:0/2783790259 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fce30089540 con 0x7fce38156be0 2023-10-11T06:12:03.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.636+0000 7fce377fe640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce1806a830 0x7fce1806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:03.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.637+0000 7fce377fe640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce1806a830 0x7fce1806cce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce28036000 tx=0x7fce28030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:03.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.640+0000 7fce34ff9640 1 -- 172.21.15.84:0/2783790259 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fce300546e0 con 0x7fce38156be0 2023-10-11T06:12:03.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.834+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fce381f3de0 con 0x7fce38156be0 2023-10-11T06:12:03.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.835+0000 7fce34ff9640 1 -- 172.21.15.84:0/2783790259 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fce30003eb0 con 0x7fce38156be0 2023-10-11T06:12:03.838 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:03.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce1806a830 msgr2=0x7fce1806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce1806a830 0x7fce1806cce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce28036000 tx=0x7fce28030000 comp rx=0 tx=0).stop 2023-10-11T06:12:03.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156be0 msgr2=0x7fce381f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:03.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156be0 0x7fce381f2530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce3000c700 tx=0x7fce3000cbc0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 shutdown_connections 2023-10-11T06:12:03.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce1806a830 0x7fce1806cce0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce38156be0 0x7fce381f2530 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce38156810 0x7fce380ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 --2- 172.21.15.84:0/2783790259 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce3806c880 0x7fce380af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:03.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.838+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 >> 172.21.15.84:0/2783790259 conn(0x7fce38069c70 msgr2=0x7fce3815df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:03.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.839+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 shutdown_connections 2023-10-11T06:12:03.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:03.839+0000 7fce3df77640 1 -- 172.21.15.84:0/2783790259 wait complete. 2023-10-11T06:12:03.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:04.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.071+0000 7f7c3738e640 1 Processor -- start 2023-10-11T06:12:04.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.072+0000 7f7c3738e640 1 -- start start 2023-10-11T06:12:04.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.073+0000 7f7c3738e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7c30150300 0x7f7c300c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.073+0000 7f7c3738e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c300ce510 0x7f7c300ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.073+0000 7f7c3738e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c300c9b90 con 0x7f7c3014ff30 2023-10-11T06:12:04.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.073+0000 7f7c3738e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3015da10 con 0x7f7c30150300 2023-10-11T06:12:04.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.073+0000 7f7c3738e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300cc990 con 0x7f7c300ce510 2023-10-11T06:12:04.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.074+0000 7f7c3592b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c300ce510 0x7f7c300ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.074+0000 7f7c34929640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7c30150300 0x7f7c300c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.074+0000 7f7c3592b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c300ce510 0x7f7c300ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:50836/0 (socket says 172.21.15.84:50836) 2023-10-11T06:12:04.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.074+0000 7f7c3592b640 1 -- 172.21.15.84:0/1483243053 learned_addr learned my addr 172.21.15.84:0/1483243053 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:04.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.074+0000 7f7c34929640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7c30150300 0x7f7c300c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:46404/0 (socket says 172.21.15.84:46404) 2023-10-11T06:12:04.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.075+0000 7f7c3592b640 1 -- 172.21.15.84:0/1483243053 >> v2:172.21.15.99:3300/0 conn(0x7f7c30150300 msgr2=0x7f7c300c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:04.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.075+0000 7f7c3592b640 1 --2- 172.21.15.84:0/1483243053 >> v2:172.21.15.99:3300/0 conn(0x7f7c30150300 0x7f7c300c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.075+0000 7f7c3592b640 1 -- 172.21.15.84:0/1483243053 >> v1:172.21.15.99:6789/0 conn(0x7f7c3014ff30 legacy=0x7f7c300cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.075+0000 7f7c3592b640 1 -- 172.21.15.84:0/1483243053 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300ceed0 con 0x7f7c300ce510 2023-10-11T06:12:04.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.075+0000 7f7c34929640 1 --2- 172.21.15.84:0/1483243053 >> v2:172.21.15.99:3300/0 conn(0x7f7c30150300 0x7f7c300c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:04.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.076+0000 7f7c3592b640 1 --2- 172.21.15.84:0/1483243053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c300ce510 0x7f7c300ce990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009e50 tx=0x7f7c2802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63558295f88c2f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.076+0000 7f7c27fff640 1 -- 172.21.15.84:0/1483243053 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2802ca80 con 0x7f7c300ce510 2023-10-11T06:12:04.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.076+0000 7f7c27fff640 1 -- 172.21.15.84:0/1483243053 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2802cbe0 con 0x7f7c300ce510 2023-10-11T06:12:04.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c27fff640 1 -- 172.21.15.84:0/1483243053 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280343f0 con 0x7f7c300ce510 2023-10-11T06:12:04.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 -- 172.21.15.84:0/1483243053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c300ce510 msgr2=0x7f7c300ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 --2- 172.21.15.84:0/1483243053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c300ce510 0x7f7c300ce990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009e50 tx=0x7f7c2802bee0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 -- 172.21.15.84:0/1483243053 shutdown_connections 2023-10-11T06:12:04.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 --2- 172.21.15.84:0/1483243053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c300ce510 0x7f7c300ce990 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 --2- 172.21.15.84:0/1483243053 >> v2:172.21.15.99:3300/0 conn(0x7f7c30150300 0x7f7c300c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 -- 172.21.15.84:0/1483243053 >> 172.21.15.84:0/1483243053 conn(0x7f7c30069c70 msgr2=0x7f7c3006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:04.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 -- 172.21.15.84:0/1483243053 shutdown_connections 2023-10-11T06:12:04.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.077+0000 7f7c3738e640 1 -- 172.21.15.84:0/1483243053 wait complete. 2023-10-11T06:12:04.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.079+0000 7f7c3738e640 1 Processor -- start 2023-10-11T06:12:04.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.079+0000 7f7c3738e640 1 -- start start 2023-10-11T06:12:04.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.080+0000 7f7c3738e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c300ce510 0x7f7c3014b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.080+0000 7f7c3738e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c3014ff30 0x7f7c30159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.080+0000 7f7c3738e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c30150300 0x7f7c30157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.080+0000 7f7c3738e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3014bdc0 con 0x7f7c3014ff30 2023-10-11T06:12:04.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.080+0000 7f7c3738e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c30154080 con 0x7f7c300ce510 2023-10-11T06:12:04.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.080+0000 7f7c3738e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c30154360 con 0x7f7c30150300 2023-10-11T06:12:04.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.080+0000 7f7c34929640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c3014ff30 0x7f7c30159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c3014ff30 0x7f7c30159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59352/0 (socket says 172.21.15.84:59352) 2023-10-11T06:12:04.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 -- 172.21.15.84:0/2957375402 learned_addr learned my addr 172.21.15.84:0/2957375402 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:04.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c3512a640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c300ce510 0x7f7c3014b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c3592b640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c30150300 0x7f7c30157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 -- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c30150300 msgr2=0x7f7c30157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c30150300 0x7f7c30157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 -- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c300ce510 msgr2=0x7f7c3014b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c300ce510 0x7f7c3014b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 -- 172.21.15.84:0/2957375402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c20009400 con 0x7f7c3014ff30 2023-10-11T06:12:04.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c34929640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c3014ff30 0x7f7c30159c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e960 tx=0x7f7c2000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.081+0000 7f7c25ffb640 1 -- 172.21.15.84:0/2957375402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000ca00 con 0x7f7c3014ff30 2023-10-11T06:12:04.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.082+0000 7f7c25ffb640 1 -- 172.21.15.84:0/2957375402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2000cb60 con 0x7f7c3014ff30 2023-10-11T06:12:04.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.082+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c28009b30 con 0x7f7c3014ff30 2023-10-11T06:12:04.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.082+0000 7f7c25ffb640 1 -- 172.21.15.84:0/2957375402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20013470 con 0x7f7c3014ff30 2023-10-11T06:12:04.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.082+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c30154920 con 0x7f7c3014ff30 2023-10-11T06:12:04.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.083+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3006ab90 con 0x7f7c3014ff30 2023-10-11T06:12:04.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.084+0000 7f7c25ffb640 1 -- 172.21.15.84:0/2957375402 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f7c200135d0 con 0x7f7c3014ff30 2023-10-11T06:12:04.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.085+0000 7f7c25ffb640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c1406a860 0x7f7c1406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.085+0000 7f7c25ffb640 1 -- 172.21.15.84:0/2957375402 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7c20089380 con 0x7f7c3014ff30 2023-10-11T06:12:04.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.085+0000 7f7c3512a640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c1406a860 0x7f7c1406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.086+0000 7f7c3512a640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c1406a860 0x7f7c1406cd10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c000f60 tx=0x7f7c2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.089+0000 7f7c25ffb640 1 -- 172.21.15.84:0/2957375402 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f7c20054520 con 0x7f7c3014ff30 2023-10-11T06:12:04.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.278+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c30073ed0 con 0x7f7c3014ff30 2023-10-11T06:12:04.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.279+0000 7f7c25ffb640 1 -- 172.21.15.84:0/2957375402 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c20057b40 con 0x7f7c3014ff30 2023-10-11T06:12:04.281 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:04.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.281+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c1406a860 msgr2=0x7f7c1406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.281+0000 7f7c3738e640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c1406a860 0x7f7c1406cd10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c000f60 tx=0x7f7c2c009040 comp rx=0 tx=0).stop 2023-10-11T06:12:04.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.281+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c3014ff30 msgr2=0x7f7c30159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.281+0000 7f7c3738e640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c3014ff30 0x7f7c30159c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e960 tx=0x7f7c2000ee20 comp rx=0 tx=0).stop 2023-10-11T06:12:04.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 shutdown_connections 2023-10-11T06:12:04.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c1406a860 0x7f7c1406cd10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c30150300 0x7f7c30157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c3014ff30 0x7f7c30159c50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 --2- 172.21.15.84:0/2957375402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c300ce510 0x7f7c3014b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 >> 172.21.15.84:0/2957375402 conn(0x7f7c30069c70 msgr2=0x7f7c300704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 shutdown_connections 2023-10-11T06:12:04.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.282+0000 7f7c3738e640 1 -- 172.21.15.84:0/2957375402 wait complete. 2023-10-11T06:12:04.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:04.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.522+0000 7f1076f92640 1 Processor -- start 2023-10-11T06:12:04.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.523+0000 7f1076f92640 1 -- start start 2023-10-11T06:12:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.524+0000 7f1076f92640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f10700d2f40 0x7f10700d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.524+0000 7f1076f92640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700cf220 con 0x7f107015f7e0 2023-10-11T06:12:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.524+0000 7f1076f92640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700cf500 con 0x7f10700d2ac0 2023-10-11T06:12:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.524+0000 7f1076f92640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10700cf7e0 con 0x7f10700d2f40 2023-10-11T06:12:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f1074d2e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f107015f7e0 0x7f10700cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44822/0 (socket says 172.21.15.84:44822) 2023-10-11T06:12:04.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f1074d2e640 1 -- 172.21.15.84:0/1698248076 learned_addr learned my addr 172.21.15.84:0/1698248076 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f107552f640 1 --2- 172.21.15.84:0/1698248076 >> v2:172.21.15.99:3300/0 conn(0x7f10700d2f40 0x7f10700d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881240568 0 0) 0x7f10700cf220 con 0x7f107015f7e0 2023-10-11T06:12:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f105c003520 con 0x7f107015f7e0 2023-10-11T06:12:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2243062139 0 0) 0x7f105c003520 con 0x7f107015f7e0 2023-10-11T06:12:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 >> v2:172.21.15.99:3300/0 conn(0x7f10700d2f40 msgr2=0x7f10700d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 --2- 172.21.15.84:0/1698248076 >> v2:172.21.15.99:3300/0 conn(0x7f10700d2f40 0x7f10700d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 >> v1:172.21.15.99:6789/0 conn(0x7f10700d2ac0 legacy=0x7f10700d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700cfac0 con 0x7f107015f7e0 2023-10-11T06:12:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.525+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f10600019f0 con 0x7f107015f7e0 2023-10-11T06:12:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 >> v1:172.21.15.84:6789/0 conn(0x7f107015f7e0 legacy=0x7f10700cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f106f7fe640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c003970 0x7f105c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f106f7fe640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f105c0047c0 0x7f105c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f106f7fe640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f105c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700cf7e0 con 0x7f105c005170 2023-10-11T06:12:04.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c00bff0 con 0x7f105c003970 2023-10-11T06:12:04.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c00c1b0 con 0x7f105c0047c0 2023-10-11T06:12:04.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f107552f640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f105c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f1074d2e640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c003970 0x7f105c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f107552f640 1 -- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f105c0047c0 msgr2=0x7f105c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f107552f640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f105c0047c0 0x7f105c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f107552f640 1 -- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c003970 msgr2=0x7f105c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f107552f640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c003970 0x7f105c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f107552f640 1 -- 172.21.15.84:0/1698248076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c003520 con 0x7f105c005170 2023-10-11T06:12:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.526+0000 7f1074d2e640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f105c003970 0x7f105c003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.527+0000 7f107552f640 1 -- 172.21.15.84:0/1698248076 reap_dead start 2023-10-11T06:12:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.527+0000 7f107552f640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f105c00b980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1068009ac0 tx=0x7f105c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301de6dd35c88789 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068004020 con 0x7f105c005170 2023-10-11T06:12:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068004180 con 0x7f105c005170 2023-10-11T06:12:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f106f7fe640 1 -- 172.21.15.84:0/1698248076 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106802c710 con 0x7f105c005170 2023-10-11T06:12:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f1076f92640 1 -- 172.21.15.84:0/1698248076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 msgr2=0x7f105c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f1076f92640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f105c00b980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1068009ac0 tx=0x7f105c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f1076f92640 1 -- 172.21.15.84:0/1698248076 shutdown_connections 2023-10-11T06:12:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f1076f92640 1 --2- 172.21.15.84:0/1698248076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f105c00b980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f1076f92640 1 -- 172.21.15.84:0/1698248076 >> 172.21.15.84:0/1698248076 conn(0x7f10700713a0 msgr2=0x7f10700717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f1076f92640 1 -- 172.21.15.84:0/1698248076 shutdown_connections 2023-10-11T06:12:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.528+0000 7f1076f92640 1 -- 172.21.15.84:0/1698248076 wait complete. 2023-10-11T06:12:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.530+0000 7f1076f92640 1 Processor -- start 2023-10-11T06:12:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.530+0000 7f1076f92640 1 -- start start 2023-10-11T06:12:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.531+0000 7f1076f92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f1070153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.531+0000 7f1076f92640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10700cca10 0x7f10700b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.531+0000 7f1076f92640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10700cd140 0x7f10700cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.531+0000 7f1076f92640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700cdaf0 con 0x7f105c005170 2023-10-11T06:12:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.531+0000 7f1076f92640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10701544b0 con 0x7f10700cca10 2023-10-11T06:12:04.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.531+0000 7f1076f92640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1070154a50 con 0x7f10700cd140 2023-10-11T06:12:04.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f1070153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f1070153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59380/0 (socket says 172.21.15.84:59380) 2023-10-11T06:12:04.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 -- 172.21.15.84:0/738386537 learned_addr learned my addr 172.21.15.84:0/738386537 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:04.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 -- 172.21.15.84:0/738386537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10700cd140 msgr2=0x7f10700cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10700cd140 0x7f10700cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 -- 172.21.15.84:0/738386537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10700cca10 msgr2=0x7f10700b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10700cca10 0x7f10700b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 -- 172.21.15.84:0/738386537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070154d30 con 0x7f105c005170 2023-10-11T06:12:04.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f1074d2e640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f1070153f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1060001a10 tx=0x7f106000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.532+0000 7f106d7fa640 1 -- 172.21.15.84:0/738386537 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060004d60 con 0x7f105c005170 2023-10-11T06:12:04.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.533+0000 7f106d7fa640 1 -- 172.21.15.84:0/738386537 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106000be80 con 0x7f105c005170 2023-10-11T06:12:04.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.533+0000 7f106d7fa640 1 -- 172.21.15.84:0/738386537 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600072d0 con 0x7f105c005170 2023-10-11T06:12:04.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.533+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1070155010 con 0x7f105c005170 2023-10-11T06:12:04.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.533+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10701555d0 con 0x7f105c005170 2023-10-11T06:12:04.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.534+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070073ed0 con 0x7f105c005170 2023-10-11T06:12:04.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.535+0000 7f106d7fa640 1 -- 172.21.15.84:0/738386537 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1060007430 con 0x7f105c005170 2023-10-11T06:12:04.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.536+0000 7f106d7fa640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105006a830 0x7f105006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.536+0000 7f106d7fa640 1 -- 172.21.15.84:0/738386537 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1060089020 con 0x7f105c005170 2023-10-11T06:12:04.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.536+0000 7f106ffff640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105006a830 0x7f105006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.537+0000 7f106ffff640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105006a830 0x7f105006cce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10700c61a0 tx=0x7f10580079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.540+0000 7f106d7fa640 1 -- 172.21.15.84:0/738386537 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1060054260 con 0x7f105c005170 2023-10-11T06:12:04.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.729+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f107015ae40 con 0x7f105c005170 2023-10-11T06:12:04.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.729+0000 7f106d7fa640 1 -- 172.21.15.84:0/738386537 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10600190a0 con 0x7f105c005170 2023-10-11T06:12:04.732 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:04.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105006a830 msgr2=0x7f105006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105006a830 0x7f105006cce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10700c61a0 tx=0x7f10580079d0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 msgr2=0x7f1070153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f1070153f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1060001a10 tx=0x7f106000b4a0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 shutdown_connections 2023-10-11T06:12:04.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105006a830 0x7f105006cce0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10700cd140 0x7f10700cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10700cca10 0x7f10700b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 --2- 172.21.15.84:0/738386537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f105c005170 0x7f1070153f70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 >> 172.21.15.84:0/738386537 conn(0x7f10700713a0 msgr2=0x7f1070160740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:04.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 shutdown_connections 2023-10-11T06:12:04.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.732+0000 7f1076f92640 1 -- 172.21.15.84:0/738386537 wait complete. 2023-10-11T06:12:04.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:04.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.965+0000 7fa2b10cc640 1 Processor -- start 2023-10-11T06:12:04.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.965+0000 7fa2b10cc640 1 -- start start 2023-10-11T06:12:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.967+0000 7fa2b10cc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa2ac0ccfb0 0x7fa2ac0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.967+0000 7fa2b10cc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac15a380 con 0x7fa2ac0afc10 2023-10-11T06:12:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.967+0000 7fa2b10cc640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac06def0 con 0x7fa2ac06d980 2023-10-11T06:12:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.967+0000 7fa2b10cc640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac06e1d0 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.967+0000 7fa2abfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa2ac0afc10 0x7fa2ac06ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44828/0 (socket says 172.21.15.84:44828) 2023-10-11T06:12:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.967+0000 7fa2abfff640 1 -- 172.21.15.84:0/1449806617 learned_addr learned my addr 172.21.15.84:0/1449806617 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:04.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.967+0000 7fa2b08cb640 1 --2- 172.21.15.84:0/1449806617 >> v2:172.21.15.99:3300/0 conn(0x7fa2ac0ccfb0 0x7fa2ac0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.968+0000 7fa2b08cb640 1 -- 172.21.15.84:0/1449806617 >> v1:172.21.15.99:6789/0 conn(0x7fa2ac06d980 legacy=0x7fa2ac0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.968+0000 7fa2b08cb640 1 -- 172.21.15.84:0/1449806617 >> v1:172.21.15.84:6789/0 conn(0x7fa2ac0afc10 legacy=0x7fa2ac06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.968+0000 7fa2b08cb640 1 -- 172.21.15.84:0/1449806617 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac06e4b0 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.970+0000 7fa2b08cb640 1 --2- 172.21.15.84:0/1449806617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0ce170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009d90 tx=0x7fa2a402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e525d1e49b2df0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.972+0000 7fa2aaffd640 1 -- 172.21.15.84:0/1449806617 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a402cc60 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.972+0000 7fa2aaffd640 1 -- 172.21.15.84:0/1449806617 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a402cdc0 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.972+0000 7fa2aaffd640 1 -- 172.21.15.84:0/1449806617 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4034660 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.973+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1449806617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac0ccfb0 msgr2=0x7fa2ac0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.973+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1449806617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0ce170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009d90 tx=0x7fa2a402beb0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.973+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1449806617 shutdown_connections 2023-10-11T06:12:04.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.973+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1449806617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0ce170 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.973+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1449806617 >> 172.21.15.84:0/1449806617 conn(0x7fa2ac099910 msgr2=0x7fa2ac0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:04.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.973+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1449806617 shutdown_connections 2023-10-11T06:12:04.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.973+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1449806617 wait complete. 2023-10-11T06:12:04.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.975+0000 7fa2b10cc640 1 Processor -- start 2023-10-11T06:12:04.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.975+0000 7fa2b10cc640 1 -- start start 2023-10-11T06:12:04.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b10cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac06d980 0x7fa2ac157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b10cc640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2ac0afc10 0x7fa2ac0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b10cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b10cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0b4a10 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b10cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0b5370 con 0x7fa2ac06d980 2023-10-11T06:12:04.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b10cc640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0b3d90 con 0x7fa2ac0afc10 2023-10-11T06:12:04.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b08cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b08cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59394/0 (socket says 172.21.15.84:59394) 2023-10-11T06:12:04.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.976+0000 7fa2b08cb640 1 -- 172.21.15.84:0/1122427823 learned_addr learned my addr 172.21.15.84:0/1122427823 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:04.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2b08cb640 1 -- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2ac0afc10 msgr2=0x7fa2ac0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2ab7fe640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2ac0afc10 0x7fa2ac0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2abfff640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac06d980 0x7fa2ac157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2b08cb640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2ac0afc10 0x7fa2ac0b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2b08cb640 1 -- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac06d980 msgr2=0x7fa2ac157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:04.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2b08cb640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac06d980 0x7fa2ac157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:04.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2b08cb640 1 -- 172.21.15.84:0/1122427823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4009a70 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2ab7fe640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2ac0afc10 0x7fa2ac0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:04.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.977+0000 7fa2b08cb640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0b33d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009eb0 tx=0x7fa2a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.978+0000 7fa2a8ff9640 1 -- 172.21.15.84:0/1122427823 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4040650 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.978+0000 7fa2a8ff9640 1 -- 172.21.15.84:0/1122427823 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a40407d0 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.978+0000 7fa2a8ff9640 1 -- 172.21.15.84:0/1122427823 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4040b80 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.978+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac0b4070 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.978+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac0b4630 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.979+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac0b4b50 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.980+0000 7fa2a8ff9640 1 -- 172.21.15.84:0/1122427823 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa2a4040ce0 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.981+0000 7fa2a8ff9640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa29806a830 0x7fa29806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:04.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.981+0000 7fa2a8ff9640 1 -- 172.21.15.84:0/1122427823 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa2a40abf80 con 0x7fa2ac0ccfb0 2023-10-11T06:12:04.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.981+0000 7fa2abfff640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa29806a830 0x7fa29806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:04.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.982+0000 7fa2abfff640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa29806a830 0x7fa29806cce0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005880 tx=0x7fa29c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:04.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:04.985+0000 7fa2a8ff9640 1 -- 172.21.15.84:0/1122427823 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa2ac0b4b50 con 0x7fa2ac0ccfb0 2023-10-11T06:12:05.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.181+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa2ac0b4b50 con 0x7fa2ac0ccfb0 2023-10-11T06:12:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.182+0000 7fa2a8ff9640 1 -- 172.21.15.84:0/1122427823 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa2ac0b4b50 con 0x7fa2ac0ccfb0 2023-10-11T06:12:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:05.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa29806a830 msgr2=0x7fa29806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa29806a830 0x7fa29806cce0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005880 tx=0x7fa29c009040 comp rx=0 tx=0).stop 2023-10-11T06:12:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2ac0ccfb0 msgr2=0x7fa2ac0b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0b33d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009eb0 tx=0x7fa2a4004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 shutdown_connections 2023-10-11T06:12:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa29806a830 0x7fa29806cce0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2ac0ccfb0 0x7fa2ac0b33d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2ac0afc10 0x7fa2ac0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 --2- 172.21.15.84:0/1122427823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2ac06d980 0x7fa2ac157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 >> 172.21.15.84:0/1122427823 conn(0x7fa2ac099910 msgr2=0x7fa2ac0ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:05.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 shutdown_connections 2023-10-11T06:12:05.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.184+0000 7fa2b10cc640 1 -- 172.21.15.84:0/1122427823 wait complete. 2023-10-11T06:12:05.203 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:12:05.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:05.204 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:05.289 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:05.290 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:12:05.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:05.291 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:12:05.356 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:12:05.357 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:05.418 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:12:05.419 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:05.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:05.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.693+0000 7f67e4d22640 1 Processor -- start 2023-10-11T06:12:05.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.693+0000 7f67e4d22640 1 -- start start 2023-10-11T06:12:05.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.694+0000 7f67e4d22640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f67e00cb540 0x7f67e0164f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:05.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.694+0000 7f67e4d22640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e0071a90 con 0x7f67e015d4f0 2023-10-11T06:12:05.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.694+0000 7f67e4d22640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00cb970 con 0x7f67e00ca300 2023-10-11T06:12:05.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.694+0000 7f67e4d22640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e0072310 con 0x7f67e00cb540 2023-10-11T06:12:05.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.695+0000 7f67de59c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f67e015d4f0 0x7f67e00c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44832/0 (socket says 172.21.15.84:44832) 2023-10-11T06:12:05.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.695+0000 7f67de59c640 1 -- 172.21.15.84:0/4267097903 learned_addr learned my addr 172.21.15.84:0/4267097903 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:05.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.695+0000 7f67ded9d640 1 --2- 172.21.15.84:0/4267097903 >> v2:172.21.15.84:3300/0 conn(0x7f67e00cb540 0x7f67e0164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.695+0000 7f67ded9d640 1 -- 172.21.15.84:0/4267097903 >> v1:172.21.15.99:6789/0 conn(0x7f67e00ca300 legacy=0x7f67e00c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.695+0000 7f67ded9d640 1 -- 172.21.15.84:0/4267097903 >> v1:172.21.15.84:6789/0 conn(0x7f67e015d4f0 legacy=0x7f67e00c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.695+0000 7f67ded9d640 1 -- 172.21.15.84:0/4267097903 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0070280 con 0x7f67e00cb540 2023-10-11T06:12:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.695+0000 7f67ddd9b640 1 -- 172.21.15.84:0/4267097903 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821942065 0 0) 0x7f67e0071a90 con 0x7f67e015d4f0 2023-10-11T06:12:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.696+0000 7f67ded9d640 1 --2- 172.21.15.84:0/4267097903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e00cb540 0x7f67e0164f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccd109382c40d65 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.696+0000 7f67ddd9b640 1 -- 172.21.15.84:0/4267097903 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d802cbd0 con 0x7f67e00cb540 2023-10-11T06:12:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67ddd9b640 1 -- 172.21.15.84:0/4267097903 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d802cd30 con 0x7f67e00cb540 2023-10-11T06:12:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67ddd9b640 1 -- 172.21.15.84:0/4267097903 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d802cea0 con 0x7f67e00cb540 2023-10-11T06:12:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67e4d22640 1 -- 172.21.15.84:0/4267097903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e00cb540 msgr2=0x7f67e0164f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67e4d22640 1 --2- 172.21.15.84:0/4267097903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e00cb540 0x7f67e0164f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d802be20 comp rx=0 tx=0).stop 2023-10-11T06:12:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67e4d22640 1 -- 172.21.15.84:0/4267097903 shutdown_connections 2023-10-11T06:12:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67e4d22640 1 --2- 172.21.15.84:0/4267097903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e00cb540 0x7f67e0164f50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67e4d22640 1 -- 172.21.15.84:0/4267097903 >> 172.21.15.84:0/4267097903 conn(0x7f67e0069c70 msgr2=0x7f67e006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67e4d22640 1 -- 172.21.15.84:0/4267097903 shutdown_connections 2023-10-11T06:12:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.697+0000 7f67e4d22640 1 -- 172.21.15.84:0/4267097903 wait complete. 2023-10-11T06:12:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.699+0000 7f67e4d22640 1 Processor -- start 2023-10-11T06:12:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.699+0000 7f67e4d22640 1 -- start start 2023-10-11T06:12:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67e4d22640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e00ca300 0x7f67e014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67e4d22640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67e00cb540 0x7f67e00c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67e4d22640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e015d4f0 0x7f67e01efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67e4d22640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00c7f00 con 0x7f67e015d4f0 2023-10-11T06:12:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67e4d22640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00c72f0 con 0x7f67e00ca300 2023-10-11T06:12:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67e4d22640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00c75d0 con 0x7f67e00cb540 2023-10-11T06:12:05.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e015d4f0 0x7f67e01efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:05.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e015d4f0 0x7f67e01efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59408/0 (socket says 172.21.15.84:59408) 2023-10-11T06:12:05.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 -- 172.21.15.84:0/17384064 learned_addr learned my addr 172.21.15.84:0/17384064 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:05.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 -- 172.21.15.84:0/17384064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67e00cb540 msgr2=0x7f67e00c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67e00cb540 0x7f67e00c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 -- 172.21.15.84:0/17384064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e00ca300 msgr2=0x7f67e014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e00ca300 0x7f67e014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.700+0000 7f67ded9d640 1 -- 172.21.15.84:0/17384064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8009a70 con 0x7f67e015d4f0 2023-10-11T06:12:05.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.701+0000 7f67ded9d640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e015d4f0 0x7f67e01efef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:05.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.701+0000 7f67d6ffd640 1 -- 172.21.15.84:0/17384064 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d802cea0 con 0x7f67e015d4f0 2023-10-11T06:12:05.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.701+0000 7f67d6ffd640 1 -- 172.21.15.84:0/17384064 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d803e630 con 0x7f67e015d4f0 2023-10-11T06:12:05.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.701+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e00c78b0 con 0x7f67e015d4f0 2023-10-11T06:12:05.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.701+0000 7f67d6ffd640 1 -- 172.21.15.84:0/17384064 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d803e9e0 con 0x7f67e015d4f0 2023-10-11T06:12:05.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.701+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e00c8390 con 0x7f67e015d4f0 2023-10-11T06:12:05.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.702+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e0073ed0 con 0x7f67e015d4f0 2023-10-11T06:12:05.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.703+0000 7f67d6ffd640 1 -- 172.21.15.84:0/17384064 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f67d803b050 con 0x7f67e015d4f0 2023-10-11T06:12:05.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.705+0000 7f67d6ffd640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f67c006a830 0x7f67c006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:05.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.705+0000 7f67d6ffd640 1 -- 172.21.15.84:0/17384064 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f67d80ab290 con 0x7f67e015d4f0 2023-10-11T06:12:05.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.705+0000 7f67de59c640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f67c006a830 0x7f67c006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:05.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.706+0000 7f67de59c640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f67c006a830 0x7f67c006cce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f67d0007fd0 tx=0x7f67d00077c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:05.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.709+0000 7f67d6ffd640 1 -- 172.21.15.84:0/17384064 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f67d8076430 con 0x7f67e015d4f0 2023-10-11T06:12:05.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.901+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67e0071f70 con 0x7f67e015d4f0 2023-10-11T06:12:05.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.903+0000 7f67d6ffd640 1 -- 172.21.15.84:0/17384064 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f67d8079a50 con 0x7f67e015d4f0 2023-10-11T06:12:05.906 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:05.906 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:05.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f67c006a830 msgr2=0x7f67c006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f67c006a830 0x7f67c006cce0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f67d0007fd0 tx=0x7f67d00077c0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e015d4f0 msgr2=0x7f67e01efef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:05.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e015d4f0 0x7f67e01efef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d8004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:05.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 shutdown_connections 2023-10-11T06:12:05.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f67c006a830 0x7f67c006cce0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e015d4f0 0x7f67e01efef0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67e00cb540 0x7f67e00c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 --2- 172.21.15.84:0/17384064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e00ca300 0x7f67e014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:05.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 >> 172.21.15.84:0/17384064 conn(0x7f67e0069c70 msgr2=0x7f67e0157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:05.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 shutdown_connections 2023-10-11T06:12:05.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:05.906+0000 7f67e4d22640 1 -- 172.21.15.84:0/17384064 wait complete. 2023-10-11T06:12:05.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:05.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:06.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.140+0000 7fda3a8ca640 1 Processor -- start 2023-10-11T06:12:06.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.140+0000 7fda3a8ca640 1 -- start start 2023-10-11T06:12:06.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda3a8ca640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34156810 0x7fda3406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda3a8ca640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fda34156be0 0x7fda3415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda3a8ca640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda340cde80 con 0x7fda3406c170 2023-10-11T06:12:06.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda3a8ca640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340ce140 con 0x7fda34156be0 2023-10-11T06:12:06.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda3a8ca640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3415da30 con 0x7fda34156810 2023-10-11T06:12:06.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda38e67640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fda3406c170 0x7fda34160020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44842/0 (socket says 172.21.15.84:44842) 2023-10-11T06:12:06.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda38e67640 1 -- 172.21.15.84:0/2501333223 learned_addr learned my addr 172.21.15.84:0/2501333223 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:06.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda2bfff640 1 --2- 172.21.15.84:0/2501333223 >> v2:172.21.15.99:3300/0 conn(0x7fda34156be0 0x7fda3415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda33fff640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34156810 0x7fda3406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033748457 0 0) 0x7fda340cde80 con 0x7fda3406c170 2023-10-11T06:12:06.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.142+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda20003520 con 0x7fda3406c170 2023-10-11T06:12:06.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161196617 0 0) 0x7fda20003520 con 0x7fda3406c170 2023-10-11T06:12:06.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34156810 msgr2=0x7fda3406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:06.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34156810 0x7fda3406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 >> v2:172.21.15.99:3300/0 conn(0x7fda34156be0 msgr2=0x7fda3415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 --2- 172.21.15.84:0/2501333223 >> v2:172.21.15.99:3300/0 conn(0x7fda34156be0 0x7fda3415d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3406df80 con 0x7fda3406c170 2023-10-11T06:12:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fda2c0019f0 con 0x7fda3406c170 2023-10-11T06:12:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 >> v1:172.21.15.84:6789/0 conn(0x7fda3406c170 legacy=0x7fda34160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34156810 0x7fda20003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda20003ef0 0x7fda200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda20005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340ce140 con 0x7fda20004d00 2023-10-11T06:12:06.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3415da30 con 0x7fda20003ef0 2023-10-11T06:12:06.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2000ba60 con 0x7fda34156810 2023-10-11T06:12:06.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.143+0000 7fda38e67640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda20005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.144+0000 7fda2bfff640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda20003ef0 0x7fda200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.144+0000 7fda38e67640 1 -- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34156810 msgr2=0x7fda20003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:06.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.144+0000 7fda38e67640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34156810 0x7fda20003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.144+0000 7fda38e67640 1 -- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda20003ef0 msgr2=0x7fda200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:06.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.144+0000 7fda38e67640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda20003ef0 0x7fda200047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.144+0000 7fda38e67640 1 -- 172.21.15.84:0/2501333223 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20003520 con 0x7fda20004d00 2023-10-11T06:12:06.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda38e67640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda20005170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00a830 tx=0x7fda200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525cccfe4bbad826 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:06.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c034630 con 0x7fda20004d00 2023-10-11T06:12:06.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c02f070 con 0x7fda20004d00 2023-10-11T06:12:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda337fe640 1 -- 172.21.15.84:0/2501333223 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c034a50 con 0x7fda20004d00 2023-10-11T06:12:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda3a8ca640 1 -- 172.21.15.84:0/2501333223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 msgr2=0x7fda20005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda3a8ca640 1 --2- 172.21.15.84:0/2501333223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda20005170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00a830 tx=0x7fda200057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda38e67640 1 -- 172.21.15.84:0/2501333223 reap_dead start 2023-10-11T06:12:06.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda3a8ca640 1 -- 172.21.15.84:0/2501333223 shutdown_connections 2023-10-11T06:12:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda3a8ca640 1 -- 172.21.15.84:0/2501333223 >> 172.21.15.84:0/2501333223 conn(0x7fda34069c70 msgr2=0x7fda3406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.145+0000 7fda3a8ca640 1 -- 172.21.15.84:0/2501333223 shutdown_connections 2023-10-11T06:12:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.146+0000 7fda3a8ca640 1 -- 172.21.15.84:0/2501333223 wait complete. 2023-10-11T06:12:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.147+0000 7fda3a8ca640 1 Processor -- start 2023-10-11T06:12:06.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.147+0000 7fda3a8ca640 1 -- start start 2023-10-11T06:12:06.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.148+0000 7fda3a8ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda3414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.148+0000 7fda3a8ca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda340ab940 0x7fda34152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.148+0000 7fda3a8ca640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34155060 0x7fda34153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.148+0000 7fda3a8ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340ad070 con 0x7fda20004d00 2023-10-11T06:12:06.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.148+0000 7fda3a8ca640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3415f090 con 0x7fda340ab940 2023-10-11T06:12:06.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.148+0000 7fda3a8ca640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3415f370 con 0x7fda34155060 2023-10-11T06:12:06.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda3414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda3414e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59424/0 (socket says 172.21.15.84:59424) 2023-10-11T06:12:06.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 -- 172.21.15.84:0/3468701346 learned_addr learned my addr 172.21.15.84:0/3468701346 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:06.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 -- 172.21.15.84:0/3468701346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34155060 msgr2=0x7fda34153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda2bfff640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda340ab940 0x7fda34152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34155060 0x7fda34153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 -- 172.21.15.84:0/3468701346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda340ab940 msgr2=0x7fda34152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda340ab940 0x7fda34152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 -- 172.21.15.84:0/3468701346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3415f650 con 0x7fda20004d00 2023-10-11T06:12:06.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda33fff640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda3414e1d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d800 tx=0x7fda2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:06.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.149+0000 7fda317fa640 1 -- 172.21.15.84:0/3468701346 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400b640 con 0x7fda20004d00 2023-10-11T06:12:06.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.150+0000 7fda317fa640 1 -- 172.21.15.84:0/3468701346 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2400b7a0 con 0x7fda20004d00 2023-10-11T06:12:06.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.150+0000 7fda317fa640 1 -- 172.21.15.84:0/3468701346 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400bb30 con 0x7fda20004d00 2023-10-11T06:12:06.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.150+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda340ad310 con 0x7fda20004d00 2023-10-11T06:12:06.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.150+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3415dbf0 con 0x7fda20004d00 2023-10-11T06:12:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.151+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda340cbd30 con 0x7fda20004d00 2023-10-11T06:12:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.152+0000 7fda317fa640 1 -- 172.21.15.84:0/3468701346 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fda2400bc90 con 0x7fda20004d00 2023-10-11T06:12:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.152+0000 7fda317fa640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fda1406a860 0x7fda1406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.153+0000 7fda317fa640 1 -- 172.21.15.84:0/3468701346 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fda240891c0 con 0x7fda20004d00 2023-10-11T06:12:06.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.153+0000 7fda2bfff640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fda1406a860 0x7fda1406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.153+0000 7fda2bfff640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fda1406a860 0x7fda1406cd10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda34154640 tx=0x7fda1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:06.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.157+0000 7fda317fa640 1 -- 172.21.15.84:0/3468701346 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fda24054360 con 0x7fda20004d00 2023-10-11T06:12:06.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.350+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda34073ed0 con 0x7fda20004d00 2023-10-11T06:12:06.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.351+0000 7fda317fa640 1 -- 172.21.15.84:0/3468701346 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda24057980 con 0x7fda20004d00 2023-10-11T06:12:06.353 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:06.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fda1406a860 msgr2=0x7fda1406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fda1406a860 0x7fda1406cd10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda34154640 tx=0x7fda1c008040 comp rx=0 tx=0).stop 2023-10-11T06:12:06.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 msgr2=0x7fda3414e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda3414e1d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d800 tx=0x7fda2400dcc0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 shutdown_connections 2023-10-11T06:12:06.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fda1406a860 0x7fda1406cd10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda34155060 0x7fda34153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda340ab940 0x7fda34152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 --2- 172.21.15.84:0/3468701346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda20004d00 0x7fda3414e1d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 >> 172.21.15.84:0/3468701346 conn(0x7fda34069c70 msgr2=0x7fda340b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:06.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 shutdown_connections 2023-10-11T06:12:06.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.354+0000 7fda3a8ca640 1 -- 172.21.15.84:0/3468701346 wait complete. 2023-10-11T06:12:06.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:06.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.572+0000 7f1f6997b640 1 Processor -- start 2023-10-11T06:12:06.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.572+0000 7f1f6997b640 1 -- start start 2023-10-11T06:12:06.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6997b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1f640cf910 0x7f1f6406b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6997b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1f640b1ee0 0x7f1f6406ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6997b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6415cba0 con 0x7f1f640b1b10 2023-10-11T06:12:06.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6997b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6409c220 con 0x7f1f640b1ee0 2023-10-11T06:12:06.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6997b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640d1360 con 0x7f1f640cf910 2023-10-11T06:12:06.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6917a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1f640b1ee0 0x7f1f6406ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6917a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1f640b1ee0 0x7f1f6406ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59438/0 (socket says 172.21.15.84:59438) 2023-10-11T06:12:06.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.574+0000 7f1f6917a640 1 -- 172.21.15.84:0/773390771 learned_addr learned my addr 172.21.15.84:0/773390771 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:06.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.575+0000 7f1f6917a640 1 -- 172.21.15.84:0/773390771 >> v2:172.21.15.99:3300/0 conn(0x7f1f640cf910 msgr2=0x7f1f6406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:06.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.575+0000 7f1f6917a640 1 --2- 172.21.15.84:0/773390771 >> v2:172.21.15.99:3300/0 conn(0x7f1f640cf910 0x7f1f6406b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.575+0000 7f1f6917a640 1 -- 172.21.15.84:0/773390771 >> v1:172.21.15.99:6789/0 conn(0x7f1f640b1b10 legacy=0x7f1f640cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:06.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.575+0000 7f1f6917a640 1 -- 172.21.15.84:0/773390771 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640d0ac0 con 0x7f1f640b1ee0 2023-10-11T06:12:06.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.576+0000 7f1f6917a640 1 --2- 172.21.15.84:0/773390771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f6406ba60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f6408cbc0 tx=0x7f1f5c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea983041b0b8de8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:06.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.576+0000 7f1f637fe640 1 -- 172.21.15.84:0/773390771 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c02cb20 con 0x7f1f640b1ee0 2023-10-11T06:12:06.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.576+0000 7f1f637fe640 1 -- 172.21.15.84:0/773390771 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c02cc80 con 0x7f1f640b1ee0 2023-10-11T06:12:06.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.576+0000 7f1f637fe640 1 -- 172.21.15.84:0/773390771 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0343f0 con 0x7f1f640b1ee0 2023-10-11T06:12:06.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 -- 172.21.15.84:0/773390771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 msgr2=0x7f1f6406ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 --2- 172.21.15.84:0/773390771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f6406ba60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f6408cbc0 tx=0x7f1f5c02bf80 comp rx=0 tx=0).stop 2023-10-11T06:12:06.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 -- 172.21.15.84:0/773390771 shutdown_connections 2023-10-11T06:12:06.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 --2- 172.21.15.84:0/773390771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f6406ba60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 --2- 172.21.15.84:0/773390771 >> v2:172.21.15.99:3300/0 conn(0x7f1f640cf910 0x7f1f6406b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 -- 172.21.15.84:0/773390771 >> 172.21.15.84:0/773390771 conn(0x7f1f6409bc20 msgr2=0x7f1f640d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:06.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 -- 172.21.15.84:0/773390771 shutdown_connections 2023-10-11T06:12:06.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.577+0000 7f1f6997b640 1 -- 172.21.15.84:0/773390771 wait complete. 2023-10-11T06:12:06.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.579+0000 7f1f6997b640 1 Processor -- start 2023-10-11T06:12:06.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.579+0000 7f1f6997b640 1 -- start start 2023-10-11T06:12:06.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f6997b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f640b1b10 0x7f1f640b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f6997b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f640b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f6997b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f640cf910 0x7f1f640cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f6997b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640b5d30 con 0x7f1f640b1ee0 2023-10-11T06:12:06.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f6997b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640b6010 con 0x7f1f640cf910 2023-10-11T06:12:06.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f6997b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640b62f0 con 0x7f1f640b1b10 2023-10-11T06:12:06.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f640b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f63fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f640b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59444/0 (socket says 172.21.15.84:59444) 2023-10-11T06:12:06.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.580+0000 7f1f63fff640 1 -- 172.21.15.84:0/414270036 learned_addr learned my addr 172.21.15.84:0/414270036 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:06.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f63fff640 1 -- 172.21.15.84:0/414270036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f640b1b10 msgr2=0x7f1f640b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f63fff640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f640b1b10 0x7f1f640b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f63fff640 1 -- 172.21.15.84:0/414270036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f640cf910 msgr2=0x7f1f640cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f63fff640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f640cf910 0x7f1f640cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f63fff640 1 -- 172.21.15.84:0/414270036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009b30 con 0x7f1f640b1ee0 2023-10-11T06:12:06.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f63fff640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f640b8070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b410 tx=0x7f1f5000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:06.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f617fa640 1 -- 172.21.15.84:0/414270036 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50011830 con 0x7f1f640b1ee0 2023-10-11T06:12:06.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f617fa640 1 -- 172.21.15.84:0/414270036 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f500119b0 con 0x7f1f640b1ee0 2023-10-11T06:12:06.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640b65d0 con 0x7f1f640b1ee0 2023-10-11T06:12:06.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f617fa640 1 -- 172.21.15.84:0/414270036 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50011d60 con 0x7f1f640b1ee0 2023-10-11T06:12:06.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.581+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640b4f20 con 0x7f1f640b1ee0 2023-10-11T06:12:06.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.583+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6404e890 con 0x7f1f640b1ee0 2023-10-11T06:12:06.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.583+0000 7f1f617fa640 1 -- 172.21.15.84:0/414270036 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1f5001a400 con 0x7f1f640b1ee0 2023-10-11T06:12:06.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.584+0000 7f1f617fa640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1f4806a830 0x7f1f4806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.584+0000 7f1f617fa640 1 -- 172.21.15.84:0/414270036 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1f50089220 con 0x7f1f640b1ee0 2023-10-11T06:12:06.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.585+0000 7f1f68979640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1f4806a830 0x7f1f4806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.585+0000 7f1f68979640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1f4806a830 0x7f1f4806cce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f58003d10 tx=0x7f1f58007a00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:06.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.593+0000 7f1f617fa640 1 -- 172.21.15.84:0/414270036 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1f50054450 con 0x7f1f640b1ee0 2023-10-11T06:12:06.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.783+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1f6408a660 con 0x7f1f640b1ee0 2023-10-11T06:12:06.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.784+0000 7f1f617fa640 1 -- 172.21.15.84:0/414270036 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1f50057a70 con 0x7f1f640b1ee0 2023-10-11T06:12:06.787 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:06.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1f4806a830 msgr2=0x7f1f4806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1f4806a830 0x7f1f4806cce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f58003d10 tx=0x7f1f58007a00 comp rx=0 tx=0).stop 2023-10-11T06:12:06.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 msgr2=0x7f1f640b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f640b8070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b410 tx=0x7f1f5000b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 shutdown_connections 2023-10-11T06:12:06.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1f4806a830 0x7f1f4806cce0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f640cf910 0x7f1f640cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f640b1ee0 0x7f1f640b8070 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 --2- 172.21.15.84:0/414270036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f640b1b10 0x7f1f640b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 >> 172.21.15.84:0/414270036 conn(0x7f1f6409bc20 msgr2=0x7f1f640af510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:06.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 shutdown_connections 2023-10-11T06:12:06.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.787+0000 7f1f6997b640 1 -- 172.21.15.84:0/414270036 wait complete. 2023-10-11T06:12:06.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:06.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.991+0000 7fb4f2adf640 1 Processor -- start 2023-10-11T06:12:06.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.991+0000 7fb4f2adf640 1 -- start start 2023-10-11T06:12:06.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.992+0000 7fb4f2adf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4ec15f7e0 0x7fb4ec0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.992+0000 7fb4f2adf640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4ec0d31d0 0x7fb4ec0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:06.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.992+0000 7fb4f2adf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4ec0cf220 con 0x7fb4ec0cd6c0 2023-10-11T06:12:06.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.992+0000 7fb4f2adf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4ec0cf500 con 0x7fb4ec15f7e0 2023-10-11T06:12:06.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.992+0000 7fb4f2adf640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4ec0cf7e0 con 0x7fb4ec0d31d0 2023-10-11T06:12:06.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f087b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4ec15f7e0 0x7fb4ec0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f087b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4ec15f7e0 0x7fb4ec0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59460/0 (socket says 172.21.15.84:59460) 2023-10-11T06:12:06.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f087b640 1 -- 172.21.15.84:0/2706297400 learned_addr learned my addr 172.21.15.84:0/2706297400 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:06.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f107c640 1 --2- 172.21.15.84:0/2706297400 >> v2:172.21.15.99:3300/0 conn(0x7fb4ec0d31d0 0x7fb4ec0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:06.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f087b640 1 -- 172.21.15.84:0/2706297400 >> v2:172.21.15.99:3300/0 conn(0x7fb4ec0d31d0 msgr2=0x7fb4ec0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:06.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f087b640 1 --2- 172.21.15.84:0/2706297400 >> v2:172.21.15.99:3300/0 conn(0x7fb4ec0d31d0 0x7fb4ec0d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:06.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f087b640 1 -- 172.21.15.84:0/2706297400 >> v1:172.21.15.84:6789/0 conn(0x7fb4ec0cd6c0 legacy=0x7fb4ec0d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:06.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.993+0000 7fb4f087b640 1 -- 172.21.15.84:0/2706297400 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec0cfac0 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.994+0000 7fb4f087b640 1 --2- 172.21.15.84:0/2706297400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec0cd180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0b2310 tx=0x7fb4e002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d7535dd5eaae65 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.994+0000 7fb4eb7fe640 1 -- 172.21.15.84:0/2706297400 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e002cb10 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.994+0000 7fb4eb7fe640 1 -- 172.21.15.84:0/2706297400 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e002cc70 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.994+0000 7fb4eb7fe640 1 -- 172.21.15.84:0/2706297400 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00343f0 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 -- 172.21.15.84:0/2706297400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 msgr2=0x7fb4ec0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/2706297400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec0cd180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0b2310 tx=0x7fb4e002bf70 comp rx=0 tx=0).stop 2023-10-11T06:12:07.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 -- 172.21.15.84:0/2706297400 shutdown_connections 2023-10-11T06:12:07.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/2706297400 >> v2:172.21.15.99:3300/0 conn(0x7fb4ec0d31d0 0x7fb4ec0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/2706297400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec0cd180 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 -- 172.21.15.84:0/2706297400 >> 172.21.15.84:0/2706297400 conn(0x7fb4ec0713a0 msgr2=0x7fb4ec0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:07.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 -- 172.21.15.84:0/2706297400 shutdown_connections 2023-10-11T06:12:07.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.995+0000 7fb4f2adf640 1 -- 172.21.15.84:0/2706297400 wait complete. 2023-10-11T06:12:07.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.997+0000 7fb4f2adf640 1 Processor -- start 2023-10-11T06:12:07.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.997+0000 7fb4f2adf640 1 -- start start 2023-10-11T06:12:07.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f2adf640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4ec0cd6c0 0x7fb4ec15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f2adf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec0d31d0 0x7fb4ec15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f2adf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec15c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f2adf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1f2e20 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f2adf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec15f0b0 con 0x7fb4ec0d31d0 2023-10-11T06:12:07.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f2adf640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1f2530 con 0x7fb4ec0cd6c0 2023-10-11T06:12:07.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f107c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec15c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f107c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec15c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59464/0 (socket says 172.21.15.84:59464) 2023-10-11T06:12:07.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f107c640 1 -- 172.21.15.84:0/325555078 learned_addr learned my addr 172.21.15.84:0/325555078 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:07.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4ebfff640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec0d31d0 0x7fb4ec15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.998+0000 7fb4f087b640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4ec0cd6c0 0x7fb4ec15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f107c640 1 -- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4ec0cd6c0 msgr2=0x7fb4ec15a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f107c640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4ec0cd6c0 0x7fb4ec15a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f107c640 1 -- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec0d31d0 msgr2=0x7fb4ec15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f107c640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec0d31d0 0x7fb4ec15bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f107c640 1 -- 172.21.15.84:0/325555078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009b30 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f107c640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec15c270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400c820 tx=0x7fb4e400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4e97fa640 1 -- 172.21.15.84:0/325555078 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4004020 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4e97fa640 1 -- 172.21.15.84:0/325555078 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e4004180 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4e97fa640 1 -- 172.21.15.84:0/325555078 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4007830 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec15c8d0 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:06.999+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec1f2ab0 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.000+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ec1f2fd0 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.001+0000 7fb4e97fa640 1 -- 172.21.15.84:0/325555078 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb4e4007990 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.002+0000 7fb4e97fa640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb4d006a830 0x7fb4d006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.002+0000 7fb4e97fa640 1 -- 172.21.15.84:0/325555078 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb4e4088e50 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.002+0000 7fb4f087b640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb4d006a830 0x7fb4d006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.003+0000 7fb4f087b640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb4d006a830 0x7fb4d006cce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0035000 tx=0x7fb4e0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.006+0000 7fb4e97fa640 1 -- 172.21.15.84:0/325555078 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb4ec1f2fd0 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.188+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4ec1f2fd0 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.189+0000 7fb4e97fa640 1 -- 172.21.15.84:0/325555078 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4e401e020 con 0x7fb4ec15f7e0 2023-10-11T06:12:07.191 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:07.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.191+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb4d006a830 msgr2=0x7fb4d006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.191+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb4d006a830 0x7fb4d006cce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0035000 tx=0x7fb4e0030000 comp rx=0 tx=0).stop 2023-10-11T06:12:07.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.191+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 msgr2=0x7fb4ec15c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.191+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec15c270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400c820 tx=0x7fb4e400cce0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.191+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 shutdown_connections 2023-10-11T06:12:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.192+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb4d006a830 0x7fb4d006cce0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.192+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4ec15f7e0 0x7fb4ec15c270 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.192+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4ec0d31d0 0x7fb4ec15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.192+0000 7fb4f2adf640 1 --2- 172.21.15.84:0/325555078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4ec0cd6c0 0x7fb4ec15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.192+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 >> 172.21.15.84:0/325555078 conn(0x7fb4ec0713a0 msgr2=0x7fb4ec0b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.192+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 shutdown_connections 2023-10-11T06:12:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.192+0000 7fb4f2adf640 1 -- 172.21.15.84:0/325555078 wait complete. 2023-10-11T06:12:07.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:07.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.414+0000 7f0f4b15c640 1 Processor -- start 2023-10-11T06:12:07.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.414+0000 7f0f4b15c640 1 -- start start 2023-10-11T06:12:07.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.415+0000 7f0f4b15c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f4415a0b0 0x7f0f440cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.415+0000 7f0f4b15c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f44071380 con 0x7f0f440ced30 2023-10-11T06:12:07.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.415+0000 7f0f4b15c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4406de10 con 0x7f0f440c8d00 2023-10-11T06:12:07.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.415+0000 7f0f4b15c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4406e0f0 con 0x7f0f4415a0b0 2023-10-11T06:12:07.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.416+0000 7f0f48ef8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f4415a0b0 0x7f0f440cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.416+0000 7f0f48ef8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f4415a0b0 0x7f0f440cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59476/0 (socket says 172.21.15.84:59476) 2023-10-11T06:12:07.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.416+0000 7f0f48ef8640 1 -- 172.21.15.84:0/2834231631 learned_addr learned my addr 172.21.15.84:0/2834231631 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:07.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.416+0000 7f0f496f9640 1 --1- 172.21.15.84:0/2834231631 >> v1:172.21.15.84:6789/0 conn(0x7f0f440ced30 0x7f0f4406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44846/0 (socket says 172.21.15.84:44846) 2023-10-11T06:12:07.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.416+0000 7f0f48ef8640 1 -- 172.21.15.84:0/2834231631 >> v1:172.21.15.99:6789/0 conn(0x7f0f440c8d00 legacy=0x7f0f440ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:07.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.416+0000 7f0f48ef8640 1 -- 172.21.15.84:0/2834231631 >> v1:172.21.15.84:6789/0 conn(0x7f0f440ced30 legacy=0x7f0f4406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.416+0000 7f0f48ef8640 1 -- 172.21.15.84:0/2834231631 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44070dc0 con 0x7f0f4415a0b0 2023-10-11T06:12:07.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.417+0000 7f0f48ef8640 1 --2- 172.21.15.84:0/2834231631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f4415a0b0 0x7f0f440cb5a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009870 tx=0x7f0f3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f469c10cc45f419a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.417+0000 7f0f437fe640 1 -- 172.21.15.84:0/2834231631 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380317e0 con 0x7f0f4415a0b0 2023-10-11T06:12:07.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.417+0000 7f0f437fe640 1 -- 172.21.15.84:0/2834231631 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f38031940 con 0x7f0f4415a0b0 2023-10-11T06:12:07.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.417+0000 7f0f437fe640 1 -- 172.21.15.84:0/2834231631 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38031cd0 con 0x7f0f4415a0b0 2023-10-11T06:12:07.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.418+0000 7f0f4b15c640 1 -- 172.21.15.84:0/2834231631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f4415a0b0 msgr2=0x7f0f440cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.418+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/2834231631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f4415a0b0 0x7f0f440cb5a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009870 tx=0x7f0f3802b880 comp rx=0 tx=0).stop 2023-10-11T06:12:07.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.418+0000 7f0f4b15c640 1 -- 172.21.15.84:0/2834231631 shutdown_connections 2023-10-11T06:12:07.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.418+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/2834231631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f4415a0b0 0x7f0f440cb5a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.418+0000 7f0f4b15c640 1 -- 172.21.15.84:0/2834231631 >> 172.21.15.84:0/2834231631 conn(0x7f0f44069c70 msgr2=0x7f0f4406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:07.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.418+0000 7f0f4b15c640 1 -- 172.21.15.84:0/2834231631 shutdown_connections 2023-10-11T06:12:07.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.418+0000 7f0f4b15c640 1 -- 172.21.15.84:0/2834231631 wait complete. 2023-10-11T06:12:07.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.420+0000 7f0f4b15c640 1 Processor -- start 2023-10-11T06:12:07.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.420+0000 7f0f4b15c640 1 -- start start 2023-10-11T06:12:07.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f4b15c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f440c8d00 0x7f0f441e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f4b15c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440ced30 0x7f0f440cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f4b15c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4415a0b0 0x7f0f440cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f4b15c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440ccb60 con 0x7f0f440ced30 2023-10-11T06:12:07.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f4b15c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440ccd90 con 0x7f0f4415a0b0 2023-10-11T06:12:07.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f4b15c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440cd070 con 0x7f0f440c8d00 2023-10-11T06:12:07.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440ced30 0x7f0f440cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440ced30 0x7f0f440cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59492/0 (socket says 172.21.15.84:59492) 2023-10-11T06:12:07.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 -- 172.21.15.84:0/918848713 learned_addr learned my addr 172.21.15.84:0/918848713 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:07.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 -- 172.21.15.84:0/918848713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f440c8d00 msgr2=0x7f0f441e4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f440c8d00 0x7f0f441e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 -- 172.21.15.84:0/918848713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4415a0b0 msgr2=0x7f0f440cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4415a0b0 0x7f0f440cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.421+0000 7f0f43fff640 1 -- 172.21.15.84:0/918848713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f380094d0 con 0x7f0f440ced30 2023-10-11T06:12:07.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.422+0000 7f0f43fff640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440ced30 0x7f0f440cde30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c430 tx=0x7f0f3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.422+0000 7f0f417fa640 1 -- 172.21.15.84:0/918848713 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30011790 con 0x7f0f440ced30 2023-10-11T06:12:07.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.422+0000 7f0f417fa640 1 -- 172.21.15.84:0/918848713 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f300118f0 con 0x7f0f440ced30 2023-10-11T06:12:07.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.422+0000 7f0f417fa640 1 -- 172.21.15.84:0/918848713 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30011ce0 con 0x7f0f440ced30 2023-10-11T06:12:07.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.422+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f440cd350 con 0x7f0f440ced30 2023-10-11T06:12:07.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.422+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f440cd910 con 0x7f0f440ced30 2023-10-11T06:12:07.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.423+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f440cdb30 con 0x7f0f440ced30 2023-10-11T06:12:07.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.424+0000 7f0f417fa640 1 -- 172.21.15.84:0/918848713 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0f3001c460 con 0x7f0f440ced30 2023-10-11T06:12:07.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.425+0000 7f0f417fa640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f2806a830 0x7f0f2806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.425+0000 7f0f417fa640 1 -- 172.21.15.84:0/918848713 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0f30088ff0 con 0x7f0f440ced30 2023-10-11T06:12:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.426+0000 7f0f48ef8640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f2806a830 0x7f0f2806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.426+0000 7f0f48ef8640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f2806a830 0x7f0f2806cce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f38036000 tx=0x7f0f38030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.430+0000 7f0f417fa640 1 -- 172.21.15.84:0/918848713 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0f30054190 con 0x7f0f440ced30 2023-10-11T06:12:07.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.606+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f44073ed0 con 0x7f0f440ced30 2023-10-11T06:12:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.607+0000 7f0f417fa640 1 -- 172.21.15.84:0/918848713 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f300577b0 con 0x7f0f440ced30 2023-10-11T06:12:07.609 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f2806a830 msgr2=0x7f0f2806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f2806a830 0x7f0f2806cce0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f38036000 tx=0x7f0f38030000 comp rx=0 tx=0).stop 2023-10-11T06:12:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440ced30 msgr2=0x7f0f440cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440ced30 0x7f0f440cde30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c430 tx=0x7f0f3000c8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 shutdown_connections 2023-10-11T06:12:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0f2806a830 0x7f0f2806cce0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4415a0b0 0x7f0f440cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f440ced30 0x7f0f440cde30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 --2- 172.21.15.84:0/918848713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f440c8d00 0x7f0f441e4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.609+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 >> 172.21.15.84:0/918848713 conn(0x7f0f44069c70 msgr2=0x7f0f4414f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.610+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 shutdown_connections 2023-10-11T06:12:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.610+0000 7f0f4b15c640 1 -- 172.21.15.84:0/918848713 wait complete. 2023-10-11T06:12:07.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:07.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.828+0000 7f0e573a2640 1 Processor -- start 2023-10-11T06:12:07.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.828+0000 7f0e573a2640 1 -- start start 2023-10-11T06:12:07.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e573a2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e50145aa0 0x7f0e5006ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e573a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0e500c8090 0x7f0e500c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e573a2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e5006d0d0 0x7f0e5006d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e573a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5006aa70 con 0x7f0e500c8090 2023-10-11T06:12:07.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e573a2640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e500c8bc0 con 0x7f0e5006d0d0 2023-10-11T06:12:07.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e573a2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e50145e70 con 0x7f0e50145aa0 2023-10-11T06:12:07.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e55b9f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0e500c8090 0x7f0e500c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e55b9f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0e500c8090 0x7f0e500c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59508/0 (socket says 172.21.15.84:59508) 2023-10-11T06:12:07.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.830+0000 7f0e55b9f640 1 -- 172.21.15.84:0/1235333390 learned_addr learned my addr 172.21.15.84:0/1235333390 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:07.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e563a0640 1 --2- 172.21.15.84:0/1235333390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e50145aa0 0x7f0e5006ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e56ba1640 1 --2- 172.21.15.84:0/1235333390 >> v2:172.21.15.99:3300/0 conn(0x7f0e5006d0d0 0x7f0e5006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e55b9f640 1 -- 172.21.15.84:0/1235333390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e50145aa0 msgr2=0x7f0e5006ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:07.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e55b9f640 1 --2- 172.21.15.84:0/1235333390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e50145aa0 0x7f0e5006ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e55b9f640 1 -- 172.21.15.84:0/1235333390 >> v2:172.21.15.99:3300/0 conn(0x7f0e5006d0d0 msgr2=0x7f0e5006d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:07.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e55b9f640 1 --2- 172.21.15.84:0/1235333390 >> v2:172.21.15.99:3300/0 conn(0x7f0e5006d0d0 0x7f0e5006d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e55b9f640 1 -- 172.21.15.84:0/1235333390 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5006a0d0 con 0x7f0e500c8090 2023-10-11T06:12:07.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e563a0640 1 --2- 172.21.15.84:0/1235333390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e50145aa0 0x7f0e5006ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:07.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.831+0000 7f0e56ba1640 1 --2- 172.21.15.84:0/1235333390 >> v2:172.21.15.99:3300/0 conn(0x7f0e5006d0d0 0x7f0e5006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:07.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.832+0000 7f0e55b9f640 1 --2- 172.21.15.84:0/1235333390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e500c8090 0x7f0e500c84e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e500cec70 tx=0x7f0e4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb882cad6457aef server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.832+0000 7f0e5539e640 1 -- 172.21.15.84:0/1235333390 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40031640 con 0x7f0e500c8090 2023-10-11T06:12:07.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.832+0000 7f0e5539e640 1 -- 172.21.15.84:0/1235333390 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e400317a0 con 0x7f0e500c8090 2023-10-11T06:12:07.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.832+0000 7f0e5539e640 1 -- 172.21.15.84:0/1235333390 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40031b30 con 0x7f0e500c8090 2023-10-11T06:12:07.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 -- 172.21.15.84:0/1235333390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e500c8090 msgr2=0x7f0e500c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1235333390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e500c8090 0x7f0e500c84e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e500cec70 tx=0x7f0e4002b8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 -- 172.21.15.84:0/1235333390 shutdown_connections 2023-10-11T06:12:07.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1235333390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e500c8090 0x7f0e500c84e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1235333390 >> v2:172.21.15.99:3300/0 conn(0x7f0e5006d0d0 0x7f0e5006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1235333390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e50145aa0 0x7f0e5006ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 -- 172.21.15.84:0/1235333390 >> 172.21.15.84:0/1235333390 conn(0x7f0e500cb7e0 msgr2=0x7f0e500cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:07.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 -- 172.21.15.84:0/1235333390 shutdown_connections 2023-10-11T06:12:07.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.833+0000 7f0e573a2640 1 -- 172.21.15.84:0/1235333390 wait complete. 2023-10-11T06:12:07.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.834+0000 7f0e573a2640 1 Processor -- start 2023-10-11T06:12:07.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.835+0000 7f0e573a2640 1 -- start start 2023-10-11T06:12:07.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e573a2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e5006d0d0 0x7f0e5015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e573a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e500c8090 0x7f0e500ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e573a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50145aa0 0x7f0e500ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e573a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500b1d10 con 0x7f0e50145aa0 2023-10-11T06:12:07.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e573a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500b1fd0 con 0x7f0e500c8090 2023-10-11T06:12:07.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e573a2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e500b22b0 con 0x7f0e5006d0d0 2023-10-11T06:12:07.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e56ba1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50145aa0 0x7f0e500ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e56ba1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50145aa0 0x7f0e500ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59510/0 (socket says 172.21.15.84:59510) 2023-10-11T06:12:07.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e56ba1640 1 -- 172.21.15.84:0/1268643419 learned_addr learned my addr 172.21.15.84:0/1268643419 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:07.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e55b9f640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e500c8090 0x7f0e500ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e563a0640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e5006d0d0 0x7f0e5015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e56ba1640 1 -- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e5006d0d0 msgr2=0x7f0e5015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e56ba1640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e5006d0d0 0x7f0e5015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e56ba1640 1 -- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e500c8090 msgr2=0x7f0e500ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:07.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.836+0000 7f0e56ba1640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e500c8090 0x7f0e500ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:07.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e56ba1640 1 -- 172.21.15.84:0/1268643419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e400094d0 con 0x7f0e50145aa0 2023-10-11T06:12:07.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e55b9f640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e500c8090 0x7f0e500ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:07.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e56ba1640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50145aa0 0x7f0e500ca920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800afd0 tx=0x7f0e4800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e46ffd640 1 -- 172.21.15.84:0/1268643419 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48004440 con 0x7f0e50145aa0 2023-10-11T06:12:07.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e46ffd640 1 -- 172.21.15.84:0/1268643419 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48003610 con 0x7f0e50145aa0 2023-10-11T06:12:07.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e46ffd640 1 -- 172.21.15.84:0/1268643419 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480039c0 con 0x7f0e50145aa0 2023-10-11T06:12:07.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e500b2590 con 0x7f0e50145aa0 2023-10-11T06:12:07.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.837+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e500b0160 con 0x7f0e50145aa0 2023-10-11T06:12:07.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.838+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e500b2ab0 con 0x7f0e50145aa0 2023-10-11T06:12:07.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.839+0000 7f0e46ffd640 1 -- 172.21.15.84:0/1268643419 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0e48003b20 con 0x7f0e50145aa0 2023-10-11T06:12:07.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.840+0000 7f0e46ffd640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0e3406a830 0x7f0e3406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:07.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.840+0000 7f0e46ffd640 1 -- 172.21.15.84:0/1268643419 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0e48089140 con 0x7f0e50145aa0 2023-10-11T06:12:07.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.840+0000 7f0e563a0640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0e3406a830 0x7f0e3406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:07.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.841+0000 7f0e563a0640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0e3406a830 0x7f0e3406cce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0075b0 tx=0x7f0e4c007540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:07.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:07.844+0000 7f0e46ffd640 1 -- 172.21.15.84:0/1268643419 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0e500b2ab0 con 0x7f0e50145aa0 2023-10-11T06:12:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.067+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0e500b2ab0 con 0x7f0e50145aa0 2023-10-11T06:12:08.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.068+0000 7f0e46ffd640 1 -- 172.21.15.84:0/1268643419 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0e500b2ab0 con 0x7f0e50145aa0 2023-10-11T06:12:08.070 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:08.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.070+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0e3406a830 msgr2=0x7f0e3406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.070+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0e3406a830 0x7f0e3406cce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0075b0 tx=0x7f0e4c007540 comp rx=0 tx=0).stop 2023-10-11T06:12:08.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.070+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50145aa0 msgr2=0x7f0e500ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.070+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50145aa0 0x7f0e500ca920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800afd0 tx=0x7f0e4800ca20 comp rx=0 tx=0).stop 2023-10-11T06:12:08.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 shutdown_connections 2023-10-11T06:12:08.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0e3406a830 0x7f0e3406cce0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50145aa0 0x7f0e500ca920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e500c8090 0x7f0e500ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 --2- 172.21.15.84:0/1268643419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e5006d0d0 0x7f0e5015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 >> 172.21.15.84:0/1268643419 conn(0x7f0e500cb7e0 msgr2=0x7f0e5015eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:08.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 shutdown_connections 2023-10-11T06:12:08.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.071+0000 7f0e573a2640 1 -- 172.21.15.84:0/1268643419 wait complete. 2023-10-11T06:12:08.089 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:12:08.090 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:08.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:08.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.367+0000 7fd53eab6640 1 Processor -- start 2023-10-11T06:12:08.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.367+0000 7fd53eab6640 1 -- start start 2023-10-11T06:12:08.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.368+0000 7fd53eab6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5380ce0d0 0x7fd5380c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.368+0000 7fd53eab6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5381593f0 con 0x7fd5380cd300 2023-10-11T06:12:08.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.368+0000 7fd53eab6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5381596c0 con 0x7fd5380adb90 2023-10-11T06:12:08.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.368+0000 7fd53eab6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53815a340 con 0x7fd5380ce0d0 2023-10-11T06:12:08.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.368+0000 7fd53d053640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd5380cd300 0x7fd53815cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:44858/0 (socket says 172.21.15.84:44858) 2023-10-11T06:12:08.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.368+0000 7fd53d053640 1 -- 172.21.15.84:0/1810144148 learned_addr learned my addr 172.21.15.84:0/1810144148 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:08.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130635275 0 0) 0x7fd5381593f0 con 0x7fd5380cd300 2023-10-11T06:12:08.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd528003520 con 0x7fd5380cd300 2023-10-11T06:12:08.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd53c852640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5380ce0d0 0x7fd5380c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3327138627 0 0) 0x7fd528003520 con 0x7fd5380cd300 2023-10-11T06:12:08.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5380ce0d0 msgr2=0x7fd5380c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:08.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5380ce0d0 0x7fd5380c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 >> v1:172.21.15.99:6789/0 conn(0x7fd5380adb90 legacy=0x7fd5380caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5381599a0 con 0x7fd5380cd300 2023-10-11T06:12:08.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fd530001670 con 0x7fd5380cd300 2023-10-11T06:12:08.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 >> v1:172.21.15.84:6789/0 conn(0x7fd5380cd300 legacy=0x7fd53815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5280039a0 0x7fd528003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5380ce0d0 0x7fd5280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd528004d00 0x7fd528005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53815a340 con 0x7fd528004d00 2023-10-11T06:12:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52800ba90 con 0x7fd5280039a0 2023-10-11T06:12:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.369+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52800bc50 con 0x7fd5380ce0d0 2023-10-11T06:12:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.370+0000 7fd53d053640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd528004d00 0x7fd528005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.370+0000 7fd53c852640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5280039a0 0x7fd528003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.370+0000 7fd53d053640 1 -- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5380ce0d0 msgr2=0x7fd5280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.370+0000 7fd53d053640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5380ce0d0 0x7fd5280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.370+0000 7fd53d053640 1 -- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5280039a0 msgr2=0x7fd528003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.370+0000 7fd53d053640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5280039a0 0x7fd528003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.370+0000 7fd53d053640 1 -- 172.21.15.84:0/1810144148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd528003520 con 0x7fd528004d00 2023-10-11T06:12:08.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.371+0000 7fd53d053640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd528004d00 0x7fd528005170 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd53000c240 tx=0x7fd5280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a33d7781a405575 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.371+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530005ad0 con 0x7fd528004d00 2023-10-11T06:12:08.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.371+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530005c30 con 0x7fd528004d00 2023-10-11T06:12:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.371+0000 7fd5377fe640 1 -- 172.21.15.84:0/1810144148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53002c8b0 con 0x7fd528004d00 2023-10-11T06:12:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.372+0000 7fd53eab6640 1 -- 172.21.15.84:0/1810144148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd528004d00 msgr2=0x7fd528005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.372+0000 7fd53eab6640 1 --2- 172.21.15.84:0/1810144148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd528004d00 0x7fd528005170 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd53000c240 tx=0x7fd5280057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.372+0000 7fd53d053640 1 -- 172.21.15.84:0/1810144148 reap_dead start 2023-10-11T06:12:08.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.372+0000 7fd53eab6640 1 -- 172.21.15.84:0/1810144148 shutdown_connections 2023-10-11T06:12:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.372+0000 7fd53eab6640 1 -- 172.21.15.84:0/1810144148 >> 172.21.15.84:0/1810144148 conn(0x7fd538069c70 msgr2=0x7fd53806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.372+0000 7fd53eab6640 1 -- 172.21.15.84:0/1810144148 shutdown_connections 2023-10-11T06:12:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.372+0000 7fd53eab6640 1 -- 172.21.15.84:0/1810144148 wait complete. 2023-10-11T06:12:08.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.374+0000 7fd53eab6640 1 Processor -- start 2023-10-11T06:12:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.374+0000 7fd53eab6640 1 -- start start 2023-10-11T06:12:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd53eab6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd528004d00 0x7fd53814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd53eab6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5380ac460 0x7fd53814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd53eab6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5380c8180 0x7fd53814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd53eab6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380b42a0 con 0x7fd5380ac460 2023-10-11T06:12:08.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd53eab6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c56d0 con 0x7fd5380c8180 2023-10-11T06:12:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd53eab6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c5920 con 0x7fd528004d00 2023-10-11T06:12:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd537fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5380ac460 0x7fd53814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd537fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5380ac460 0x7fd53814bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59524/0 (socket says 172.21.15.84:59524) 2023-10-11T06:12:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.375+0000 7fd537fff640 1 -- 172.21.15.84:0/4036746690 learned_addr learned my addr 172.21.15.84:0/4036746690 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:08.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd537fff640 1 -- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd528004d00 msgr2=0x7fd53814d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd53d053640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5380c8180 0x7fd53814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd53c852640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd528004d00 0x7fd53814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd537fff640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd528004d00 0x7fd53814d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd537fff640 1 -- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5380c8180 msgr2=0x7fd53814c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd537fff640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5380c8180 0x7fd53814c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd537fff640 1 -- 172.21.15.84:0/4036746690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380c5c00 con 0x7fd5380ac460 2023-10-11T06:12:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd53d053640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5380c8180 0x7fd53814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd53c852640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd528004d00 0x7fd53814d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd537fff640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5380ac460 0x7fd53814bb60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c450 tx=0x7fd52400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd5357fa640 1 -- 172.21.15.84:0/4036746690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240052d0 con 0x7fd5380ac460 2023-10-11T06:12:08.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd5357fa640 1 -- 172.21.15.84:0/4036746690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524005430 con 0x7fd5380ac460 2023-10-11T06:12:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd5357fa640 1 -- 172.21.15.84:0/4036746690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524012870 con 0x7fd5380ac460 2023-10-11T06:12:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd538147a00 con 0x7fd5380ac460 2023-10-11T06:12:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.376+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53814db80 con 0x7fd5380ac460 2023-10-11T06:12:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.377+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5380c5df0 con 0x7fd5380ac460 2023-10-11T06:12:08.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.378+0000 7fd5357fa640 1 -- 172.21.15.84:0/4036746690 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd5240129d0 con 0x7fd5380ac460 2023-10-11T06:12:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.379+0000 7fd5357fa640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd51c06a860 0x7fd51c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.379+0000 7fd5357fa640 1 -- 172.21.15.84:0/4036746690 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd524088b50 con 0x7fd5380ac460 2023-10-11T06:12:08.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.379+0000 7fd53c852640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd51c06a860 0x7fd51c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.380+0000 7fd53c852640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd51c06a860 0x7fd51c06cd10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5380cc730 tx=0x7fd52c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:08.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.383+0000 7fd5357fa640 1 -- 172.21.15.84:0/4036746690 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd524053cf0 con 0x7fd5380ac460 2023-10-11T06:12:08.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.559+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd538073ed0 con 0x7fd5380ac460 2023-10-11T06:12:08.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.562+0000 7fd5357fa640 1 -- 172.21.15.84:0/4036746690 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fd524057310 con 0x7fd5380ac460 2023-10-11T06:12:08.564 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:08.565 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:08.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd51c06a860 msgr2=0x7fd51c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd51c06a860 0x7fd51c06cd10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5380cc730 tx=0x7fd52c00b040 comp rx=0 tx=0).stop 2023-10-11T06:12:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5380ac460 msgr2=0x7fd53814bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5380ac460 0x7fd53814bb60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c450 tx=0x7fd52400c910 comp rx=0 tx=0).stop 2023-10-11T06:12:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 shutdown_connections 2023-10-11T06:12:08.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd51c06a860 0x7fd51c06cd10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5380c8180 0x7fd53814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5380ac460 0x7fd53814bb60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 --2- 172.21.15.84:0/4036746690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd528004d00 0x7fd53814d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 >> 172.21.15.84:0/4036746690 conn(0x7fd538069c70 msgr2=0x7fd5380c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:08.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 shutdown_connections 2023-10-11T06:12:08.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.564+0000 7fd53eab6640 1 -- 172.21.15.84:0/4036746690 wait complete. 2023-10-11T06:12:08.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:08.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:08.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.783+0000 7f3e13fff640 1 Processor -- start 2023-10-11T06:12:08.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.783+0000 7f3e13fff640 1 -- start start 2023-10-11T06:12:08.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.785+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140adda0 0x7f3e14069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.785+0000 7f3e13fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e1406a1b0 0x7f3e140afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.785+0000 7f3e13fff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e140b0570 con 0x7f3e140b00f0 2023-10-11T06:12:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.785+0000 7f3e13fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140b07c0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.785+0000 7f3e13fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140aedd0 con 0x7f3e140adda0 2023-10-11T06:12:08.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e0a7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e1406a1b0 0x7f3e140afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e0a7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e1406a1b0 0x7f3e140afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59538/0 (socket says 172.21.15.84:59538) 2023-10-11T06:12:08.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e0a7fc640 1 -- 172.21.15.84:0/3507893561 learned_addr learned my addr 172.21.15.84:0/3507893561 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:08.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e12ffd640 1 --2- 172.21.15.84:0/3507893561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140adda0 0x7f3e14069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e0a7fc640 1 -- 172.21.15.84:0/3507893561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140adda0 msgr2=0x7f3e14069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:08.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e0a7fc640 1 --2- 172.21.15.84:0/3507893561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140adda0 0x7f3e14069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e0a7fc640 1 -- 172.21.15.84:0/3507893561 >> v1:172.21.15.99:6789/0 conn(0x7f3e140b00f0 legacy=0x7f3e140cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:08.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.786+0000 7f3e0a7fc640 1 -- 172.21.15.84:0/3507893561 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1406a5e0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.787+0000 7f3e0a7fc640 1 --2- 172.21.15.84:0/3507893561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e140afbb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009870 tx=0x7f3dfc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf42e37e83d58b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.787+0000 7f3e127fc640 1 -- 172.21.15.84:0/3507893561 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0317c0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.787+0000 7f3e127fc640 1 -- 172.21.15.84:0/3507893561 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc031920 con 0x7f3e1406a1b0 2023-10-11T06:12:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.787+0000 7f3e127fc640 1 -- 172.21.15.84:0/3507893561 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc031cb0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 -- 172.21.15.84:0/3507893561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 msgr2=0x7f3e140afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 --2- 172.21.15.84:0/3507893561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e140afbb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009870 tx=0x7f3dfc02b860 comp rx=0 tx=0).stop 2023-10-11T06:12:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 -- 172.21.15.84:0/3507893561 shutdown_connections 2023-10-11T06:12:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 --2- 172.21.15.84:0/3507893561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e140afbb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 --2- 172.21.15.84:0/3507893561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140adda0 0x7f3e14069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 -- 172.21.15.84:0/3507893561 >> 172.21.15.84:0/3507893561 conn(0x7f3e14098a60 msgr2=0x7f3e1406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 -- 172.21.15.84:0/3507893561 shutdown_connections 2023-10-11T06:12:08.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.788+0000 7f3e13fff640 1 -- 172.21.15.84:0/3507893561 wait complete. 2023-10-11T06:12:08.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.790+0000 7f3e13fff640 1 Processor -- start 2023-10-11T06:12:08.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.790+0000 7f3e13fff640 1 -- start start 2023-10-11T06:12:08.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e14155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e140adda0 0x7f3e1415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e13fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140b00f0 0x7f3e140b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e13fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1415a000 con 0x7f3e1406a1b0 2023-10-11T06:12:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e13fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140c3350 con 0x7f3e140adda0 2023-10-11T06:12:08.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e13fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141599b0 con 0x7f3e140b00f0 2023-10-11T06:12:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e14155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e14155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59554/0 (socket says 172.21.15.84:59554) 2023-10-11T06:12:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.791+0000 7f3e12ffd640 1 -- 172.21.15.84:0/1310659575 learned_addr learned my addr 172.21.15.84:0/1310659575 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e12ffd640 1 -- 172.21.15.84:0/1310659575 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140b00f0 msgr2=0x7f3e140b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e12ffd640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140b00f0 0x7f3e140b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e12ffd640 1 -- 172.21.15.84:0/1310659575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e140adda0 msgr2=0x7f3e1415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e12ffd640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e140adda0 0x7f3e1415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e12ffd640 1 -- 172.21.15.84:0/1310659575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0094d0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e12ffd640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e14155cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c7f0 tx=0x7f3e0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:08.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e0bfff640 1 -- 172.21.15.84:0/1310659575 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04004220 con 0x7f3e1406a1b0 2023-10-11T06:12:08.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e0bfff640 1 -- 172.21.15.84:0/1310659575 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e040043a0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e14159c90 con 0x7f3e1406a1b0 2023-10-11T06:12:08.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e0bfff640 1 -- 172.21.15.84:0/1310659575 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04007860 con 0x7f3e1406a1b0 2023-10-11T06:12:08.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.792+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1415a9d0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.793+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e141597f0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.794+0000 7f3e0bfff640 1 -- 172.21.15.84:0/1310659575 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3e04007bd0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.795+0000 7f3e0bfff640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3df406a830 0x7f3df406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.795+0000 7f3e0bfff640 1 -- 172.21.15.84:0/1310659575 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3e04089070 con 0x7f3e1406a1b0 2023-10-11T06:12:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.796+0000 7f3e0a7fc640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3df406a830 0x7f3df406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.796+0000 7f3e0a7fc640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3df406a830 0x7f3df406cce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc036000 tx=0x7f3dfc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:08.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.799+0000 7f3e0bfff640 1 -- 172.21.15.84:0/1310659575 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3e040bbba0 con 0x7f3e1406a1b0 2023-10-11T06:12:08.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.990+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e1404e890 con 0x7f3e1406a1b0 2023-10-11T06:12:08.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.995+0000 7f3e0bfff640 1 -- 172.21.15.84:0/1310659575 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e0401e020 con 0x7f3e1406a1b0 2023-10-11T06:12:08.997 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:09.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3df406a830 msgr2=0x7f3df406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3df406a830 0x7f3df406cce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc036000 tx=0x7f3dfc030000 comp rx=0 tx=0).stop 2023-10-11T06:12:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 msgr2=0x7f3e14155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e14155cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c7f0 tx=0x7f3e0400ccb0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 shutdown_connections 2023-10-11T06:12:09.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3df406a830 0x7f3df406cce0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e140b00f0 0x7f3e140b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e140adda0 0x7f3e1415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 --2- 172.21.15.84:0/1310659575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e1406a1b0 0x7f3e14155cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 >> 172.21.15.84:0/1310659575 conn(0x7f3e14098a60 msgr2=0x7f3e1415b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:09.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 shutdown_connections 2023-10-11T06:12:09.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:08.998+0000 7f3e13fff640 1 -- 172.21.15.84:0/1310659575 wait complete. 2023-10-11T06:12:09.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:09.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.233+0000 7f81751c4640 1 Processor -- start 2023-10-11T06:12:09.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.233+0000 7f81751c4640 1 -- start start 2023-10-11T06:12:09.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.234+0000 7f81751c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f81701591f0 0x7f81700cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.234+0000 7f81751c4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817015cd40 con 0x7f8170158e20 2023-10-11T06:12:09.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.234+0000 7f81751c4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817015cfe0 con 0x7f81700cc890 2023-10-11T06:12:09.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.234+0000 7f81751c4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817015d2c0 con 0x7f81701591f0 2023-10-11T06:12:09.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8170158e20 0x7f81700cae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49804/0 (socket says 172.21.15.84:49804) 2023-10-11T06:12:09.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816ffff640 1 -- 172.21.15.84:0/2802723439 learned_addr learned my addr 172.21.15.84:0/2802723439 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:09.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050560495 0 0) 0x7f817015cd40 con 0x7f8170158e20 2023-10-11T06:12:09.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815c003520 con 0x7f8170158e20 2023-10-11T06:12:09.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f81749c3640 1 --2- 172.21.15.84:0/2802723439 >> v2:172.21.15.99:3300/0 conn(0x7f81701591f0 0x7f81700cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3578959137 0 0) 0x7f815c003520 con 0x7f8170158e20 2023-10-11T06:12:09.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 >> v2:172.21.15.99:3300/0 conn(0x7f81701591f0 msgr2=0x7f81700cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:09.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 --2- 172.21.15.84:0/2802723439 >> v2:172.21.15.99:3300/0 conn(0x7f81701591f0 0x7f81700cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 >> v1:172.21.15.99:6789/0 conn(0x7f81700cc890 legacy=0x7f81700cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:09.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817015d5a0 con 0x7f8170158e20 2023-10-11T06:12:09.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.235+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f8160001670 con 0x7f8170158e20 2023-10-11T06:12:09.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 >> v1:172.21.15.84:6789/0 conn(0x7f8170158e20 legacy=0x7f81700cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816effd640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f815c003970 0x7f815c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816effd640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f815c0047c0 0x7f815c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816effd640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0051e0 0x7f815c00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817015d2c0 con 0x7f815c0047c0 2023-10-11T06:12:09.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c010530 con 0x7f815c0051e0 2023-10-11T06:12:09.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0106f0 con 0x7f815c003970 2023-10-11T06:12:09.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816f7fe640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f815c0047c0 0x7f815c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f81749c3640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0051e0 0x7f815c00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816ffff640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f815c003970 0x7f815c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816f7fe640 1 -- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f815c003970 msgr2=0x7f815c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:09.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816f7fe640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f815c003970 0x7f815c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816f7fe640 1 -- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0051e0 msgr2=0x7f815c00fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:09.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816f7fe640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0051e0 0x7f815c00fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f816f7fe640 1 -- 172.21.15.84:0/2802723439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c003520 con 0x7f815c0047c0 2023-10-11T06:12:09.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.236+0000 7f81749c3640 1 -- 172.21.15.84:0/2802723439 reap_dead start 2023-10-11T06:12:09.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.238+0000 7f816f7fe640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f815c0047c0 0x7f815c004c30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8158003db0 tx=0x7f815c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d64da98db39d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:09.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.238+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158032600 con 0x7f815c0047c0 2023-10-11T06:12:09.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.238+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815802e050 con 0x7f815c0047c0 2023-10-11T06:12:09.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.238+0000 7f816effd640 1 -- 172.21.15.84:0/2802723439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81580329e0 con 0x7f815c0047c0 2023-10-11T06:12:09.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.238+0000 7f81751c4640 1 -- 172.21.15.84:0/2802723439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f815c0047c0 msgr2=0x7f815c004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.238+0000 7f81751c4640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f815c0047c0 0x7f815c004c30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8158003db0 tx=0x7f815c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.239+0000 7f81751c4640 1 -- 172.21.15.84:0/2802723439 shutdown_connections 2023-10-11T06:12:09.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.239+0000 7f81751c4640 1 --2- 172.21.15.84:0/2802723439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f815c0047c0 0x7f815c004c30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.239+0000 7f81751c4640 1 -- 172.21.15.84:0/2802723439 >> 172.21.15.84:0/2802723439 conn(0x7f817006bfb0 msgr2=0x7f817006c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:09.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.239+0000 7f81751c4640 1 -- 172.21.15.84:0/2802723439 shutdown_connections 2023-10-11T06:12:09.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.239+0000 7f81751c4640 1 -- 172.21.15.84:0/2802723439 wait complete. 2023-10-11T06:12:09.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.240+0000 7f81751c4640 1 Processor -- start 2023-10-11T06:12:09.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.241+0000 7f81751c4640 1 -- start start 2023-10-11T06:12:09.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81751c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0047c0 0x7f81701489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81751c4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81701485b0 0x7f8170157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81751c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81701564a0 0x7f8170157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81751c4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170157670 con 0x7f81701564a0 2023-10-11T06:12:09.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81751c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170156a60 con 0x7f815c0047c0 2023-10-11T06:12:09.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81751c4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8170156d40 con 0x7f81701485b0 2023-10-11T06:12:09.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81701564a0 0x7f8170157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81701564a0 0x7f8170157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53028/0 (socket says 172.21.15.84:53028) 2023-10-11T06:12:09.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 -- 172.21.15.84:0/2275218992 learned_addr learned my addr 172.21.15.84:0/2275218992 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:09.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f816ffff640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0047c0 0x7f81701489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f816f7fe640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81701485b0 0x7f8170157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 -- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81701485b0 msgr2=0x7f8170157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81701485b0 0x7f8170157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 -- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0047c0 msgr2=0x7f81701489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0047c0 0x7f81701489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f81749c3640 1 -- 172.21.15.84:0/2275218992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170157020 con 0x7f81701564a0 2023-10-11T06:12:09.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.242+0000 7f816f7fe640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81701485b0 0x7f8170157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:09.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.243+0000 7f81749c3640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81701564a0 0x7f8170157fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f817006ba00 tx=0x7f8168009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:09.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.243+0000 7f816cff9640 1 -- 172.21.15.84:0/2275218992 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168014610 con 0x7f81701564a0 2023-10-11T06:12:09.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.243+0000 7f816cff9640 1 -- 172.21.15.84:0/2275218992 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816800f050 con 0x7f81701564a0 2023-10-11T06:12:09.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.243+0000 7f816cff9640 1 -- 172.21.15.84:0/2275218992 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81680149b0 con 0x7f81701564a0 2023-10-11T06:12:09.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.243+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8170157300 con 0x7f81701564a0 2023-10-11T06:12:09.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.243+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81701ed0f0 con 0x7f81701564a0 2023-10-11T06:12:09.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.244+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817015aa40 con 0x7f81701564a0 2023-10-11T06:12:09.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.245+0000 7f816cff9640 1 -- 172.21.15.84:0/2275218992 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8168014b10 con 0x7f81701564a0 2023-10-11T06:12:09.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.246+0000 7f816cff9640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f815006a830 0x7f815006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.246+0000 7f816cff9640 1 -- 172.21.15.84:0/2275218992 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8168088f00 con 0x7f81701564a0 2023-10-11T06:12:09.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.246+0000 7f816ffff640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f815006a830 0x7f815006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.247+0000 7f816ffff640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f815006a830 0x7f815006cce0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8160001eb0 tx=0x7f8160009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:09.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.250+0000 7f816cff9640 1 -- 172.21.15.84:0/2275218992 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8168054140 con 0x7f81701564a0 2023-10-11T06:12:09.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.440+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f81701577b0 con 0x7f81701564a0 2023-10-11T06:12:09.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.441+0000 7f816cff9640 1 -- 172.21.15.84:0/2275218992 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f81701577b0 con 0x7f81701564a0 2023-10-11T06:12:09.444 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:09.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f815006a830 msgr2=0x7f815006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f815006a830 0x7f815006cce0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8160001eb0 tx=0x7f8160009040 comp rx=0 tx=0).stop 2023-10-11T06:12:09.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81701564a0 msgr2=0x7f8170157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81701564a0 0x7f8170157fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f817006ba00 tx=0x7f8168009980 comp rx=0 tx=0).stop 2023-10-11T06:12:09.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 shutdown_connections 2023-10-11T06:12:09.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f815006a830 0x7f815006cce0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81701564a0 0x7f8170157fe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81701485b0 0x7f8170157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 --2- 172.21.15.84:0/2275218992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f815c0047c0 0x7f81701489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 >> 172.21.15.84:0/2275218992 conn(0x7f817006bfb0 msgr2=0x7f81700adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:09.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 shutdown_connections 2023-10-11T06:12:09.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.444+0000 7f81751c4640 1 -- 172.21.15.84:0/2275218992 wait complete. 2023-10-11T06:12:09.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.680+0000 7ff826841640 1 Processor -- start 2023-10-11T06:12:09.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.680+0000 7ff826841640 1 -- start start 2023-10-11T06:12:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff826841640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff82014ff30 0x7ff8200c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff826841640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200c9b90 con 0x7ff8200ce510 2023-10-11T06:12:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff826841640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82015da10 con 0x7ff820150300 2023-10-11T06:12:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff826841640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200cc990 con 0x7ff82014ff30 2023-10-11T06:12:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff81ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff82014ff30 0x7ff8200c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff81ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff82014ff30 0x7ff8200c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53030/0 (socket says 172.21.15.84:53030) 2023-10-11T06:12:09.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff81ffff640 1 -- 172.21.15.84:0/890079165 learned_addr learned my addr 172.21.15.84:0/890079165 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:09.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.682+0000 7ff81ffff640 1 -- 172.21.15.84:0/890079165 >> v1:172.21.15.99:6789/0 conn(0x7ff820150300 legacy=0x7ff8200cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:09.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.683+0000 7ff81ffff640 1 -- 172.21.15.84:0/890079165 >> v1:172.21.15.84:6789/0 conn(0x7ff8200ce510 legacy=0x7ff8200ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.683+0000 7ff81ffff640 1 -- 172.21.15.84:0/890079165 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82015ce00 con 0x7ff82014ff30 2023-10-11T06:12:09.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.684+0000 7ff81ffff640 1 --2- 172.21.15.84:0/890079165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff82014ff30 0x7ff8200c9650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff810009870 tx=0x7ff81002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5eae895ab296e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:09.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.684+0000 7ff81effd640 1 -- 172.21.15.84:0/890079165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81002f8e0 con 0x7ff82014ff30 2023-10-11T06:12:09.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.684+0000 7ff81effd640 1 -- 172.21.15.84:0/890079165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81002fa40 con 0x7ff82014ff30 2023-10-11T06:12:09.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.684+0000 7ff81effd640 1 -- 172.21.15.84:0/890079165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81002fdd0 con 0x7ff82014ff30 2023-10-11T06:12:09.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.684+0000 7ff826841640 1 -- 172.21.15.84:0/890079165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff82014ff30 msgr2=0x7ff8200c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.684+0000 7ff826841640 1 --2- 172.21.15.84:0/890079165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff82014ff30 0x7ff8200c9650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff810009870 tx=0x7ff81002db20 comp rx=0 tx=0).stop 2023-10-11T06:12:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.685+0000 7ff826841640 1 -- 172.21.15.84:0/890079165 shutdown_connections 2023-10-11T06:12:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.685+0000 7ff826841640 1 --2- 172.21.15.84:0/890079165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff82014ff30 0x7ff8200c9650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.685+0000 7ff826841640 1 -- 172.21.15.84:0/890079165 >> 172.21.15.84:0/890079165 conn(0x7ff820069c70 msgr2=0x7ff82006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.685+0000 7ff826841640 1 -- 172.21.15.84:0/890079165 shutdown_connections 2023-10-11T06:12:09.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.685+0000 7ff826841640 1 -- 172.21.15.84:0/890079165 wait complete. 2023-10-11T06:12:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.686+0000 7ff826841640 1 Processor -- start 2023-10-11T06:12:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.687+0000 7ff826841640 1 -- start start 2023-10-11T06:12:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.687+0000 7ff826841640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff8200ce510 0x7ff8201565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.687+0000 7ff826841640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff82014ff30 0x7ff82015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.687+0000 7ff826841640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820150300 0x7ff82013cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.687+0000 7ff826841640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82015bae0 con 0x7ff820150300 2023-10-11T06:12:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.687+0000 7ff826841640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82013ac00 con 0x7ff82014ff30 2023-10-11T06:12:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.687+0000 7ff826841640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200b45c0 con 0x7ff8200ce510 2023-10-11T06:12:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820150300 0x7ff82013cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820150300 0x7ff82013cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53046/0 (socket says 172.21.15.84:53046) 2023-10-11T06:12:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 -- 172.21.15.84:0/1059366186 learned_addr learned my addr 172.21.15.84:0/1059366186 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:09.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff81ffff640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff8200ce510 0x7ff8201565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff81f7fe640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff82014ff30 0x7ff82015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 -- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff8200ce510 msgr2=0x7ff8201565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff8200ce510 0x7ff8201565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 -- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff82014ff30 msgr2=0x7ff82015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff82014ff30 0x7ff82015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff824dde640 1 -- 172.21.15.84:0/1059366186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100094d0 con 0x7ff820150300 2023-10-11T06:12:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.688+0000 7ff81f7fe640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff82014ff30 0x7ff82015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.689+0000 7ff824dde640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820150300 0x7ff82013cee0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c7a0 tx=0x7ff81800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.689+0000 7ff81cff9640 1 -- 172.21.15.84:0/1059366186 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818003b70 con 0x7ff820150300 2023-10-11T06:12:09.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.689+0000 7ff81cff9640 1 -- 172.21.15.84:0/1059366186 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818003cf0 con 0x7ff820150300 2023-10-11T06:12:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.689+0000 7ff81cff9640 1 -- 172.21.15.84:0/1059366186 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818003ea0 con 0x7ff820150300 2023-10-11T06:12:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.689+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff820156b20 con 0x7ff820150300 2023-10-11T06:12:09.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.689+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82015b420 con 0x7ff820150300 2023-10-11T06:12:09.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.690+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82015bc20 con 0x7ff820150300 2023-10-11T06:12:09.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.691+0000 7ff81cff9640 1 -- 172.21.15.84:0/1059366186 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff81800aa90 con 0x7ff820150300 2023-10-11T06:12:09.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.692+0000 7ff81cff9640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff80c06a830 0x7ff80c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.692+0000 7ff81cff9640 1 -- 172.21.15.84:0/1059366186 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff818088f00 con 0x7ff820150300 2023-10-11T06:12:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.693+0000 7ff81ffff640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff80c06a830 0x7ff80c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:09.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.693+0000 7ff81ffff640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff80c06a830 0x7ff80c06cce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff810038000 tx=0x7ff810008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:09.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.697+0000 7ff81cff9640 1 -- 172.21.15.84:0/1059366186 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff82015bc20 con 0x7ff820150300 2023-10-11T06:12:09.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.892+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff82015bc20 con 0x7ff820150300 2023-10-11T06:12:09.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.893+0000 7ff81cff9640 1 -- 172.21.15.84:0/1059366186 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff82015bc20 con 0x7ff820150300 2023-10-11T06:12:09.896 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:09.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff80c06a830 msgr2=0x7ff80c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff80c06a830 0x7ff80c06cce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff810038000 tx=0x7ff810008030 comp rx=0 tx=0).stop 2023-10-11T06:12:09.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820150300 msgr2=0x7ff82013cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:09.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820150300 0x7ff82013cee0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c7a0 tx=0x7ff81800cc60 comp rx=0 tx=0).stop 2023-10-11T06:12:09.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 shutdown_connections 2023-10-11T06:12:09.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff80c06a830 0x7ff80c06cce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820150300 0x7ff82013cee0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff82014ff30 0x7ff82015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 --2- 172.21.15.84:0/1059366186 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff8200ce510 0x7ff8201565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:09.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.896+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 >> 172.21.15.84:0/1059366186 conn(0x7ff820069c70 msgr2=0x7ff8200b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:09.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.897+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 shutdown_connections 2023-10-11T06:12:09.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:09.897+0000 7ff826841640 1 -- 172.21.15.84:0/1059366186 wait complete. 2023-10-11T06:12:09.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:10.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.131+0000 7fd2d370a640 1 Processor -- start 2023-10-11T06:12:10.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.131+0000 7fd2d370a640 1 -- start start 2023-10-11T06:12:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.133+0000 7fd2d370a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 0x7fd2cc0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.133+0000 7fd2d370a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0c9240 con 0x7fd2cc15a0b0 2023-10-11T06:12:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.133+0000 7fd2d370a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc071380 con 0x7fd2cc0ced30 2023-10-11T06:12:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.133+0000 7fd2d370a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc06e0f0 con 0x7fd2cc0cb5a0 2023-10-11T06:12:10.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.134+0000 7fd2d14a6640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd2cc15a0b0 0x7fd2cc0ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49832/0 (socket says 172.21.15.84:49832) 2023-10-11T06:12:10.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.134+0000 7fd2d14a6640 1 -- 172.21.15.84:0/3762206964 learned_addr learned my addr 172.21.15.84:0/3762206964 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.134+0000 7fd2d1ca7640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 0x7fd2cc0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.134+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105001940 0 0) 0x7fd2c8001110 con 0x7fd2cc15a0b0 2023-10-11T06:12:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.134+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b8003520 con 0x7fd2cc15a0b0 2023-10-11T06:12:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 670652315 0 0) 0x7fd2b8003520 con 0x7fd2cc15a0b0 2023-10-11T06:12:10.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 msgr2=0x7fd2cc0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 0x7fd2cc0c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 >> v1:172.21.15.99:6789/0 conn(0x7fd2cc0ced30 legacy=0x7fd2cc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc06de10 con 0x7fd2cc15a0b0 2023-10-11T06:12:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fd2c8001800 con 0x7fd2cc15a0b0 2023-10-11T06:12:10.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 >> v1:172.21.15.84:6789/0 conn(0x7fd2cc15a0b0 legacy=0x7fd2cc0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2b80039a0 0x7fd2b8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b80047c0 0x7fd2b8004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 0x7fd2b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc06e0f0 con 0x7fd2b80039a0 2023-10-11T06:12:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b8005500 con 0x7fd2b80047c0 2023-10-11T06:12:10.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b8005670 con 0x7fd2cc0cb5a0 2023-10-11T06:12:10.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2b80039a0 0x7fd2b8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2d1ca7640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 0x7fd2b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.135+0000 7fd2d0ca5640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b80047c0 0x7fd2b8004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.136+0000 7fd2d14a6640 1 -- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 msgr2=0x7fd2b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:10.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.136+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2cc0cb5a0 0x7fd2b800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.136+0000 7fd2d14a6640 1 -- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b80047c0 msgr2=0x7fd2b8004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.136+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b80047c0 0x7fd2b8004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.136+0000 7fd2d14a6640 1 -- 172.21.15.84:0/3762206964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b8003520 con 0x7fd2b80039a0 2023-10-11T06:12:10.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.137+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2b80039a0 0x7fd2b8003db0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8007fc0 tx=0x7fd2b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2df9fb1c35f5630 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.137+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8005ad0 con 0x7fd2b80039a0 2023-10-11T06:12:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.137+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8005c50 con 0x7fd2b80039a0 2023-10-11T06:12:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.137+0000 7fd2c3fff640 1 -- 172.21.15.84:0/3762206964 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c802c7e0 con 0x7fd2b80039a0 2023-10-11T06:12:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.138+0000 7fd2d370a640 1 -- 172.21.15.84:0/3762206964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2b80039a0 msgr2=0x7fd2b8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.138+0000 7fd2d370a640 1 --2- 172.21.15.84:0/3762206964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2b80039a0 0x7fd2b8003db0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8007fc0 tx=0x7fd2b80057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.138+0000 7fd2d1ca7640 1 -- 172.21.15.84:0/3762206964 reap_dead start 2023-10-11T06:12:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.138+0000 7fd2d370a640 1 -- 172.21.15.84:0/3762206964 shutdown_connections 2023-10-11T06:12:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.138+0000 7fd2d370a640 1 -- 172.21.15.84:0/3762206964 >> 172.21.15.84:0/3762206964 conn(0x7fd2cc069c70 msgr2=0x7fd2cc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.138+0000 7fd2d370a640 1 -- 172.21.15.84:0/3762206964 shutdown_connections 2023-10-11T06:12:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.138+0000 7fd2d370a640 1 -- 172.21.15.84:0/3762206964 wait complete. 2023-10-11T06:12:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.141+0000 7fd2d370a640 1 Processor -- start 2023-10-11T06:12:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.141+0000 7fd2d370a640 1 -- start start 2023-10-11T06:12:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.142+0000 7fd2d370a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b80039a0 0x7fd2cc13bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.142+0000 7fd2d370a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2cc0c72d0 0x7fd2cc0c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.142+0000 7fd2d370a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2cc06c430 0x7fd2cc06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.142+0000 7fd2d370a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc06cde0 con 0x7fd2cc06c430 2023-10-11T06:12:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.142+0000 7fd2d370a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc13c440 con 0x7fd2cc0c72d0 2023-10-11T06:12:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.142+0000 7fd2d370a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0c7d20 con 0x7fd2b80039a0 2023-10-11T06:12:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2cc06c430 0x7fd2cc06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2cc06c430 0x7fd2cc06c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53060/0 (socket says 172.21.15.84:53060) 2023-10-11T06:12:10.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 -- 172.21.15.84:0/2523715245 learned_addr learned my addr 172.21.15.84:0/2523715245 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:10.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b80039a0 0x7fd2cc13bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d0ca5640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2cc0c72d0 0x7fd2cc0c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 -- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b80039a0 msgr2=0x7fd2cc13bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b80039a0 0x7fd2cc13bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 -- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2cc0c72d0 msgr2=0x7fd2cc0c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2cc0c72d0 0x7fd2cc0c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 -- 172.21.15.84:0/2523715245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0c8000 con 0x7fd2cc06c430 2023-10-11T06:12:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b80039a0 0x7fd2cc13bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d0ca5640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2cc0c72d0 0x7fd2cc0c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.143+0000 7fd2d1ca7640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2cc06c430 0x7fd2cc06c8a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0b01a0 tx=0x7fd2c4011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:10.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.144+0000 7fd2c1ffb640 1 -- 172.21.15.84:0/2523715245 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400e040 con 0x7fd2cc06c430 2023-10-11T06:12:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.144+0000 7fd2c1ffb640 1 -- 172.21.15.84:0/2523715245 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c400f040 con 0x7fd2cc06c430 2023-10-11T06:12:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.144+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc0c82e0 con 0x7fd2cc06c430 2023-10-11T06:12:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.144+0000 7fd2c1ffb640 1 -- 172.21.15.84:0/2523715245 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4015b60 con 0x7fd2cc06c430 2023-10-11T06:12:10.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.144+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc13c010 con 0x7fd2cc06c430 2023-10-11T06:12:10.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.145+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc13c640 con 0x7fd2cc06c430 2023-10-11T06:12:10.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.146+0000 7fd2c1ffb640 1 -- 172.21.15.84:0/2523715245 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd2c4015cc0 con 0x7fd2cc06c430 2023-10-11T06:12:10.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.147+0000 7fd2c1ffb640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2b006a860 0x7fd2b006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.147+0000 7fd2c1ffb640 1 -- 172.21.15.84:0/2523715245 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd2c4088ff0 con 0x7fd2cc06c430 2023-10-11T06:12:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.147+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2b006a860 0x7fd2b006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.148+0000 7fd2d14a6640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2b006a860 0x7fd2b006cd10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8034000 tx=0x7fd2c8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:10.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.151+0000 7fd2c1ffb640 1 -- 172.21.15.84:0/2523715245 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd2cc13c640 con 0x7fd2cc06c430 2023-10-11T06:12:10.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.338+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2cc13c610 con 0x7fd2cc06c430 2023-10-11T06:12:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.338+0000 7fd2c1ffb640 1 -- 172.21.15.84:0/2523715245 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd2c40541b0 con 0x7fd2cc06c430 2023-10-11T06:12:10.341 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2b006a860 msgr2=0x7fd2b006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2b006a860 0x7fd2b006cd10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8034000 tx=0x7fd2c8009030 comp rx=0 tx=0).stop 2023-10-11T06:12:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2cc06c430 msgr2=0x7fd2cc06c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2cc06c430 0x7fd2cc06c8a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0b01a0 tx=0x7fd2c4011d40 comp rx=0 tx=0).stop 2023-10-11T06:12:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 shutdown_connections 2023-10-11T06:12:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2b006a860 0x7fd2b006cd10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2cc06c430 0x7fd2cc06c8a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2cc0c72d0 0x7fd2cc0c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 --2- 172.21.15.84:0/2523715245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b80039a0 0x7fd2cc13bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 >> 172.21.15.84:0/2523715245 conn(0x7fd2cc069c70 msgr2=0x7fd2cc0cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 shutdown_connections 2023-10-11T06:12:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.341+0000 7fd2d370a640 1 -- 172.21.15.84:0/2523715245 wait complete. 2023-10-11T06:12:10.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:10.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.562+0000 7f5596ec7640 1 Processor -- start 2023-10-11T06:12:10.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.562+0000 7f5596ec7640 1 -- start start 2023-10-11T06:12:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5596ec7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f559015d4f0 0x7f55900cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5596ec7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f55900ca300 0x7f55900c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5596ec7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5590071a90 con 0x7f55900c9190 2023-10-11T06:12:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5596ec7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5590072310 con 0x7f559015d4f0 2023-10-11T06:12:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5596ec7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5590070280 con 0x7f55900ca300 2023-10-11T06:12:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5595464640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f55900c9190 0x7f559015dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49846/0 (socket says 172.21.15.84:49846) 2023-10-11T06:12:10.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5595464640 1 -- 172.21.15.84:0/2141182243 learned_addr learned my addr 172.21.15.84:0/2141182243 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:10.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.564+0000 7f5594c63640 1 --2- 172.21.15.84:0/2141182243 >> v2:172.21.15.84:3300/0 conn(0x7f559015d4f0 0x7f55900cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.565+0000 7f558ffff640 1 --2- 172.21.15.84:0/2141182243 >> v2:172.21.15.99:3300/0 conn(0x7f55900ca300 0x7f55900c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.565+0000 7f558f7fe640 1 -- 172.21.15.84:0/2141182243 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121222838 0 0) 0x7f5590071a90 con 0x7f55900c9190 2023-10-11T06:12:10.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.565+0000 7f558f7fe640 1 -- 172.21.15.84:0/2141182243 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5580003520 con 0x7f55900c9190 2023-10-11T06:12:10.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.565+0000 7f5594c63640 1 -- 172.21.15.84:0/2141182243 >> v2:172.21.15.99:3300/0 conn(0x7f55900ca300 msgr2=0x7f55900c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:10.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.565+0000 7f5594c63640 1 --2- 172.21.15.84:0/2141182243 >> v2:172.21.15.99:3300/0 conn(0x7f55900ca300 0x7f55900c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.565+0000 7f5594c63640 1 -- 172.21.15.84:0/2141182243 >> v1:172.21.15.84:6789/0 conn(0x7f55900c9190 legacy=0x7f559015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.565+0000 7f5594c63640 1 -- 172.21.15.84:0/2141182243 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5590071cc0 con 0x7f559015d4f0 2023-10-11T06:12:10.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.566+0000 7f5594c63640 1 --2- 172.21.15.84:0/2141182243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f55900cb540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55900b0980 tx=0x7f558402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692fe444e9a8ff16 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:10.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.566+0000 7f558f7fe640 1 -- 172.21.15.84:0/2141182243 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584031830 con 0x7f559015d4f0 2023-10-11T06:12:10.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.566+0000 7f558f7fe640 1 -- 172.21.15.84:0/2141182243 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584031990 con 0x7f559015d4f0 2023-10-11T06:12:10.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.566+0000 7f558f7fe640 1 -- 172.21.15.84:0/2141182243 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584031d00 con 0x7f559015d4f0 2023-10-11T06:12:10.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 -- 172.21.15.84:0/2141182243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 msgr2=0x7f55900cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 --2- 172.21.15.84:0/2141182243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f55900cb540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55900b0980 tx=0x7f558402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 -- 172.21.15.84:0/2141182243 shutdown_connections 2023-10-11T06:12:10.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 --2- 172.21.15.84:0/2141182243 >> v2:172.21.15.99:3300/0 conn(0x7f55900ca300 0x7f55900c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 --2- 172.21.15.84:0/2141182243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f55900cb540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 -- 172.21.15.84:0/2141182243 >> 172.21.15.84:0/2141182243 conn(0x7f5590069c70 msgr2=0x7f559006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:10.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 -- 172.21.15.84:0/2141182243 shutdown_connections 2023-10-11T06:12:10.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.567+0000 7f5596ec7640 1 -- 172.21.15.84:0/2141182243 wait complete. 2023-10-11T06:12:10.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.569+0000 7f5596ec7640 1 Processor -- start 2023-10-11T06:12:10.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.569+0000 7f5596ec7640 1 -- start start 2023-10-11T06:12:10.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.570+0000 7f5596ec7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900c9190 0x7f5590157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.570+0000 7f5596ec7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55900ca300 0x7f55900af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.570+0000 7f5596ec7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f559015e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.570+0000 7f5596ec7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55900af6d0 con 0x7f559015d4f0 2023-10-11T06:12:10.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5596ec7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55901efc10 con 0x7f55900c9190 2023-10-11T06:12:10.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5596ec7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55901efef0 con 0x7f55900ca300 2023-10-11T06:12:10.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f559015e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f559015e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53088/0 (socket says 172.21.15.84:53088) 2023-10-11T06:12:10.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 -- 172.21.15.84:0/3867189192 learned_addr learned my addr 172.21.15.84:0/3867189192 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:10.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5594c63640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900c9190 0x7f5590157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f558ffff640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55900ca300 0x7f55900af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 -- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55900ca300 msgr2=0x7f55900af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55900ca300 0x7f55900af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 -- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900c9190 msgr2=0x7f5590157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900c9190 0x7f5590157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.571+0000 7f5595464640 1 -- 172.21.15.84:0/3867189192 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55840093a0 con 0x7f559015d4f0 2023-10-11T06:12:10.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.572+0000 7f5595464640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f559015e100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f558800caa0 tx=0x7f558800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.572+0000 7f558d7fa640 1 -- 172.21.15.84:0/3867189192 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558800ac00 con 0x7f559015d4f0 2023-10-11T06:12:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.572+0000 7f558d7fa640 1 -- 172.21.15.84:0/3867189192 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558800ad60 con 0x7f559015d4f0 2023-10-11T06:12:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.572+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55901f01d0 con 0x7f559015d4f0 2023-10-11T06:12:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.572+0000 7f558d7fa640 1 -- 172.21.15.84:0/3867189192 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588012620 con 0x7f559015d4f0 2023-10-11T06:12:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.572+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559015baa0 con 0x7f559015d4f0 2023-10-11T06:12:10.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.573+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5590073ed0 con 0x7f559015d4f0 2023-10-11T06:12:10.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.574+0000 7f558d7fa640 1 -- 172.21.15.84:0/3867189192 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5588007070 con 0x7f559015d4f0 2023-10-11T06:12:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.575+0000 7f558d7fa640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f557406a830 0x7f557406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.575+0000 7f558d7fa640 1 -- 172.21.15.84:0/3867189192 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5588088fe0 con 0x7f559015d4f0 2023-10-11T06:12:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.575+0000 7f5594c63640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f557406a830 0x7f557406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:10.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.576+0000 7f5594c63640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f557406a830 0x7f557406cce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5584036000 tx=0x7f5584008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:10.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.579+0000 7f558d7fa640 1 -- 172.21.15.84:0/3867189192 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f55880541e0 con 0x7f559015d4f0 2023-10-11T06:12:10.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.779+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f55900b09b0 con 0x7f559015d4f0 2023-10-11T06:12:10.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.780+0000 7f558d7fa640 1 -- 172.21.15.84:0/3867189192 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5588017080 con 0x7f559015d4f0 2023-10-11T06:12:10.783 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:10.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f557406a830 msgr2=0x7f557406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f557406a830 0x7f557406cce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5584036000 tx=0x7f5584008030 comp rx=0 tx=0).stop 2023-10-11T06:12:10.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 msgr2=0x7f559015e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:10.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f559015e100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f558800caa0 tx=0x7f558800cf60 comp rx=0 tx=0).stop 2023-10-11T06:12:10.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 shutdown_connections 2023-10-11T06:12:10.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f557406a830 0x7f557406cce0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f559015d4f0 0x7f559015e100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55900ca300 0x7f55900af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 --2- 172.21.15.84:0/3867189192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55900c9190 0x7f5590157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:10.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 >> 172.21.15.84:0/3867189192 conn(0x7f5590069c70 msgr2=0x7f55901584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:10.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 shutdown_connections 2023-10-11T06:12:10.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:10.783+0000 7f5596ec7640 1 -- 172.21.15.84:0/3867189192 wait complete. 2023-10-11T06:12:10.805 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2023-10-11T06:12:10.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:10.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-11T06:12:10.913 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:10.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:12:10.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:10.915 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T06:12:10.976 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:12:10.977 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:11.036 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:12:11.036 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:11.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:11.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.326+0000 7fddbcc5f640 1 Processor -- start 2023-10-11T06:12:11.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.326+0000 7fddbcc5f640 1 -- start start 2023-10-11T06:12:11.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.327+0000 7fddbcc5f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fddb815d4f0 0x7fddb80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.327+0000 7fddbcc5f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddb80ca300 0x7fddb80c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.327+0000 7fddbcc5f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb8071a90 con 0x7fddb80c9190 2023-10-11T06:12:11.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.327+0000 7fddbcc5f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb8072310 con 0x7fddb80ca300 2023-10-11T06:12:11.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.327+0000 7fddbcc5f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb8070280 con 0x7fddb815d4f0 2023-10-11T06:12:11.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.327+0000 7fddaffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddb80ca300 0x7fddb80c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddaffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddb80ca300 0x7fddb80c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53104/0 (socket says 172.21.15.84:53104) 2023-10-11T06:12:11.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddaffff640 1 -- 172.21.15.84:0/1651539457 learned_addr learned my addr 172.21.15.84:0/1651539457 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:11.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddb659c640 1 --2- 172.21.15.84:0/1651539457 >> v2:172.21.15.99:3300/0 conn(0x7fddb815d4f0 0x7fddb80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddaffff640 1 -- 172.21.15.84:0/1651539457 >> v2:172.21.15.99:3300/0 conn(0x7fddb815d4f0 msgr2=0x7fddb80cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:11.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddaffff640 1 --2- 172.21.15.84:0/1651539457 >> v2:172.21.15.99:3300/0 conn(0x7fddb815d4f0 0x7fddb80cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddaffff640 1 -- 172.21.15.84:0/1651539457 >> v1:172.21.15.99:6789/0 conn(0x7fddb80c9190 legacy=0x7fddb815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:11.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddaffff640 1 -- 172.21.15.84:0/1651539457 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8071cc0 con 0x7fddb80ca300 2023-10-11T06:12:11.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.328+0000 7fddb659c640 1 --2- 172.21.15.84:0/1651539457 >> v2:172.21.15.99:3300/0 conn(0x7fddb815d4f0 0x7fddb80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:11.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.329+0000 7fddaffff640 1 --2- 172.21.15.84:0/1651539457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb80ca300 0x7fddb80c8c50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fddb80cf6a0 tx=0x7fdda002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db5b61e32b8b0db server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:11.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.330+0000 7fddb5d9b640 1 -- 172.21.15.84:0/1651539457 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0030870 con 0x7fddb80ca300 2023-10-11T06:12:11.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.330+0000 7fddb5d9b640 1 -- 172.21.15.84:0/1651539457 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda00309d0 con 0x7fddb80ca300 2023-10-11T06:12:11.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.330+0000 7fddb5d9b640 1 -- 172.21.15.84:0/1651539457 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0030d60 con 0x7fddb80ca300 2023-10-11T06:12:11.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.330+0000 7fddbcc5f640 1 -- 172.21.15.84:0/1651539457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb80ca300 msgr2=0x7fddb80c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.330+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/1651539457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb80ca300 0x7fddb80c8c50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fddb80cf6a0 tx=0x7fdda002b9b0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.330+0000 7fddbcc5f640 1 -- 172.21.15.84:0/1651539457 shutdown_connections 2023-10-11T06:12:11.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.330+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/1651539457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb80ca300 0x7fddb80c8c50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.331+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/1651539457 >> v2:172.21.15.99:3300/0 conn(0x7fddb815d4f0 0x7fddb80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.331+0000 7fddbcc5f640 1 -- 172.21.15.84:0/1651539457 >> 172.21.15.84:0/1651539457 conn(0x7fddb8069c70 msgr2=0x7fddb806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:11.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.331+0000 7fddbcc5f640 1 -- 172.21.15.84:0/1651539457 shutdown_connections 2023-10-11T06:12:11.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.331+0000 7fddbcc5f640 1 -- 172.21.15.84:0/1651539457 wait complete. 2023-10-11T06:12:11.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.332+0000 7fddbcc5f640 1 Processor -- start 2023-10-11T06:12:11.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.332+0000 7fddbcc5f640 1 -- start start 2023-10-11T06:12:11.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.333+0000 7fddbcc5f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb80c9190 0x7fddb8157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.333+0000 7fddbcc5f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb80ca300 0x7fddb80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.333+0000 7fddbcc5f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb815d4f0 0x7fddb80c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.333+0000 7fddbcc5f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb81efee0 con 0x7fddb815d4f0 2023-10-11T06:12:11.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.333+0000 7fddbcc5f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815b460 con 0x7fddb80ca300 2023-10-11T06:12:11.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.333+0000 7fddbcc5f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb815b740 con 0x7fddb80c9190 2023-10-11T06:12:11.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb815d4f0 0x7fddb80c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb815d4f0 0x7fddb80c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53116/0 (socket says 172.21.15.84:53116) 2023-10-11T06:12:11.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 -- 172.21.15.84:0/2031134530 learned_addr learned my addr 172.21.15.84:0/2031134530 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:11.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddaffff640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb80ca300 0x7fddb80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb659c640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb80c9190 0x7fddb8157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 -- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb80c9190 msgr2=0x7fddb8157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb80c9190 0x7fddb8157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 -- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb80ca300 msgr2=0x7fddb80c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb80ca300 0x7fddb80c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 -- 172.21.15.84:0/2031134530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda00094d0 con 0x7fddb815d4f0 2023-10-11T06:12:11.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddaffff640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb80ca300 0x7fddb80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:11.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb659c640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb80c9190 0x7fddb8157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:11.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.334+0000 7fddb6d9d640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb815d4f0 0x7fddb80c7de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c6b0 tx=0x7fddb000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:11.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.335+0000 7fddaeffd640 1 -- 172.21.15.84:0/2031134530 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0003b70 con 0x7fddb815d4f0 2023-10-11T06:12:11.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.335+0000 7fddaeffd640 1 -- 172.21.15.84:0/2031134530 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0003cf0 con 0x7fddb815d4f0 2023-10-11T06:12:11.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.335+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb815ba20 con 0x7fddb815d4f0 2023-10-11T06:12:11.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.335+0000 7fddaeffd640 1 -- 172.21.15.84:0/2031134530 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0003ea0 con 0x7fddb815d4f0 2023-10-11T06:12:11.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.335+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb80a82d0 con 0x7fddb815d4f0 2023-10-11T06:12:11.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.336+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb8073ed0 con 0x7fddb815d4f0 2023-10-11T06:12:11.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.337+0000 7fddaeffd640 1 -- 172.21.15.84:0/2031134530 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fddb0005310 con 0x7fddb815d4f0 2023-10-11T06:12:11.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.338+0000 7fddaeffd640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd9806a830 0x7fdd9806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.338+0000 7fddaeffd640 1 -- 172.21.15.84:0/2031134530 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fddb0088dc0 con 0x7fddb815d4f0 2023-10-11T06:12:11.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.338+0000 7fddb659c640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd9806a830 0x7fdd9806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.339+0000 7fddb659c640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd9806a830 0x7fdd9806cce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fddb81f33f0 tx=0x7fdda8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:11.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.343+0000 7fddaeffd640 1 -- 172.21.15.84:0/2031134530 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fddb0053ff0 con 0x7fddb815d4f0 2023-10-11T06:12:11.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.531+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddb80b09b0 con 0x7fddb815d4f0 2023-10-11T06:12:11.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.533+0000 7fddaeffd640 1 -- 172.21.15.84:0/2031134530 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fddb0057610 con 0x7fddb815d4f0 2023-10-11T06:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:11.536 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:11.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.535+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd9806a830 msgr2=0x7fdd9806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.535+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd9806a830 0x7fdd9806cce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fddb81f33f0 tx=0x7fdda8008040 comp rx=0 tx=0).stop 2023-10-11T06:12:11.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.535+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb815d4f0 msgr2=0x7fddb80c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.535+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb815d4f0 0x7fddb80c7de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c6b0 tx=0x7fddb000cb70 comp rx=0 tx=0).stop 2023-10-11T06:12:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 shutdown_connections 2023-10-11T06:12:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd9806a830 0x7fdd9806cce0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddb815d4f0 0x7fddb80c7de0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb80ca300 0x7fddb80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 --2- 172.21.15.84:0/2031134530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb80c9190 0x7fddb8157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 >> 172.21.15.84:0/2031134530 conn(0x7fddb8069c70 msgr2=0x7fddb815e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 shutdown_connections 2023-10-11T06:12:11.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.536+0000 7fddbcc5f640 1 -- 172.21.15.84:0/2031134530 wait complete. 2023-10-11T06:12:11.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:11.558 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:11.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.770+0000 7f83f9a10640 1 Processor -- start 2023-10-11T06:12:11.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.770+0000 7f83f9a10640 1 -- start start 2023-10-11T06:12:11.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f9a10640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f414ff30 0x7f83f40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f9a10640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f83f4150300 0x7f83f40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f9a10640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83f40cc7c0 0x7f83f40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f9a10640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f40c9b90 con 0x7f83f40cc7c0 2023-10-11T06:12:11.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f9a10640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f415da10 con 0x7f83f4150300 2023-10-11T06:12:11.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f9a10640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f415ce00 con 0x7f83f414ff30 2023-10-11T06:12:11.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83f40cc7c0 0x7f83f40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83f40cc7c0 0x7f83f40ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53120/0 (socket says 172.21.15.84:53120) 2023-10-11T06:12:11.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 -- 172.21.15.84:0/1005514587 learned_addr learned my addr 172.21.15.84:0/1005514587 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:11.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f2ffd640 1 --2- 172.21.15.84:0/1005514587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f414ff30 0x7f83f40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83ea7fc640 1 --2- 172.21.15.84:0/1005514587 >> v2:172.21.15.99:3300/0 conn(0x7f83f4150300 0x7f83f40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 -- 172.21.15.84:0/1005514587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f414ff30 msgr2=0x7f83f40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:11.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 --2- 172.21.15.84:0/1005514587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f414ff30 0x7f83f40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 -- 172.21.15.84:0/1005514587 >> v2:172.21.15.99:3300/0 conn(0x7f83f4150300 msgr2=0x7f83f40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:11.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 --2- 172.21.15.84:0/1005514587 >> v2:172.21.15.99:3300/0 conn(0x7f83f4150300 0x7f83f40cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.773+0000 7f83f37fe640 1 -- 172.21.15.84:0/1005514587 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f415d0e0 con 0x7f83f40cc7c0 2023-10-11T06:12:11.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.774+0000 7f83f37fe640 1 --2- 172.21.15.84:0/1005514587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f40cc7c0 0x7f83f40ce510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009870 tx=0x7f83ec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5174ddf25cd6b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:11.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.774+0000 7f83f27fc640 1 -- 172.21.15.84:0/1005514587 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0317c0 con 0x7f83f40cc7c0 2023-10-11T06:12:11.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.774+0000 7f83f27fc640 1 -- 172.21.15.84:0/1005514587 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec031920 con 0x7f83f40cc7c0 2023-10-11T06:12:11.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.774+0000 7f83f27fc640 1 -- 172.21.15.84:0/1005514587 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec031cb0 con 0x7f83f40cc7c0 2023-10-11T06:12:11.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 -- 172.21.15.84:0/1005514587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f40cc7c0 msgr2=0x7f83f40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 --2- 172.21.15.84:0/1005514587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f40cc7c0 0x7f83f40ce510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009870 tx=0x7f83ec02b860 comp rx=0 tx=0).stop 2023-10-11T06:12:11.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 -- 172.21.15.84:0/1005514587 shutdown_connections 2023-10-11T06:12:11.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 --2- 172.21.15.84:0/1005514587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f40cc7c0 0x7f83f40ce510 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 --2- 172.21.15.84:0/1005514587 >> v2:172.21.15.99:3300/0 conn(0x7f83f4150300 0x7f83f40cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 --2- 172.21.15.84:0/1005514587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f414ff30 0x7f83f40c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 -- 172.21.15.84:0/1005514587 >> 172.21.15.84:0/1005514587 conn(0x7f83f4069c70 msgr2=0x7f83f406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:11.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 -- 172.21.15.84:0/1005514587 shutdown_connections 2023-10-11T06:12:11.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.775+0000 7f83f9a10640 1 -- 172.21.15.84:0/1005514587 wait complete. 2023-10-11T06:12:11.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.777+0000 7f83f9a10640 1 Processor -- start 2023-10-11T06:12:11.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.777+0000 7f83f9a10640 1 -- start start 2023-10-11T06:12:11.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f9a10640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83f414ff30 0x7f83f41f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f9a10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f4150300 0x7f83f40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f9a10640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f415b8b0 0x7f83f40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f9a10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40b1e80 con 0x7f83f4150300 2023-10-11T06:12:11.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f9a10640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f415c6c0 con 0x7f83f414ff30 2023-10-11T06:12:11.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f9a10640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f415be20 con 0x7f83f415b8b0 2023-10-11T06:12:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83ea7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f4150300 0x7f83f40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83ea7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f4150300 0x7f83f40b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53130/0 (socket says 172.21.15.84:53130) 2023-10-11T06:12:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83ea7fc640 1 -- 172.21.15.84:0/461976997 learned_addr learned my addr 172.21.15.84:0/461976997 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f37fe640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f415b8b0 0x7f83f40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.778+0000 7f83f2ffd640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83f414ff30 0x7f83f41f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83ea7fc640 1 -- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f415b8b0 msgr2=0x7f83f40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83ea7fc640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f415b8b0 0x7f83f40ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83ea7fc640 1 -- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83f414ff30 msgr2=0x7f83f41f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83ea7fc640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83f414ff30 0x7f83f41f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83ea7fc640 1 -- 172.21.15.84:0/461976997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0094d0 con 0x7f83f4150300 2023-10-11T06:12:11.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83ea7fc640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f4150300 0x7f83f40b3f30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009fd0 tx=0x7f83dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:11.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83f0ff9640 1 -- 172.21.15.84:0/461976997 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc014620 con 0x7f83f4150300 2023-10-11T06:12:11.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83f0ff9640 1 -- 172.21.15.84:0/461976997 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc014780 con 0x7f83f4150300 2023-10-11T06:12:11.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83f0ff9640 1 -- 172.21.15.84:0/461976997 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc014b30 con 0x7f83f4150300 2023-10-11T06:12:11.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f415c100 con 0x7f83f4150300 2023-10-11T06:12:11.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.779+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f40b4470 con 0x7f83f4150300 2023-10-11T06:12:11.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.780+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f406ab90 con 0x7f83f4150300 2023-10-11T06:12:11.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.781+0000 7f83f0ff9640 1 -- 172.21.15.84:0/461976997 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f83dc014c90 con 0x7f83f4150300 2023-10-11T06:12:11.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.782+0000 7f83f0ff9640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f83d406a830 0x7f83d406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:11.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.782+0000 7f83f0ff9640 1 -- 172.21.15.84:0/461976997 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f83dc089a80 con 0x7f83f4150300 2023-10-11T06:12:11.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.782+0000 7f83f2ffd640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f83d406a830 0x7f83d406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:11.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.783+0000 7f83f2ffd640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f83d406a830 0x7f83d406cce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f83e4006fd0 tx=0x7f83e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:11.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.786+0000 7f83f0ff9640 1 -- 172.21.15.84:0/461976997 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f83dc054c20 con 0x7f83f4150300 2023-10-11T06:12:11.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.979+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83f4073ed0 con 0x7f83f4150300 2023-10-11T06:12:11.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.980+0000 7f83f0ff9640 1 -- 172.21.15.84:0/461976997 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83dc058240 con 0x7f83f4150300 2023-10-11T06:12:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:11.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f83d406a830 msgr2=0x7f83d406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f83d406a830 0x7f83d406cce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f83e4006fd0 tx=0x7f83e4008040 comp rx=0 tx=0).stop 2023-10-11T06:12:11.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f4150300 msgr2=0x7f83f40b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:11.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f4150300 0x7f83f40b3f30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009fd0 tx=0x7f83dc00e4b0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 shutdown_connections 2023-10-11T06:12:11.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f83d406a830 0x7f83d406cce0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83f415b8b0 0x7f83f40ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83f4150300 0x7f83f40b3f30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 --2- 172.21.15.84:0/461976997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83f414ff30 0x7f83f41f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:11.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 >> 172.21.15.84:0/461976997 conn(0x7f83f4069c70 msgr2=0x7f83f406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:11.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 shutdown_connections 2023-10-11T06:12:11.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:11.983+0000 7f83f9a10640 1 -- 172.21.15.84:0/461976997 wait complete. 2023-10-11T06:12:12.003 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:12.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.220+0000 7f45041a9640 1 Processor -- start 2023-10-11T06:12:12.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.220+0000 7f45041a9640 1 -- start start 2023-10-11T06:12:12.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f45041a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44fc15a0b0 0x7f44fc0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f45041a9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f44fc0ced30 0x7f44fc06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f45041a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc071380 con 0x7f44fc0c8d00 2023-10-11T06:12:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f45041a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc06e0f0 con 0x7f44fc15a0b0 2023-10-11T06:12:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f45041a9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc0cbae0 con 0x7f44fc0ced30 2023-10-11T06:12:12.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f4501f45640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44fc15a0b0 0x7f44fc0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f4501f45640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44fc15a0b0 0x7f44fc0cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53132/0 (socket says 172.21.15.84:53132) 2023-10-11T06:12:12.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f4501f45640 1 -- 172.21.15.84:0/4143986627 learned_addr learned my addr 172.21.15.84:0/4143986627 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:12.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f4501f45640 1 -- 172.21.15.84:0/4143986627 >> v2:172.21.15.99:3300/0 conn(0x7f44fc0ced30 msgr2=0x7f44fc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:12.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f4501f45640 1 --2- 172.21.15.84:0/4143986627 >> v2:172.21.15.99:3300/0 conn(0x7f44fc0ced30 0x7f44fc06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f4501f45640 1 -- 172.21.15.84:0/4143986627 >> v1:172.21.15.84:6789/0 conn(0x7f44fc0c8d00 legacy=0x7f44fc0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.222+0000 7f4501f45640 1 -- 172.21.15.84:0/4143986627 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc06de10 con 0x7f44fc15a0b0 2023-10-11T06:12:12.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.223+0000 7f4501f45640 1 --2- 172.21.15.84:0/4143986627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc15a0b0 0x7f44fc0cb5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0c8cb0 tx=0x7f44f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a7a90639f5e7aa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:12.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.223+0000 7f4500f43640 1 -- 172.21.15.84:0/4143986627 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80317e0 con 0x7f44fc15a0b0 2023-10-11T06:12:12.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.223+0000 7f4500f43640 1 -- 172.21.15.84:0/4143986627 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8031940 con 0x7f44fc15a0b0 2023-10-11T06:12:12.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.223+0000 7f4500f43640 1 -- 172.21.15.84:0/4143986627 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8031cd0 con 0x7f44fc15a0b0 2023-10-11T06:12:12.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 -- 172.21.15.84:0/4143986627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc15a0b0 msgr2=0x7f44fc0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 --2- 172.21.15.84:0/4143986627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc15a0b0 0x7f44fc0cb5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0c8cb0 tx=0x7f44f802b880 comp rx=0 tx=0).stop 2023-10-11T06:12:12.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 -- 172.21.15.84:0/4143986627 shutdown_connections 2023-10-11T06:12:12.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 --2- 172.21.15.84:0/4143986627 >> v2:172.21.15.99:3300/0 conn(0x7f44fc0ced30 0x7f44fc06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 --2- 172.21.15.84:0/4143986627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc15a0b0 0x7f44fc0cb5a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 -- 172.21.15.84:0/4143986627 >> 172.21.15.84:0/4143986627 conn(0x7f44fc069c70 msgr2=0x7f44fc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:12.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 -- 172.21.15.84:0/4143986627 shutdown_connections 2023-10-11T06:12:12.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.224+0000 7f45041a9640 1 -- 172.21.15.84:0/4143986627 wait complete. 2023-10-11T06:12:12.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.226+0000 7f45041a9640 1 Processor -- start 2023-10-11T06:12:12.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.227+0000 7f45041a9640 1 -- start start 2023-10-11T06:12:12.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f45041a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc0c8d00 0x7f44fc14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f45041a9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44fc0ced30 0x7f44fc0c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f45041a9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44fc15a0b0 0x7f44fc1efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f45041a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc14e930 con 0x7f44fc0c8d00 2023-10-11T06:12:12.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f45041a9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc15c280 con 0x7f44fc0ced30 2023-10-11T06:12:12.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f45041a9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc15c560 con 0x7f44fc15a0b0 2023-10-11T06:12:12.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f4501f45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc0c8d00 0x7f44fc14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f4501f45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc0c8d00 0x7f44fc14e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53142/0 (socket says 172.21.15.84:53142) 2023-10-11T06:12:12.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.228+0000 7f4501f45640 1 -- 172.21.15.84:0/3004286926 learned_addr learned my addr 172.21.15.84:0/3004286926 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:12.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4501744640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44fc0ced30 0x7f44fc0c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4502746640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44fc15a0b0 0x7f44fc1efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4501f45640 1 -- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44fc15a0b0 msgr2=0x7f44fc1efd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4501f45640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44fc15a0b0 0x7f44fc1efd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4501f45640 1 -- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44fc0ced30 msgr2=0x7f44fc0c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4501f45640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44fc0ced30 0x7f44fc0c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4501f45640 1 -- 172.21.15.84:0/3004286926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80094d0 con 0x7f44fc0c8d00 2023-10-11T06:12:12.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4502746640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44fc15a0b0 0x7f44fc1efd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:12.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f4501f45640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc0c8d00 0x7f44fc14e3f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f44f8000c00 tx=0x7f44f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:12.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f44f27fc640 1 -- 172.21.15.84:0/3004286926 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80329c0 con 0x7f44fc0c8d00 2023-10-11T06:12:12.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f44f27fc640 1 -- 172.21.15.84:0/3004286926 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8032b40 con 0x7f44fc0c8d00 2023-10-11T06:12:12.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f44f27fc640 1 -- 172.21.15.84:0/3004286926 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80453f0 con 0x7f44fc0c8d00 2023-10-11T06:12:12.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.229+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc15c840 con 0x7f44fc0c8d00 2023-10-11T06:12:12.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.230+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc15ce00 con 0x7f44fc0c8d00 2023-10-11T06:12:12.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.231+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc073ed0 con 0x7f44fc0c8d00 2023-10-11T06:12:12.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.231+0000 7f44f27fc640 1 -- 172.21.15.84:0/3004286926 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f44f804d3f0 con 0x7f44fc0c8d00 2023-10-11T06:12:12.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.232+0000 7f44f27fc640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f44e006a830 0x7f44e006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.232+0000 7f44f27fc640 1 -- 172.21.15.84:0/3004286926 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f44f80aaf30 con 0x7f44fc0c8d00 2023-10-11T06:12:12.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.233+0000 7f4501744640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f44e006a830 0x7f44e006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.233+0000 7f4501744640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f44e006a830 0x7f44e006cce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0af510 tx=0x7f44ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:12.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.236+0000 7f44f27fc640 1 -- 172.21.15.84:0/3004286926 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f44f8076220 con 0x7f44fc0c8d00 2023-10-11T06:12:12.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.424+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44fc070760 con 0x7f44fc0c8d00 2023-10-11T06:12:12.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.424+0000 7f44f27fc640 1 -- 172.21.15.84:0/3004286926 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f44f8079840 con 0x7f44fc0c8d00 2023-10-11T06:12:12.427 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:12.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f44e006a830 msgr2=0x7f44e006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f44e006a830 0x7f44e006cce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0af510 tx=0x7f44ec009040 comp rx=0 tx=0).stop 2023-10-11T06:12:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc0c8d00 msgr2=0x7f44fc14e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc0c8d00 0x7f44fc14e3f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f44f8000c00 tx=0x7f44f80046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 shutdown_connections 2023-10-11T06:12:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f44e006a830 0x7f44e006cce0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44fc15a0b0 0x7f44fc1efd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44fc0ced30 0x7f44fc0c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 --2- 172.21.15.84:0/3004286926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44fc0c8d00 0x7f44fc14e3f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 >> 172.21.15.84:0/3004286926 conn(0x7f44fc069c70 msgr2=0x7f44fc14ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 shutdown_connections 2023-10-11T06:12:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.427+0000 7f45041a9640 1 -- 172.21.15.84:0/3004286926 wait complete. 2023-10-11T06:12:12.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:12.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.665+0000 7fde62993640 1 Processor -- start 2023-10-11T06:12:12.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.666+0000 7fde62993640 1 -- start start 2023-10-11T06:12:12.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.667+0000 7fde62993640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fde5c0b1b10 0x7fde5c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.667+0000 7fde62993640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0b1ee0 0x7fde5c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.667+0000 7fde62993640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c15cba0 con 0x7fde5c0cecb0 2023-10-11T06:12:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.667+0000 7fde62993640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c09c220 con 0x7fde5c0b1b10 2023-10-11T06:12:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.667+0000 7fde62993640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0d1360 con 0x7fde5c0b1ee0 2023-10-11T06:12:12.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde62192640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fde5c0cecb0 0x7fde5c0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49858/0 (socket says 172.21.15.84:49858) 2023-10-11T06:12:12.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde62192640 1 -- 172.21.15.84:0/1128764390 learned_addr learned my addr 172.21.15.84:0/1128764390 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde61991640 1 --2- 172.21.15.84:0/1128764390 >> v2:172.21.15.99:3300/0 conn(0x7fde5c0b1b10 0x7fde5c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde61190640 1 --2- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0b1ee0 0x7fde5c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde6098f640 1 -- 172.21.15.84:0/1128764390 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480831964 0 0) 0x7fde5c15cba0 con 0x7fde5c0cecb0 2023-10-11T06:12:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde6098f640 1 -- 172.21.15.84:0/1128764390 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde48003520 con 0x7fde5c0cecb0 2023-10-11T06:12:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde61991640 1 -- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0b1ee0 msgr2=0x7fde5c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde61991640 1 --2- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0b1ee0 0x7fde5c0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde61991640 1 -- 172.21.15.84:0/1128764390 >> v1:172.21.15.84:6789/0 conn(0x7fde5c0cecb0 legacy=0x7fde5c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde61991640 1 -- 172.21.15.84:0/1128764390 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0d0ac0 con 0x7fde5c0b1b10 2023-10-11T06:12:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.668+0000 7fde61190640 1 --2- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0b1ee0 0x7fde5c0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:12.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.670+0000 7fde61991640 1 --2- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1b10 0x7fde5c0cf200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde58009790 tx=0x7fde5802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c9ef83e1f18939 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:12.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.670+0000 7fde6098f640 1 -- 172.21.15.84:0/1128764390 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58030800 con 0x7fde5c0b1b10 2023-10-11T06:12:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.670+0000 7fde6098f640 1 -- 172.21.15.84:0/1128764390 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58030960 con 0x7fde5c0b1b10 2023-10-11T06:12:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.670+0000 7fde6098f640 1 -- 172.21.15.84:0/1128764390 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58030cd0 con 0x7fde5c0b1b10 2023-10-11T06:12:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 -- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1b10 msgr2=0x7fde5c0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 --2- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1b10 0x7fde5c0cf200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde58009790 tx=0x7fde5802b940 comp rx=0 tx=0).stop 2023-10-11T06:12:12.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 -- 172.21.15.84:0/1128764390 shutdown_connections 2023-10-11T06:12:12.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 --2- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0b1ee0 0x7fde5c0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 --2- 172.21.15.84:0/1128764390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1b10 0x7fde5c0cf200 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 -- 172.21.15.84:0/1128764390 >> 172.21.15.84:0/1128764390 conn(0x7fde5c09bc20 msgr2=0x7fde5c0d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:12.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 -- 172.21.15.84:0/1128764390 shutdown_connections 2023-10-11T06:12:12.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.671+0000 7fde62993640 1 -- 172.21.15.84:0/1128764390 wait complete. 2023-10-11T06:12:12.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.673+0000 7fde62993640 1 Processor -- start 2023-10-11T06:12:12.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.674+0000 7fde62993640 1 -- start start 2023-10-11T06:12:12.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde62993640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde5c0b1b10 0x7fde5c157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde62993640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1ee0 0x7fde5c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde62993640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0cecb0 0x7fde5c1f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde62993640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15de50 con 0x7fde5c0b1b10 2023-10-11T06:12:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde62993640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15d240 con 0x7fde5c0b1ee0 2023-10-11T06:12:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde62993640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15d520 con 0x7fde5c0cecb0 2023-10-11T06:12:12.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde61991640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde5c0b1b10 0x7fde5c157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde61991640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde5c0b1b10 0x7fde5c157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53150/0 (socket says 172.21.15.84:53150) 2023-10-11T06:12:12.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde61991640 1 -- 172.21.15.84:0/69099419 learned_addr learned my addr 172.21.15.84:0/69099419 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:12.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.675+0000 7fde61190640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1ee0 0x7fde5c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde62192640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0cecb0 0x7fde5c1f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde61991640 1 -- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0cecb0 msgr2=0x7fde5c1f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde61991640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0cecb0 0x7fde5c1f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde61991640 1 -- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1ee0 msgr2=0x7fde5c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde61991640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1ee0 0x7fde5c15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde61991640 1 -- 172.21.15.84:0/69099419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c009400 con 0x7fde5c0b1b10 2023-10-11T06:12:12.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde61190640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1ee0 0x7fde5c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:12.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde61991640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde5c0b1b10 0x7fde5c157140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fde580098b0 tx=0x7fde58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:12.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde527fc640 1 -- 172.21.15.84:0/69099419 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5803e040 con 0x7fde5c0b1b10 2023-10-11T06:12:12.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde527fc640 1 -- 172.21.15.84:0/69099419 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58039c90 con 0x7fde5c0b1b10 2023-10-11T06:12:12.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.676+0000 7fde527fc640 1 -- 172.21.15.84:0/69099419 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58006db0 con 0x7fde5c0b1b10 2023-10-11T06:12:12.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.677+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde580093a0 con 0x7fde5c0b1b10 2023-10-11T06:12:12.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.677+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c15dae0 con 0x7fde5c0b1b10 2023-10-11T06:12:12.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.678+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c04e890 con 0x7fde5c0b1b10 2023-10-11T06:12:12.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.679+0000 7fde527fc640 1 -- 172.21.15.84:0/69099419 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fde58043390 con 0x7fde5c0b1b10 2023-10-11T06:12:12.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.680+0000 7fde527fc640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fde4006a830 0x7fde4006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:12.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.680+0000 7fde527fc640 1 -- 172.21.15.84:0/69099419 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fde580ab2e0 con 0x7fde5c0b1b10 2023-10-11T06:12:12.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.680+0000 7fde61190640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fde4006a830 0x7fde4006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:12.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.681+0000 7fde61190640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fde4006a830 0x7fde4006cce0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde5c15fc00 tx=0x7fde4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:12.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.683+0000 7fde527fc640 1 -- 172.21.15.84:0/69099419 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fde58076480 con 0x7fde5c0b1b10 2023-10-11T06:12:12.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.875+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde5c08a660 con 0x7fde5c0b1b10 2023-10-11T06:12:12.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.876+0000 7fde527fc640 1 -- 172.21.15.84:0/69099419 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde58079aa0 con 0x7fde5c0b1b10 2023-10-11T06:12:12.878 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:12.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fde4006a830 msgr2=0x7fde4006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fde4006a830 0x7fde4006cce0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde5c15fc00 tx=0x7fde4c008040 comp rx=0 tx=0).stop 2023-10-11T06:12:12.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde5c0b1b10 msgr2=0x7fde5c157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:12.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde5c0b1b10 0x7fde5c157140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fde580098b0 tx=0x7fde58004730 comp rx=0 tx=0).stop 2023-10-11T06:12:12.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 shutdown_connections 2023-10-11T06:12:12.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fde4006a830 0x7fde4006cce0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde5c0cecb0 0x7fde5c1f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde5c0b1ee0 0x7fde5c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 --2- 172.21.15.84:0/69099419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde5c0b1b10 0x7fde5c157140 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:12.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 >> 172.21.15.84:0/69099419 conn(0x7fde5c09bc20 msgr2=0x7fde5c1e4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:12.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 shutdown_connections 2023-10-11T06:12:12.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:12.879+0000 7fde62993640 1 -- 172.21.15.84:0/69099419 wait complete. 2023-10-11T06:12:12.900 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:13.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.115+0000 7f85495f5640 1 Processor -- start 2023-10-11T06:12:13.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.115+0000 7f85495f5640 1 -- start start 2023-10-11T06:12:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.116+0000 7f85495f5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f854414ff30 0x7f85440c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f85495f5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85440ce510 0x7f85440ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f85495f5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85440c9b90 con 0x7f8544150300 2023-10-11T06:12:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f85495f5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854415da10 con 0x7f854414ff30 2023-10-11T06:12:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f85495f5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85440cc990 con 0x7f85440ce510 2023-10-11T06:12:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f8542ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f854414ff30 0x7f85440c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f8542ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f854414ff30 0x7f85440c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53154/0 (socket says 172.21.15.84:53154) 2023-10-11T06:12:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f8542ffd640 1 -- 172.21.15.84:0/1970718236 learned_addr learned my addr 172.21.15.84:0/1970718236 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f8542ffd640 1 -- 172.21.15.84:0/1970718236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85440ce510 msgr2=0x7f85440ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f85437fe640 1 --2- 172.21.15.84:0/1970718236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85440ce510 0x7f85440ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.117+0000 7f8542ffd640 1 --2- 172.21.15.84:0/1970718236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85440ce510 0x7f85440ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.118+0000 7f8542ffd640 1 -- 172.21.15.84:0/1970718236 >> v1:172.21.15.99:6789/0 conn(0x7f8544150300 legacy=0x7f85440cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.118+0000 7f8542ffd640 1 -- 172.21.15.84:0/1970718236 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85440ceed0 con 0x7f854414ff30 2023-10-11T06:12:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.118+0000 7f85437fe640 1 --2- 172.21.15.84:0/1970718236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85440ce510 0x7f85440ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.119+0000 7f8542ffd640 1 --2- 172.21.15.84:0/1970718236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f854414ff30 0x7f85440c9650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8534009870 tx=0x7f853402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1170b5c7b90e50 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.119+0000 7f85427fc640 1 -- 172.21.15.84:0/1970718236 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534031640 con 0x7f854414ff30 2023-10-11T06:12:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.119+0000 7f85427fc640 1 -- 172.21.15.84:0/1970718236 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85340317a0 con 0x7f854414ff30 2023-10-11T06:12:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.119+0000 7f85427fc640 1 -- 172.21.15.84:0/1970718236 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534031b30 con 0x7f854414ff30 2023-10-11T06:12:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.119+0000 7f85495f5640 1 -- 172.21.15.84:0/1970718236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f854414ff30 msgr2=0x7f85440c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.120+0000 7f85495f5640 1 --2- 172.21.15.84:0/1970718236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f854414ff30 0x7f85440c9650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8534009870 tx=0x7f853402b8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.120+0000 7f85495f5640 1 -- 172.21.15.84:0/1970718236 shutdown_connections 2023-10-11T06:12:13.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.120+0000 7f85495f5640 1 --2- 172.21.15.84:0/1970718236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85440ce510 0x7f85440ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.120+0000 7f85495f5640 1 --2- 172.21.15.84:0/1970718236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f854414ff30 0x7f85440c9650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.120+0000 7f85495f5640 1 -- 172.21.15.84:0/1970718236 >> 172.21.15.84:0/1970718236 conn(0x7f8544069c70 msgr2=0x7f854406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:13.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.120+0000 7f85495f5640 1 -- 172.21.15.84:0/1970718236 shutdown_connections 2023-10-11T06:12:13.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.120+0000 7f85495f5640 1 -- 172.21.15.84:0/1970718236 wait complete. 2023-10-11T06:12:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.121+0000 7f85495f5640 1 Processor -- start 2023-10-11T06:12:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.122+0000 7f85495f5640 1 -- start start 2023-10-11T06:12:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.122+0000 7f85495f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85440ce510 0x7f8544159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85495f5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f854414ff30 0x7f854415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85495f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544150300 0x7f854406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85495f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854415acf0 con 0x7f8544150300 2023-10-11T06:12:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85495f5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8544159dd0 con 0x7f85440ce510 2023-10-11T06:12:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85495f5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854415a070 con 0x7f854414ff30 2023-10-11T06:12:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544150300 0x7f854406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544150300 0x7f854406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53166/0 (socket says 172.21.15.84:53166) 2023-10-11T06:12:13.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 -- 172.21.15.84:0/717963272 learned_addr learned my addr 172.21.15.84:0/717963272 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:13.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 -- 172.21.15.84:0/717963272 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f854414ff30 msgr2=0x7f854415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f854414ff30 0x7f854415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 -- 172.21.15.84:0/717963272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85440ce510 msgr2=0x7f8544159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f8542ffd640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85440ce510 0x7f8544159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85440ce510 0x7f8544159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.123+0000 7f85437fe640 1 -- 172.21.15.84:0/717963272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340094d0 con 0x7f8544150300 2023-10-11T06:12:13.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.124+0000 7f85437fe640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544150300 0x7f854406c7c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c7f0 tx=0x7f853c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:13.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.124+0000 7f853bfff640 1 -- 172.21.15.84:0/717963272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c007820 con 0x7f8544150300 2023-10-11T06:12:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.124+0000 7f853bfff640 1 -- 172.21.15.84:0/717963272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c007980 con 0x7f8544150300 2023-10-11T06:12:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.124+0000 7f853bfff640 1 -- 172.21.15.84:0/717963272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c007d30 con 0x7f8544150300 2023-10-11T06:12:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.124+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854415a350 con 0x7f8544150300 2023-10-11T06:12:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.124+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854415a910 con 0x7f8544150300 2023-10-11T06:12:13.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.125+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854415ae30 con 0x7f8544150300 2023-10-11T06:12:13.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.126+0000 7f853bfff640 1 -- 172.21.15.84:0/717963272 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f853c0113f0 con 0x7f8544150300 2023-10-11T06:12:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.128+0000 7f853bfff640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f852406a830 0x7f852406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.128+0000 7f853bfff640 1 -- 172.21.15.84:0/717963272 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f853c0893c0 con 0x7f8544150300 2023-10-11T06:12:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.128+0000 7f8542ffd640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f852406a830 0x7f852406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.129+0000 7f8542ffd640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f852406a830 0x7f852406cce0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8534036000 tx=0x7f853403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:13.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.132+0000 7f853bfff640 1 -- 172.21.15.84:0/717963272 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f854415ae30 con 0x7f8544150300 2023-10-11T06:12:13.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.319+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f854415ae30 con 0x7f8544150300 2023-10-11T06:12:13.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.320+0000 7f853bfff640 1 -- 172.21.15.84:0/717963272 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f854415ae30 con 0x7f8544150300 2023-10-11T06:12:13.322 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:13.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.323+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f852406a830 msgr2=0x7f852406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.323+0000 7f85495f5640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f852406a830 0x7f852406cce0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8534036000 tx=0x7f853403c040 comp rx=0 tx=0).stop 2023-10-11T06:12:13.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.323+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544150300 msgr2=0x7f854406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.323+0000 7f85495f5640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544150300 0x7f854406c7c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c7f0 tx=0x7f853c00ccb0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 shutdown_connections 2023-10-11T06:12:13.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f852406a830 0x7f852406cce0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8544150300 0x7f854406c7c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f854414ff30 0x7f854415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 --2- 172.21.15.84:0/717963272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85440ce510 0x7f8544159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 >> 172.21.15.84:0/717963272 conn(0x7f8544069c70 msgr2=0x7f854414d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:13.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 shutdown_connections 2023-10-11T06:12:13.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.324+0000 7f85495f5640 1 -- 172.21.15.84:0/717963272 wait complete. 2023-10-11T06:12:13.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:13.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.559+0000 7f343f684640 1 Processor -- start 2023-10-11T06:12:13.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.559+0000 7f343f684640 1 -- start start 2023-10-11T06:12:13.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343f684640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f343815d4f0 0x7f34380cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343f684640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34380ca300 0x7f34380c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343f684640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3438071a90 con 0x7f34380c9190 2023-10-11T06:12:13.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343f684640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3438072310 con 0x7f343815d4f0 2023-10-11T06:12:13.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343f684640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3438070280 con 0x7f34380ca300 2023-10-11T06:12:13.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343d420640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f343815d4f0 0x7f34380cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343d420640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f343815d4f0 0x7f34380cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53182/0 (socket says 172.21.15.84:53182) 2023-10-11T06:12:13.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.561+0000 7f343d420640 1 -- 172.21.15.84:0/3706555614 learned_addr learned my addr 172.21.15.84:0/3706555614 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:13.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.562+0000 7f343cc1f640 1 --2- 172.21.15.84:0/3706555614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34380ca300 0x7f34380c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.562+0000 7f343d420640 1 -- 172.21.15.84:0/3706555614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34380ca300 msgr2=0x7f34380c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:13.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.562+0000 7f343d420640 1 --2- 172.21.15.84:0/3706555614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34380ca300 0x7f34380c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.562+0000 7f343d420640 1 -- 172.21.15.84:0/3706555614 >> v1:172.21.15.99:6789/0 conn(0x7f34380c9190 legacy=0x7f343815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:13.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.562+0000 7f343d420640 1 -- 172.21.15.84:0/3706555614 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438071cc0 con 0x7f343815d4f0 2023-10-11T06:12:13.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.563+0000 7f343d420640 1 --2- 172.21.15.84:0/3706555614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f343815d4f0 0x7f34380cb540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3434009850 tx=0x7f343402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e42f6c983da6c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:13.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.564+0000 7f342ffff640 1 -- 172.21.15.84:0/3706555614 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434004230 con 0x7f343815d4f0 2023-10-11T06:12:13.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.564+0000 7f342ffff640 1 -- 172.21.15.84:0/3706555614 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434004390 con 0x7f343815d4f0 2023-10-11T06:12:13.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.564+0000 7f342ffff640 1 -- 172.21.15.84:0/3706555614 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434031830 con 0x7f343815d4f0 2023-10-11T06:12:13.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 -- 172.21.15.84:0/3706555614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f343815d4f0 msgr2=0x7f34380cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 --2- 172.21.15.84:0/3706555614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f343815d4f0 0x7f34380cb540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3434009850 tx=0x7f343402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 -- 172.21.15.84:0/3706555614 shutdown_connections 2023-10-11T06:12:13.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 --2- 172.21.15.84:0/3706555614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34380ca300 0x7f34380c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 --2- 172.21.15.84:0/3706555614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f343815d4f0 0x7f34380cb540 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 -- 172.21.15.84:0/3706555614 >> 172.21.15.84:0/3706555614 conn(0x7f3438069c70 msgr2=0x7f343806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:13.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 -- 172.21.15.84:0/3706555614 shutdown_connections 2023-10-11T06:12:13.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.565+0000 7f343f684640 1 -- 172.21.15.84:0/3706555614 wait complete. 2023-10-11T06:12:13.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.566+0000 7f343f684640 1 Processor -- start 2023-10-11T06:12:13.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.567+0000 7f343f684640 1 -- start start 2023-10-11T06:12:13.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.567+0000 7f343f684640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34380c9190 0x7f34380c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343f684640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34380ca300 0x7f34381554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343f684640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f343815d4f0 0x7f3438153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343f684640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34380c5e40 con 0x7f34380ca300 2023-10-11T06:12:13.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343f684640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438155a20 con 0x7f34380c9190 2023-10-11T06:12:13.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343f684640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3438153b50 con 0x7f343815d4f0 2023-10-11T06:12:13.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34380ca300 0x7f34381554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34380ca300 0x7f34381554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53194/0 (socket says 172.21.15.84:53194) 2023-10-11T06:12:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 -- 172.21.15.84:0/1992559726 learned_addr learned my addr 172.21.15.84:0/1992559726 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343d420640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34380c9190 0x7f34380c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343dc21640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f343815d4f0 0x7f3438153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 -- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f343815d4f0 msgr2=0x7f3438153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f343815d4f0 0x7f3438153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 -- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34380c9190 msgr2=0x7f34380c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34380c9190 0x7f34380c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343cc1f640 1 -- 172.21.15.84:0/1992559726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009460 con 0x7f34380ca300 2023-10-11T06:12:13.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.568+0000 7f343d420640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34380c9190 0x7f34380c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.569+0000 7f343cc1f640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34380ca300 0x7f34381554e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f342800d5f0 tx=0x7f342800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.569+0000 7f342dffb640 1 -- 172.21.15.84:0/1992559726 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34280125f0 con 0x7f34380ca300 2023-10-11T06:12:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.569+0000 7f342dffb640 1 -- 172.21.15.84:0/1992559726 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3428012750 con 0x7f34380ca300 2023-10-11T06:12:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.569+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34381564f0 con 0x7f34380ca300 2023-10-11T06:12:13.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.569+0000 7f342dffb640 1 -- 172.21.15.84:0/1992559726 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428012b00 con 0x7f34380ca300 2023-10-11T06:12:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.569+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3438155f30 con 0x7f34380ca300 2023-10-11T06:12:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.570+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3438073ed0 con 0x7f34380ca300 2023-10-11T06:12:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.571+0000 7f342dffb640 1 -- 172.21.15.84:0/1992559726 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3428012c60 con 0x7f34380ca300 2023-10-11T06:12:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.572+0000 7f342dffb640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f341c06a830 0x7f341c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.572+0000 7f342dffb640 1 -- 172.21.15.84:0/1992559726 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f34280894b0 con 0x7f34380ca300 2023-10-11T06:12:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.572+0000 7f343d420640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f341c06a830 0x7f341c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:13.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.573+0000 7f343d420640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f341c06a830 0x7f341c06cce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3434035000 tx=0x7f3434008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:13.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.576+0000 7f342dffb640 1 -- 172.21.15.84:0/1992559726 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f34280546e0 con 0x7f34380ca300 2023-10-11T06:12:13.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.776+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f34380b09b0 con 0x7f34380ca300 2023-10-11T06:12:13.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.776+0000 7f342dffb640 1 -- 172.21.15.84:0/1992559726 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3428057d00 con 0x7f34380ca300 2023-10-11T06:12:13.778 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f341c06a830 msgr2=0x7f341c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f341c06a830 0x7f341c06cce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3434035000 tx=0x7f3434008030 comp rx=0 tx=0).stop 2023-10-11T06:12:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34380ca300 msgr2=0x7f34381554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34380ca300 0x7f34381554e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f342800d5f0 tx=0x7f342800dab0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 shutdown_connections 2023-10-11T06:12:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f341c06a830 0x7f341c06cce0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f343815d4f0 0x7f3438153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34380ca300 0x7f34381554e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 --2- 172.21.15.84:0/1992559726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34380c9190 0x7f34380c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 >> 172.21.15.84:0/1992559726 conn(0x7f3438069c70 msgr2=0x7f34380ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 shutdown_connections 2023-10-11T06:12:13.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:13.779+0000 7f343f684640 1 -- 172.21.15.84:0/1992559726 wait complete. 2023-10-11T06:12:13.798 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:12:13.798 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:13.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:14.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.061+0000 7fc9eb4b6640 1 Processor -- start 2023-10-11T06:12:14.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.061+0000 7fc9eb4b6640 1 -- start start 2023-10-11T06:12:14.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.062+0000 7fc9eb4b6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9e40ca300 0x7fc9e40cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.062+0000 7fc9eb4b6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e40c9360 0x7fc9e40ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.062+0000 7fc9eb4b6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e4071a90 con 0x7fc9e415d4f0 2023-10-11T06:12:14.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.062+0000 7fc9eb4b6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e4072310 con 0x7fc9e40ca300 2023-10-11T06:12:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.062+0000 7fc9eb4b6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4070280 con 0x7fc9e40c9360 2023-10-11T06:12:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e8a51640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9e40ca300 0x7fc9e40cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e8a51640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9e40ca300 0x7fc9e40cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53208/0 (socket says 172.21.15.84:53208) 2023-10-11T06:12:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1560506478 learned_addr learned my addr 172.21.15.84:0/1560506478 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e9a53640 1 --2- 172.21.15.84:0/1560506478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e40c9360 0x7fc9e40ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1560506478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e40c9360 msgr2=0x7fc9e40ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:14.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e8a51640 1 --2- 172.21.15.84:0/1560506478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e40c9360 0x7fc9e40ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1560506478 >> v1:172.21.15.99:6789/0 conn(0x7fc9e415d4f0 legacy=0x7fc9e40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:14.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.063+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1560506478 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4071cc0 con 0x7fc9e40ca300 2023-10-11T06:12:14.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.065+0000 7fc9e8a51640 1 --2- 172.21.15.84:0/1560506478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e40cb540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40b0980 tx=0x7fc9d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c888352bad327c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.065+0000 7fc9dbfff640 1 -- 172.21.15.84:0/1560506478 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4031830 con 0x7fc9e40ca300 2023-10-11T06:12:14.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.065+0000 7fc9dbfff640 1 -- 172.21.15.84:0/1560506478 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4031990 con 0x7fc9e40ca300 2023-10-11T06:12:14.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.065+0000 7fc9dbfff640 1 -- 172.21.15.84:0/1560506478 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4031d20 con 0x7fc9e40ca300 2023-10-11T06:12:14.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1560506478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 msgr2=0x7fc9e40cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1560506478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e40cb540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40b0980 tx=0x7fc9d402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1560506478 shutdown_connections 2023-10-11T06:12:14.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1560506478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e40c9360 0x7fc9e40ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1560506478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e40cb540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1560506478 >> 172.21.15.84:0/1560506478 conn(0x7fc9e4069c70 msgr2=0x7fc9e406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:14.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1560506478 shutdown_connections 2023-10-11T06:12:14.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.066+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1560506478 wait complete. 2023-10-11T06:12:14.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.068+0000 7fc9eb4b6640 1 Processor -- start 2023-10-11T06:12:14.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.068+0000 7fc9eb4b6640 1 -- start start 2023-10-11T06:12:14.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9eb4b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e40c9360 0x7fc9e415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9eb4b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e4159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9eb4b6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e415d4f0 0x7fc9e4159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9eb4b6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40cdc00 con 0x7fc9e40ca300 2023-10-11T06:12:14.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9eb4b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415a370 con 0x7fc9e40c9360 2023-10-11T06:12:14.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9eb4b6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415a650 con 0x7fc9e415d4f0 2023-10-11T06:12:14.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9e8a51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e4159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9e8a51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e4159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53212/0 (socket says 172.21.15.84:53212) 2023-10-11T06:12:14.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1685705916 learned_addr learned my addr 172.21.15.84:0/1685705916 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:14.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9e9252640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e40c9360 0x7fc9e415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9e9a53640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e415d4f0 0x7fc9e4159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.069+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e415d4f0 msgr2=0x7fc9e4159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9e8a51640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e415d4f0 0x7fc9e4159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e40c9360 msgr2=0x7fc9e415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9e8a51640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e40c9360 0x7fc9e415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9e8a51640 1 -- 172.21.15.84:0/1685705916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d40094d0 con 0x7fc9e40ca300 2023-10-11T06:12:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9e9252640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e40c9360 0x7fc9e415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9e8a51640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e4159840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4000c00 tx=0x7fc9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9d9ffb640 1 -- 172.21.15.84:0/1685705916 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40317d0 con 0x7fc9e40ca300 2023-10-11T06:12:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9d9ffb640 1 -- 172.21.15.84:0/1685705916 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d403aa60 con 0x7fc9e40ca300 2023-10-11T06:12:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9d9ffb640 1 -- 172.21.15.84:0/1685705916 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d403ae10 con 0x7fc9e40ca300 2023-10-11T06:12:14.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e415a930 con 0x7fc9e40ca300 2023-10-11T06:12:14.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.070+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e415aef0 con 0x7fc9e40ca300 2023-10-11T06:12:14.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.071+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e415b0e0 con 0x7fc9e40ca300 2023-10-11T06:12:14.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.072+0000 7fc9d9ffb640 1 -- 172.21.15.84:0/1685705916 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc9d404d450 con 0x7fc9e40ca300 2023-10-11T06:12:14.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.073+0000 7fc9d9ffb640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc9c806a830 0x7fc9c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.073+0000 7fc9d9ffb640 1 -- 172.21.15.84:0/1685705916 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc9d40ab100 con 0x7fc9e40ca300 2023-10-11T06:12:14.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.073+0000 7fc9e9252640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc9c806a830 0x7fc9c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.074+0000 7fc9e9252640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc9c806a830 0x7fc9c806cce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40cd200 tx=0x7fc9e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.077+0000 7fc9d9ffb640 1 -- 172.21.15.84:0/1685705916 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc9d4076330 con 0x7fc9e40ca300 2023-10-11T06:12:14.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.267+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9e4073ed0 con 0x7fc9e40ca300 2023-10-11T06:12:14.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.268+0000 7fc9d9ffb640 1 -- 172.21.15.84:0/1685705916 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fc9d4079950 con 0x7fc9e40ca300 2023-10-11T06:12:14.271 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:14.271 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:14.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc9c806a830 msgr2=0x7fc9c806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc9c806a830 0x7fc9c806cce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40cd200 tx=0x7fc9e0008040 comp rx=0 tx=0).stop 2023-10-11T06:12:14.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 msgr2=0x7fc9e4159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e4159840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4000c00 tx=0x7fc9d40046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 shutdown_connections 2023-10-11T06:12:14.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc9c806a830 0x7fc9c806cce0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9e415d4f0 0x7fc9e4159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9e40ca300 0x7fc9e4159840 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 --2- 172.21.15.84:0/1685705916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e40c9360 0x7fc9e415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.271+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 >> 172.21.15.84:0/1685705916 conn(0x7fc9e4069c70 msgr2=0x7fc9e40c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:14.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.272+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 shutdown_connections 2023-10-11T06:12:14.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.272+0000 7fc9eb4b6640 1 -- 172.21.15.84:0/1685705916 wait complete. 2023-10-11T06:12:14.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:14.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.509+0000 7fa9e0bbb640 1 Processor -- start 2023-10-11T06:12:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.509+0000 7fa9e0bbb640 1 -- start start 2023-10-11T06:12:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.510+0000 7fa9e0bbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc06c880 0x7fa9dc06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.510+0000 7fa9e0bbb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9dc156be0 0x7fa9dc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.510+0000 7fa9e0bbb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9dc0cde80 con 0x7fa9dc156810 2023-10-11T06:12:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.510+0000 7fa9e0bbb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc0ce140 con 0x7fa9dc156be0 2023-10-11T06:12:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.510+0000 7fa9e0bbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc15da30 con 0x7fa9dc06c880 2023-10-11T06:12:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9dad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9dc156be0 0x7fa9dc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9dad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9dc156be0 0x7fa9dc15d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53228/0 (socket says 172.21.15.84:53228) 2023-10-11T06:12:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9dad9d640 1 -- 172.21.15.84:0/3449718156 learned_addr learned my addr 172.21.15.84:0/3449718156 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9d3fff640 1 --2- 172.21.15.84:0/3449718156 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc06c880 0x7fa9dc06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9dad9d640 1 -- 172.21.15.84:0/3449718156 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc06c880 msgr2=0x7fa9dc06d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:14.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9dad9d640 1 --2- 172.21.15.84:0/3449718156 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc06c880 0x7fa9dc06d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9dad9d640 1 -- 172.21.15.84:0/3449718156 >> v1:172.21.15.84:6789/0 conn(0x7fa9dc156810 legacy=0x7fa9dc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9d9d9b640 1 -- 172.21.15.84:0/3449718156 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324633398 0 0) 0x7fa9dc0cde80 con 0x7fa9dc156810 2023-10-11T06:12:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9dad9d640 1 -- 172.21.15.84:0/3449718156 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc06df80 con 0x7fa9dc156be0 2023-10-11T06:12:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.511+0000 7fa9d3fff640 1 --2- 172.21.15.84:0/3449718156 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc06c880 0x7fa9dc06d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:14.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.512+0000 7fa9dad9d640 1 --2- 172.21.15.84:0/3449718156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc156be0 0x7fa9dc15d4f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009e50 tx=0x7fa9d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0d981cbcc58e3b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.513+0000 7fa9d9d9b640 1 -- 172.21.15.84:0/3449718156 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d402ca70 con 0x7fa9dc156be0 2023-10-11T06:12:14.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.513+0000 7fa9d9d9b640 1 -- 172.21.15.84:0/3449718156 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d402cbd0 con 0x7fa9dc156be0 2023-10-11T06:12:14.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.513+0000 7fa9d9d9b640 1 -- 172.21.15.84:0/3449718156 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40343f0 con 0x7fa9dc156be0 2023-10-11T06:12:14.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.513+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3449718156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc156be0 msgr2=0x7fa9dc15d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.513+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3449718156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc156be0 0x7fa9dc15d4f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009e50 tx=0x7fa9d402bed0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.514+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3449718156 shutdown_connections 2023-10-11T06:12:14.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.514+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3449718156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc156be0 0x7fa9dc15d4f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.514+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3449718156 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc06c880 0x7fa9dc06d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.514+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3449718156 >> 172.21.15.84:0/3449718156 conn(0x7fa9dc069c70 msgr2=0x7fa9dc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:14.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.514+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3449718156 shutdown_connections 2023-10-11T06:12:14.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.514+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3449718156 wait complete. 2023-10-11T06:12:14.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.516+0000 7fa9e0bbb640 1 Processor -- start 2023-10-11T06:12:14.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.516+0000 7fa9e0bbb640 1 -- start start 2023-10-11T06:12:14.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.517+0000 7fa9e0bbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc06c880 0x7fa9dc1f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.517+0000 7fa9e0bbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc156810 0x7fa9dc0abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.517+0000 7fa9e0bbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9dc156be0 0x7fa9dc0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.517+0000 7fa9e0bbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1528c0 con 0x7fa9dc06c880 2023-10-11T06:12:14.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.517+0000 7fa9e0bbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1f2d20 con 0x7fa9dc156be0 2023-10-11T06:12:14.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.517+0000 7fa9e0bbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1f60a0 con 0x7fa9dc156810 2023-10-11T06:12:14.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.517+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc06c880 0x7fa9dc1f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc06c880 0x7fa9dc1f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53236/0 (socket says 172.21.15.84:53236) 2023-10-11T06:12:14.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 -- 172.21.15.84:0/3605994484 learned_addr learned my addr 172.21.15.84:0/3605994484 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:14.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 -- 172.21.15.84:0/3605994484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc156810 msgr2=0x7fa9dc0abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc156810 0x7fa9dc0abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 -- 172.21.15.84:0/3605994484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9dc156be0 msgr2=0x7fa9dc0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9dc156be0 0x7fa9dc0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 -- 172.21.15.84:0/3605994484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4009b30 con 0x7fa9dc06c880 2023-10-11T06:12:14.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9da59c640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc06c880 0x7fa9dc1f5b60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c4b0 tx=0x7fa9cc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9d2ffd640 1 -- 172.21.15.84:0/3605994484 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc005480 con 0x7fa9dc06c880 2023-10-11T06:12:14.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9d2ffd640 1 -- 172.21.15.84:0/3605994484 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0035f0 con 0x7fa9dc06c880 2023-10-11T06:12:14.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9d2ffd640 1 -- 172.21.15.84:0/3605994484 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0039e0 con 0x7fa9dc06c880 2023-10-11T06:12:14.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.518+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc1f2090 con 0x7fa9dc06c880 2023-10-11T06:12:14.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.519+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc1f2650 con 0x7fa9dc06c880 2023-10-11T06:12:14.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.520+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc073ed0 con 0x7fa9dc06c880 2023-10-11T06:12:14.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.521+0000 7fa9d2ffd640 1 -- 172.21.15.84:0/3605994484 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa9cc003b40 con 0x7fa9dc06c880 2023-10-11T06:12:14.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.521+0000 7fa9d2ffd640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa9bc06a830 0x7fa9bc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.522+0000 7fa9d2ffd640 1 -- 172.21.15.84:0/3605994484 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa9cc0885d0 con 0x7fa9dc06c880 2023-10-11T06:12:14.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.522+0000 7fa9d3fff640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa9bc06a830 0x7fa9bc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.522+0000 7fa9d3fff640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa9bc06a830 0x7fa9bc06cce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.526+0000 7fa9d2ffd640 1 -- 172.21.15.84:0/3605994484 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa9cc053880 con 0x7fa9dc06c880 2023-10-11T06:12:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.715+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9dc15d190 con 0x7fa9dc06c880 2023-10-11T06:12:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.716+0000 7fa9d2ffd640 1 -- 172.21.15.84:0/3605994484 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9cc0170a0 con 0x7fa9dc06c880 2023-10-11T06:12:14.717 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.718+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa9bc06a830 msgr2=0x7fa9bc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.718+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa9bc06a830 0x7fa9bc06cce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4008040 comp rx=0 tx=0).stop 2023-10-11T06:12:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.718+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc06c880 msgr2=0x7fa9dc1f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.718+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc06c880 0x7fa9dc1f5b60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c4b0 tx=0x7fa9cc00c970 comp rx=0 tx=0).stop 2023-10-11T06:12:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 shutdown_connections 2023-10-11T06:12:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa9bc06a830 0x7fa9bc06cce0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9dc156be0 0x7fa9dc0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9dc156810 0x7fa9dc0abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 --2- 172.21.15.84:0/3605994484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9dc06c880 0x7fa9dc1f5b60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 >> 172.21.15.84:0/3605994484 conn(0x7fa9dc069c70 msgr2=0x7fa9dc0c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 shutdown_connections 2023-10-11T06:12:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.719+0000 7fa9e0bbb640 1 -- 172.21.15.84:0/3605994484 wait complete. 2023-10-11T06:12:14.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:14.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.953+0000 7ff3297a2640 1 Processor -- start 2023-10-11T06:12:14.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.953+0000 7ff3297a2640 1 -- start start 2023-10-11T06:12:14.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff3297a2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff324164f70 0x7ff3240affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff3297a2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3240c9160 con 0x7ff3240b0510 2023-10-11T06:12:14.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff3297a2640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3240c8ba0 con 0x7ff324160630 2023-10-11T06:12:14.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff3297a2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3240b0990 con 0x7ff324164f70 2023-10-11T06:12:14.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff31bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff324164f70 0x7ff3240affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff31bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff324164f70 0x7ff3240affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53252/0 (socket says 172.21.15.84:53252) 2023-10-11T06:12:14.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff31bfff640 1 -- 172.21.15.84:0/3044532360 learned_addr learned my addr 172.21.15.84:0/3044532360 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:14.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff328fa1640 1 --1- 172.21.15.84:0/3044532360 >> v1:172.21.15.84:6789/0 conn(0x7ff3240b0510 0x7ff3240cc4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49872/0 (socket says 172.21.15.84:49872) 2023-10-11T06:12:14.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.955+0000 7ff31bfff640 1 -- 172.21.15.84:0/3044532360 >> v1:172.21.15.99:6789/0 conn(0x7ff324160630 legacy=0x7ff3240c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:14.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.956+0000 7ff31bfff640 1 -- 172.21.15.84:0/3044532360 >> v1:172.21.15.84:6789/0 conn(0x7ff3240b0510 legacy=0x7ff3240cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.956+0000 7ff3237fe640 1 -- 172.21.15.84:0/3044532360 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229462585 0 0) 0x7ff3240c9160 con 0x7ff3240b0510 2023-10-11T06:12:14.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.956+0000 7ff31bfff640 1 -- 172.21.15.84:0/3044532360 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32406a6f0 con 0x7ff324164f70 2023-10-11T06:12:14.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.957+0000 7ff31bfff640 1 --2- 172.21.15.84:0/3044532360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324164f70 0x7ff3240affd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009870 tx=0x7ff30c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb842fe9b8be19b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.957+0000 7ff3237fe640 1 -- 172.21.15.84:0/3044532360 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c031870 con 0x7ff324164f70 2023-10-11T06:12:14.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.957+0000 7ff3237fe640 1 -- 172.21.15.84:0/3044532360 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c0319d0 con 0x7ff324164f70 2023-10-11T06:12:14.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.957+0000 7ff3237fe640 1 -- 172.21.15.84:0/3044532360 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c031d60 con 0x7ff324164f70 2023-10-11T06:12:14.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.958+0000 7ff3297a2640 1 -- 172.21.15.84:0/3044532360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324164f70 msgr2=0x7ff3240affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.958+0000 7ff3297a2640 1 --2- 172.21.15.84:0/3044532360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324164f70 0x7ff3240affd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009870 tx=0x7ff30c02b910 comp rx=0 tx=0).stop 2023-10-11T06:12:14.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.958+0000 7ff3297a2640 1 -- 172.21.15.84:0/3044532360 shutdown_connections 2023-10-11T06:12:14.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.958+0000 7ff3297a2640 1 --2- 172.21.15.84:0/3044532360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324164f70 0x7ff3240affd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.958+0000 7ff3297a2640 1 -- 172.21.15.84:0/3044532360 >> 172.21.15.84:0/3044532360 conn(0x7ff32406bfb0 msgr2=0x7ff32406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:14.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.958+0000 7ff3297a2640 1 -- 172.21.15.84:0/3044532360 shutdown_connections 2023-10-11T06:12:14.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.958+0000 7ff3297a2640 1 -- 172.21.15.84:0/3044532360 wait complete. 2023-10-11T06:12:14.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.960+0000 7ff3297a2640 1 Processor -- start 2023-10-11T06:12:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.960+0000 7ff3297a2640 1 -- start start 2023-10-11T06:12:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff3297a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3240b0510 0x7ff3241f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff3297a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324160630 0x7ff3240cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff3297a2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff324164f70 0x7ff3240b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff3297a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3241565d0 con 0x7ff324160630 2023-10-11T06:12:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff3297a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240cf3d0 con 0x7ff3240b0510 2023-10-11T06:12:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff3297a2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240b2970 con 0x7ff324164f70 2023-10-11T06:12:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff31bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324160630 0x7ff3240cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff31bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324160630 0x7ff3240cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53258/0 (socket says 172.21.15.84:53258) 2023-10-11T06:12:14.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.961+0000 7ff31bfff640 1 -- 172.21.15.84:0/1693470393 learned_addr learned my addr 172.21.15.84:0/1693470393 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff31bfff640 1 -- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff324164f70 msgr2=0x7ff3240b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff31bfff640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff324164f70 0x7ff3240b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff31bfff640 1 -- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3240b0510 msgr2=0x7ff3241f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff323fff640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3240b0510 0x7ff3241f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff31bfff640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3240b0510 0x7ff3241f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:14.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff31bfff640 1 -- 172.21.15.84:0/1693470393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c0094d0 con 0x7ff324160630 2023-10-11T06:12:14.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff323fff640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3240b0510 0x7ff3241f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:14.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff31bfff640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324160630 0x7ff3240cee90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009870 tx=0x7ff30c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff3217fa640 1 -- 172.21.15.84:0/1693470393 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c004230 con 0x7ff324160630 2023-10-11T06:12:14.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff3217fa640 1 -- 172.21.15.84:0/1693470393 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c004390 con 0x7ff324160630 2023-10-11T06:12:14.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3240cd020 con 0x7ff324160630 2023-10-11T06:12:14.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff3217fa640 1 -- 172.21.15.84:0/1693470393 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c03aba0 con 0x7ff324160630 2023-10-11T06:12:14.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.962+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240cdda0 con 0x7ff324160630 2023-10-11T06:12:14.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.963+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32404e890 con 0x7ff324160630 2023-10-11T06:12:14.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.964+0000 7ff3217fa640 1 -- 172.21.15.84:0/1693470393 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff30c042070 con 0x7ff324160630 2023-10-11T06:12:14.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.965+0000 7ff3217fa640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff30406a830 0x7ff30406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:14.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.965+0000 7ff3217fa640 1 -- 172.21.15.84:0/1693470393 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff30c02d040 con 0x7ff324160630 2023-10-11T06:12:14.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.966+0000 7ff323fff640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff30406a830 0x7ff30406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:14.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.966+0000 7ff323fff640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff30406a830 0x7ff30406cce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff314000f60 tx=0x7ff314009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:14.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:14.970+0000 7ff3217fa640 1 -- 172.21.15.84:0/1693470393 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff30c0765a0 con 0x7ff324160630 2023-10-11T06:12:15.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.160+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff32406a330 con 0x7ff324160630 2023-10-11T06:12:15.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.161+0000 7ff3217fa640 1 -- 172.21.15.84:0/1693470393 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff30c035020 con 0x7ff324160630 2023-10-11T06:12:15.163 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff30406a830 msgr2=0x7ff30406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff30406a830 0x7ff30406cce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff314000f60 tx=0x7ff314009040 comp rx=0 tx=0).stop 2023-10-11T06:12:15.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324160630 msgr2=0x7ff3240cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324160630 0x7ff3240cee90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009870 tx=0x7ff30c0046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 shutdown_connections 2023-10-11T06:12:15.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff30406a830 0x7ff30406cce0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff324164f70 0x7ff3240b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff324160630 0x7ff3240cee90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 --2- 172.21.15.84:0/1693470393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3240b0510 0x7ff3241f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 >> 172.21.15.84:0/1693470393 conn(0x7ff32406bfb0 msgr2=0x7ff324160da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 shutdown_connections 2023-10-11T06:12:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.164+0000 7ff3297a2640 1 -- 172.21.15.84:0/1693470393 wait complete. 2023-10-11T06:12:15.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:15.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.400+0000 7f21f6fe4640 1 Processor -- start 2023-10-11T06:12:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.400+0000 7f21f6fe4640 1 -- start start 2023-10-11T06:12:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21f6fe4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21f00d2ac0 0x7f21f00d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21f6fe4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f00cf220 con 0x7f21f015f7e0 2023-10-11T06:12:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21f6fe4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f00cf500 con 0x7f21f00d0940 2023-10-11T06:12:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21f6fe4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f00d3480 con 0x7f21f00d2ac0 2023-10-11T06:12:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21effff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21f00d2ac0 0x7f21f00d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21effff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21f00d2ac0 0x7f21f00d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53264/0 (socket says 172.21.15.84:53264) 2023-10-11T06:12:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21effff640 1 -- 172.21.15.84:0/1640407353 learned_addr learned my addr 172.21.15.84:0/1640407353 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:15.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.402+0000 7f21f4d80640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f21f015f7e0 0x7f21f00cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49888/0 (socket says 172.21.15.84:49888) 2023-10-11T06:12:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.403+0000 7f21effff640 1 -- 172.21.15.84:0/1640407353 >> v1:172.21.15.99:6789/0 conn(0x7f21f00d0940 legacy=0x7f21f00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.403+0000 7f21effff640 1 -- 172.21.15.84:0/1640407353 >> v1:172.21.15.84:6789/0 conn(0x7f21f015f7e0 legacy=0x7f21f00cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.403+0000 7f21effff640 1 -- 172.21.15.84:0/1640407353 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f00cf7e0 con 0x7f21f00d2ac0 2023-10-11T06:12:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.404+0000 7f21effff640 1 --2- 172.21.15.84:0/1640407353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f00d2ac0 0x7f21f00d2f40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959c082c974b875e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.404+0000 7f21ef7fe640 1 -- 172.21.15.84:0/1640407353 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc030860 con 0x7f21f00d2ac0 2023-10-11T06:12:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.404+0000 7f21ef7fe640 1 -- 172.21.15.84:0/1640407353 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc0309c0 con 0x7f21f00d2ac0 2023-10-11T06:12:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.404+0000 7f21ef7fe640 1 -- 172.21.15.84:0/1640407353 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc030d50 con 0x7f21f00d2ac0 2023-10-11T06:12:15.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.405+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1640407353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f00d2ac0 msgr2=0x7f21f00d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.405+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1640407353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f00d2ac0 0x7f21f00d2f40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009870 tx=0x7f21dc02b9a0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.405+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1640407353 shutdown_connections 2023-10-11T06:12:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.405+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1640407353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f00d2ac0 0x7f21f00d2f40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.405+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1640407353 >> 172.21.15.84:0/1640407353 conn(0x7f21f00713a0 msgr2=0x7f21f00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.405+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1640407353 shutdown_connections 2023-10-11T06:12:15.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.405+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1640407353 wait complete. 2023-10-11T06:12:15.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.407+0000 7f21f6fe4640 1 Processor -- start 2023-10-11T06:12:15.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.407+0000 7f21f6fe4640 1 -- start start 2023-10-11T06:12:15.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f6fe4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21f00d0940 0x7f21f0149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f6fe4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21f00d2ac0 0x7f21f00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f6fe4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f015f7e0 0x7f21f00cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f6fe4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f00cc920 con 0x7f21f015f7e0 2023-10-11T06:12:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f6fe4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f01f2da0 con 0x7f21f00d0940 2023-10-11T06:12:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f6fe4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f01f2ac0 con 0x7f21f00d2ac0 2023-10-11T06:12:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f5581640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f015f7e0 0x7f21f00cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f5581640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f015f7e0 0x7f21f00cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53274/0 (socket says 172.21.15.84:53274) 2023-10-11T06:12:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.408+0000 7f21f5581640 1 -- 172.21.15.84:0/1967507548 learned_addr learned my addr 172.21.15.84:0/1967507548 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f4d80640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21f00d0940 0x7f21f0149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21effff640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21f00d2ac0 0x7f21f00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f5581640 1 -- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21f00d2ac0 msgr2=0x7f21f00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f5581640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21f00d2ac0 0x7f21f00cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f5581640 1 -- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21f00d0940 msgr2=0x7f21f0149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f5581640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21f00d0940 0x7f21f0149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f5581640 1 -- 172.21.15.84:0/1967507548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0094d0 con 0x7f21f015f7e0 2023-10-11T06:12:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f4d80640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21f00d0940 0x7f21f0149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21effff640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21f00d2ac0 0x7f21f00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f5581640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f015f7e0 0x7f21f00cd6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c780 tx=0x7f21e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21ed7fa640 1 -- 172.21.15.84:0/1967507548 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8003d80 con 0x7f21f015f7e0 2023-10-11T06:12:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21ed7fa640 1 -- 172.21.15.84:0/1967507548 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e800ce90 con 0x7f21f015f7e0 2023-10-11T06:12:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21ed7fa640 1 -- 172.21.15.84:0/1967507548 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e80052b0 con 0x7f21f015f7e0 2023-10-11T06:12:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.409+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f01f55b0 con 0x7f21f015f7e0 2023-10-11T06:12:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.410+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f01f5b70 con 0x7f21f015f7e0 2023-10-11T06:12:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.411+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f01f5d60 con 0x7f21f015f7e0 2023-10-11T06:12:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.412+0000 7f21ed7fa640 1 -- 172.21.15.84:0/1967507548 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f21e8005410 con 0x7f21f015f7e0 2023-10-11T06:12:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.412+0000 7f21ed7fa640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21d406a830 0x7f21d406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.413+0000 7f21ed7fa640 1 -- 172.21.15.84:0/1967507548 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f21e8088510 con 0x7f21f015f7e0 2023-10-11T06:12:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.413+0000 7f21f4d80640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21d406a830 0x7f21d406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.414+0000 7f21f4d80640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21d406a830 0x7f21d406cce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21e4000f60 tx=0x7f21e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.417+0000 7f21ed7fa640 1 -- 172.21.15.84:0/1967507548 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f21e8053740 con 0x7f21f015f7e0 2023-10-11T06:12:15.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.604+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21f0073ed0 con 0x7f21f015f7e0 2023-10-11T06:12:15.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.605+0000 7f21ed7fa640 1 -- 172.21.15.84:0/1967507548 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21e8056d60 con 0x7f21f015f7e0 2023-10-11T06:12:15.606 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:15.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.607+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21d406a830 msgr2=0x7f21d406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.607+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21d406a830 0x7f21d406cce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21e4000f60 tx=0x7f21e4009040 comp rx=0 tx=0).stop 2023-10-11T06:12:15.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.607+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f015f7e0 msgr2=0x7f21f00cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.607+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f015f7e0 0x7f21f00cd6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c780 tx=0x7f21e800cc40 comp rx=0 tx=0).stop 2023-10-11T06:12:15.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 shutdown_connections 2023-10-11T06:12:15.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21d406a830 0x7f21d406cce0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f015f7e0 0x7f21f00cd6a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21f00d2ac0 0x7f21f00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 --2- 172.21.15.84:0/1967507548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21f00d0940 0x7f21f0149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 >> 172.21.15.84:0/1967507548 conn(0x7f21f00713a0 msgr2=0x7f21f0160590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:15.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 shutdown_connections 2023-10-11T06:12:15.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.608+0000 7f21f6fe4640 1 -- 172.21.15.84:0/1967507548 wait complete. 2023-10-11T06:12:15.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:15.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.848+0000 7fac14d92640 1 Processor -- start 2023-10-11T06:12:15.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.849+0000 7fac14d92640 1 -- start start 2023-10-11T06:12:15.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac14d92640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac1015f7e0 0x7fac100cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac14d92640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac100cd6c0 0x7fac100d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac14d92640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100d3000 0x7fac1015fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac14d92640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac100d3460 con 0x7fac1015f7e0 2023-10-11T06:12:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac14d92640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac100cf220 con 0x7fac100cd6c0 2023-10-11T06:12:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac14d92640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac100cf500 con 0x7fac100d3000 2023-10-11T06:12:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac0e59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac1015f7e0 0x7fac100cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac0e59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac1015f7e0 0x7fac100cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53288/0 (socket says 172.21.15.84:53288) 2023-10-11T06:12:15.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.850+0000 7fac0e59c640 1 -- 172.21.15.84:0/720554711 learned_addr learned my addr 172.21.15.84:0/720554711 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.851+0000 7fac0e59c640 1 -- 172.21.15.84:0/720554711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100d3000 msgr2=0x7fac1015fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.851+0000 7fac0e59c640 1 --2- 172.21.15.84:0/720554711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100d3000 0x7fac1015fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.851+0000 7fac0e59c640 1 -- 172.21.15.84:0/720554711 >> v2:172.21.15.99:3300/0 conn(0x7fac100cd6c0 msgr2=0x7fac100d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.851+0000 7fac0e59c640 1 --2- 172.21.15.84:0/720554711 >> v2:172.21.15.99:3300/0 conn(0x7fac100cd6c0 0x7fac100d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.851+0000 7fac0e59c640 1 -- 172.21.15.84:0/720554711 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac100cf7e0 con 0x7fac1015f7e0 2023-10-11T06:12:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.852+0000 7fac0e59c640 1 --2- 172.21.15.84:0/720554711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100cd180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fac1008adb0 tx=0x7fac0002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1238563a189e046 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.852+0000 7fac0d59a640 1 -- 172.21.15.84:0/720554711 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0002ca80 con 0x7fac1015f7e0 2023-10-11T06:12:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.852+0000 7fac0d59a640 1 -- 172.21.15.84:0/720554711 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0002cbe0 con 0x7fac1015f7e0 2023-10-11T06:12:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.852+0000 7fac0d59a640 1 -- 172.21.15.84:0/720554711 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac000343f0 con 0x7fac1015f7e0 2023-10-11T06:12:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 -- 172.21.15.84:0/720554711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 msgr2=0x7fac100cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 --2- 172.21.15.84:0/720554711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100cd180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fac1008adb0 tx=0x7fac0002bee0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 -- 172.21.15.84:0/720554711 shutdown_connections 2023-10-11T06:12:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 --2- 172.21.15.84:0/720554711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100d3000 0x7fac1015fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 --2- 172.21.15.84:0/720554711 >> v2:172.21.15.99:3300/0 conn(0x7fac100cd6c0 0x7fac100d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 --2- 172.21.15.84:0/720554711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100cd180 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 -- 172.21.15.84:0/720554711 >> 172.21.15.84:0/720554711 conn(0x7fac100713a0 msgr2=0x7fac100717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 -- 172.21.15.84:0/720554711 shutdown_connections 2023-10-11T06:12:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.853+0000 7fac14d92640 1 -- 172.21.15.84:0/720554711 wait complete. 2023-10-11T06:12:15.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.855+0000 7fac14d92640 1 Processor -- start 2023-10-11T06:12:15.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.855+0000 7fac14d92640 1 -- start start 2023-10-11T06:12:15.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac14d92640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100cd6c0 0x7fac100cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac14d92640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac100d3000 0x7fac100b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac14d92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac14d92640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac100cbe70 con 0x7fac1015f7e0 2023-10-11T06:12:15.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac14d92640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac100c9280 con 0x7fac100d3000 2023-10-11T06:12:15.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac14d92640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac100c9560 con 0x7fac100cd6c0 2023-10-11T06:12:15.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac0ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac0ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53296/0 (socket says 172.21.15.84:53296) 2023-10-11T06:12:15.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.856+0000 7fac0ed9d640 1 -- 172.21.15.84:0/542552108 learned_addr learned my addr 172.21.15.84:0/542552108 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:15.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0dd9b640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac100d3000 0x7fac100b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0e59c640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100cd6c0 0x7fac100cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0ed9d640 1 -- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100cd6c0 msgr2=0x7fac100cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0ed9d640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100cd6c0 0x7fac100cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0ed9d640 1 -- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac100d3000 msgr2=0x7fac100b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:15.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0ed9d640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac100d3000 0x7fac100b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:15.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0ed9d640 1 -- 172.21.15.84:0/542552108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac00009b30 con 0x7fac1015f7e0 2023-10-11T06:12:15.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0dd9b640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac100d3000 0x7fac100b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:15.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac0ed9d640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100b8070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fac0800b410 tx=0x7fac0800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:15.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac06ffd640 1 -- 172.21.15.84:0/542552108 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08011830 con 0x7fac1015f7e0 2023-10-11T06:12:15.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac06ffd640 1 -- 172.21.15.84:0/542552108 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac080119b0 con 0x7fac1015f7e0 2023-10-11T06:12:15.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac100cc110 con 0x7fac1015f7e0 2023-10-11T06:12:15.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac06ffd640 1 -- 172.21.15.84:0/542552108 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08011d60 con 0x7fac1015f7e0 2023-10-11T06:12:15.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.857+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac100c9c90 con 0x7fac1015f7e0 2023-10-11T06:12:15.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.859+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac100cc350 con 0x7fac1015f7e0 2023-10-11T06:12:15.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.859+0000 7fac06ffd640 1 -- 172.21.15.84:0/542552108 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fac0801a400 con 0x7fac1015f7e0 2023-10-11T06:12:15.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.860+0000 7fac06ffd640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabf006a830 0x7fabf006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:15.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.860+0000 7fac06ffd640 1 -- 172.21.15.84:0/542552108 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fac080890b0 con 0x7fac1015f7e0 2023-10-11T06:12:15.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.861+0000 7fac0e59c640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabf006a830 0x7fabf006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:15.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.861+0000 7fac0e59c640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabf006a830 0x7fabf006cce0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac00035000 tx=0x7fac00030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:15.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:15.865+0000 7fac06ffd640 1 -- 172.21.15.84:0/542552108 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fac100cc350 con 0x7fac1015f7e0 2023-10-11T06:12:16.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.053+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac100cc350 con 0x7fac1015f7e0 2023-10-11T06:12:16.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.054+0000 7fac06ffd640 1 -- 172.21.15.84:0/542552108 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac100cc350 con 0x7fac1015f7e0 2023-10-11T06:12:16.056 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:16.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.057+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabf006a830 msgr2=0x7fabf006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.057+0000 7fac14d92640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabf006a830 0x7fabf006cce0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac00035000 tx=0x7fac00030000 comp rx=0 tx=0).stop 2023-10-11T06:12:16.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.057+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 msgr2=0x7fac100b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.057+0000 7fac14d92640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100b8070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fac0800b410 tx=0x7fac0800b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.057+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 shutdown_connections 2023-10-11T06:12:16.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.057+0000 7fac14d92640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabf006a830 0x7fabf006cce0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.057+0000 7fac14d92640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac1015f7e0 0x7fac100b8070 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.058+0000 7fac14d92640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac100d3000 0x7fac100b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.058+0000 7fac14d92640 1 --2- 172.21.15.84:0/542552108 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac100cd6c0 0x7fac100cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.058+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 >> 172.21.15.84:0/542552108 conn(0x7fac100713a0 msgr2=0x7fac10071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:16.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.058+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 shutdown_connections 2023-10-11T06:12:16.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.058+0000 7fac14d92640 1 -- 172.21.15.84:0/542552108 wait complete. 2023-10-11T06:12:16.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.286+0000 7f5632bfa640 1 Processor -- start 2023-10-11T06:12:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.286+0000 7f5632bfa640 1 -- start start 2023-10-11T06:12:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.288+0000 7f5632bfa640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 0x7f562c0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.288+0000 7f5632bfa640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0c98c0 con 0x7f562c159ec0 2023-10-11T06:12:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.288+0000 7f5632bfa640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0ca770 con 0x7f562c0715f0 2023-10-11T06:12:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.288+0000 7f5632bfa640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c070040 con 0x7f562c0b3f50 2023-10-11T06:12:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.288+0000 7f56313f7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f562c159ec0 0x7f562c0b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49890/0 (socket says 172.21.15.84:49890) 2023-10-11T06:12:16.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.288+0000 7f56313f7640 1 -- 172.21.15.84:0/606582826 learned_addr learned my addr 172.21.15.84:0/606582826 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f56323f9640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 0x7f562c0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208092662 0 0) 0x7f562c0c98c0 con 0x7f562c159ec0 2023-10-11T06:12:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5618003520 con 0x7f562c159ec0 2023-10-11T06:12:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401113253 0 0) 0x7f5618003520 con 0x7f562c159ec0 2023-10-11T06:12:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 msgr2=0x7f562c0c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 0x7f562c0c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 >> v1:172.21.15.99:6789/0 conn(0x7f562c0715f0 legacy=0x7f562c0719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c0c9af0 con 0x7f562c159ec0 2023-10-11T06:12:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f561c001670 con 0x7f562c159ec0 2023-10-11T06:12:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 >> v1:172.21.15.84:6789/0 conn(0x7f562c159ec0 legacy=0x7f562c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56180039a0 0x7f5618003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56180047c0 0x7f5618004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 0x7f5618005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c070040 con 0x7f56180039a0 2023-10-11T06:12:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561800fe70 con 0x7f56180047c0 2023-10-11T06:12:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.289+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5618010030 con 0x7f562c0b3f50 2023-10-11T06:12:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f5631bf8640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56180039a0 0x7f5618003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f56323f9640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 0x7f5618005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f5631bf8640 1 -- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 msgr2=0x7f5618005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f5631bf8640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 0x7f5618005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f5631bf8640 1 -- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56180047c0 msgr2=0x7f5618004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f5631bf8640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56180047c0 0x7f5618004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f5631bf8640 1 -- 172.21.15.84:0/606582826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618003520 con 0x7f56180039a0 2023-10-11T06:12:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.290+0000 7f56323f9640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f562c0b3f50 0x7f5618005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:16.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.291+0000 7f5631bf8640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56180039a0 0x7f5618003db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5628001750 tx=0x7f56180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503774059067b2f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:16.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.291+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280052d0 con 0x7f56180039a0 2023-10-11T06:12:16.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.291+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562802e050 con 0x7f56180039a0 2023-10-11T06:12:16.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.291+0000 7f5630bf6640 1 -- 172.21.15.84:0/606582826 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280035f0 con 0x7f56180039a0 2023-10-11T06:12:16.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.292+0000 7f5632bfa640 1 -- 172.21.15.84:0/606582826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56180039a0 msgr2=0x7f5618003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.292+0000 7f5632bfa640 1 --2- 172.21.15.84:0/606582826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56180039a0 0x7f5618003db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5628001750 tx=0x7f56180057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.292+0000 7f56323f9640 1 -- 172.21.15.84:0/606582826 reap_dead start 2023-10-11T06:12:16.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.292+0000 7f5632bfa640 1 -- 172.21.15.84:0/606582826 shutdown_connections 2023-10-11T06:12:16.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.292+0000 7f5632bfa640 1 -- 172.21.15.84:0/606582826 >> 172.21.15.84:0/606582826 conn(0x7f562c069c70 msgr2=0x7f562c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:16.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.292+0000 7f5632bfa640 1 -- 172.21.15.84:0/606582826 shutdown_connections 2023-10-11T06:12:16.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.292+0000 7f5632bfa640 1 -- 172.21.15.84:0/606582826 wait complete. 2023-10-11T06:12:16.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.294+0000 7f5632bfa640 1 Processor -- start 2023-10-11T06:12:16.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.294+0000 7f5632bfa640 1 -- start start 2023-10-11T06:12:16.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f5632bfa640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56180039a0 0x7f562c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f5632bfa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f562c0c3130 0x7f562c0c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f5632bfa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f562c15c400 0x7f562c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f5632bfa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0c2c30 con 0x7f562c15c400 2023-10-11T06:12:16.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f5632bfa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15cdb0 con 0x7f562c0c3130 2023-10-11T06:12:16.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f5632bfa640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0b1170 con 0x7f56180039a0 2023-10-11T06:12:16.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f56323f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f562c15c400 0x7f562c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:16.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f56323f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f562c15c400 0x7f562c15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53314/0 (socket says 172.21.15.84:53314) 2023-10-11T06:12:16.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.295+0000 7f56323f9640 1 -- 172.21.15.84:0/4127675010 learned_addr learned my addr 172.21.15.84:0/4127675010 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:16.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f5631bf8640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56180039a0 0x7f562c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:16.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56323f9640 1 -- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56180039a0 msgr2=0x7f562c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56323f9640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56180039a0 0x7f562c154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56323f9640 1 -- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f562c0c3130 msgr2=0x7f562c0c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56323f9640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f562c0c3130 0x7f562c0c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56323f9640 1 -- 172.21.15.84:0/4127675010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c0b13c0 con 0x7f562c15c400 2023-10-11T06:12:16.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f5631bf8640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56180039a0 0x7f562c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:16.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56323f9640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f562c15c400 0x7f562c15c870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5618004740 tx=0x7f562400f0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:16.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56227fc640 1 -- 172.21.15.84:0/4127675010 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624003b90 con 0x7f562c15c400 2023-10-11T06:12:16.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56227fc640 1 -- 172.21.15.84:0/4127675010 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5624011040 con 0x7f562c15c400 2023-10-11T06:12:16.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f56227fc640 1 -- 172.21.15.84:0/4127675010 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624004020 con 0x7f562c15c400 2023-10-11T06:12:16.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c0b16a0 con 0x7f562c15c400 2023-10-11T06:12:16.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.296+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c0b1c60 con 0x7f562c15c400 2023-10-11T06:12:16.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.297+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c04e890 con 0x7f562c15c400 2023-10-11T06:12:16.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.298+0000 7f56227fc640 1 -- 172.21.15.84:0/4127675010 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5624004180 con 0x7f562c15c400 2023-10-11T06:12:16.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.299+0000 7f56227fc640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561006a830 0x7f561006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:16.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.299+0000 7f56227fc640 1 -- 172.21.15.84:0/4127675010 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5624088e20 con 0x7f562c15c400 2023-10-11T06:12:16.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.299+0000 7f5631bf8640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561006a830 0x7f561006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:16.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.300+0000 7f5631bf8640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561006a830 0x7f561006cce0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5628033000 tx=0x7f5628009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:16.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.303+0000 7f56227fc640 1 -- 172.21.15.84:0/4127675010 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5624053fc0 con 0x7f562c15c400 2023-10-11T06:12:16.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.499+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f562c0b1e20 con 0x7f562c15c400 2023-10-11T06:12:16.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.500+0000 7f56227fc640 1 -- 172.21.15.84:0/4127675010 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f56240575e0 con 0x7f562c15c400 2023-10-11T06:12:16.501 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:16.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561006a830 msgr2=0x7f561006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561006a830 0x7f561006cce0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5628033000 tx=0x7f5628009030 comp rx=0 tx=0).stop 2023-10-11T06:12:16.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f562c15c400 msgr2=0x7f562c15c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:16.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f562c15c400 0x7f562c15c870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5618004740 tx=0x7f562400f0d0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 shutdown_connections 2023-10-11T06:12:16.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f561006a830 0x7f561006cce0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f562c15c400 0x7f562c15c870 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f562c0c3130 0x7f562c0c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 --2- 172.21.15.84:0/4127675010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56180039a0 0x7f562c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:16.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 >> 172.21.15.84:0/4127675010 conn(0x7f562c069c70 msgr2=0x7f562c0aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:16.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 shutdown_connections 2023-10-11T06:12:16.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:16.502+0000 7f5632bfa640 1 -- 172.21.15.84:0/4127675010 wait complete. 2023-10-11T06:12:16.521 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:12:16.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:16.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-11T06:12:16.619 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:16.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:12:16.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:16.620 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T06:12:16.670 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:12:16.670 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:16.734 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:12:16.735 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:16.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:17.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.016+0000 7f5b2ffff640 1 Processor -- start 2023-10-11T06:12:17.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.016+0000 7f5b2ffff640 1 -- start start 2023-10-11T06:12:17.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.018+0000 7f5b2ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b3015c330 0x7f5b300b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.018+0000 7f5b2ffff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3015c700 con 0x7f5b300afe10 2023-10-11T06:12:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.018+0000 7f5b2ffff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b300b02a0 con 0x7f5b300d0b00 2023-10-11T06:12:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.018+0000 7f5b2ffff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b300b0f20 con 0x7f5b3015c330 2023-10-11T06:12:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b267fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5b300afe10 0x7f5b300d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:49904/0 (socket says 172.21.15.84:49904) 2023-10-11T06:12:17.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b267fc640 1 -- 172.21.15.84:0/1950030840 learned_addr learned my addr 172.21.15.84:0/1950030840 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:17.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b2effd640 1 --2- 172.21.15.84:0/1950030840 >> v2:172.21.15.84:3300/0 conn(0x7f5b3015c330 0x7f5b300b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b2e7fc640 1 -- 172.21.15.84:0/1950030840 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727356222 0 0) 0x7f5b18001110 con 0x7f5b300afe10 2023-10-11T06:12:17.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b2e7fc640 1 -- 172.21.15.84:0/1950030840 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b10003520 con 0x7f5b300afe10 2023-10-11T06:12:17.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b2effd640 1 -- 172.21.15.84:0/1950030840 >> v1:172.21.15.99:6789/0 conn(0x7f5b300d0b00 legacy=0x7f5b300cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:17.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b2effd640 1 -- 172.21.15.84:0/1950030840 >> v1:172.21.15.84:6789/0 conn(0x7f5b300afe10 legacy=0x7f5b300d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.019+0000 7f5b2effd640 1 -- 172.21.15.84:0/1950030840 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300b0580 con 0x7f5b3015c330 2023-10-11T06:12:17.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.020+0000 7f5b2effd640 1 --2- 172.21.15.84:0/1950030840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b3015c330 0x7f5b300b1340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009790 tx=0x7f5b2002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7094e9a23e9b6c33 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.020+0000 7f5b2e7fc640 1 -- 172.21.15.84:0/1950030840 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b200316f0 con 0x7f5b3015c330 2023-10-11T06:12:17.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.020+0000 7f5b2e7fc640 1 -- 172.21.15.84:0/1950030840 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b20031850 con 0x7f5b3015c330 2023-10-11T06:12:17.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.020+0000 7f5b2e7fc640 1 -- 172.21.15.84:0/1950030840 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20031be0 con 0x7f5b3015c330 2023-10-11T06:12:17.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.021+0000 7f5b2ffff640 1 -- 172.21.15.84:0/1950030840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b3015c330 msgr2=0x7f5b300b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.021+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/1950030840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b3015c330 0x7f5b300b1340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009790 tx=0x7f5b2002b790 comp rx=0 tx=0).stop 2023-10-11T06:12:17.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.021+0000 7f5b2ffff640 1 -- 172.21.15.84:0/1950030840 shutdown_connections 2023-10-11T06:12:17.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.021+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/1950030840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b3015c330 0x7f5b300b1340 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.021+0000 7f5b2ffff640 1 -- 172.21.15.84:0/1950030840 >> 172.21.15.84:0/1950030840 conn(0x7f5b3009bb10 msgr2=0x7f5b300d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:17.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.021+0000 7f5b2ffff640 1 -- 172.21.15.84:0/1950030840 shutdown_connections 2023-10-11T06:12:17.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.021+0000 7f5b2ffff640 1 -- 172.21.15.84:0/1950030840 wait complete. 2023-10-11T06:12:17.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.023+0000 7f5b2ffff640 1 Processor -- start 2023-10-11T06:12:17.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.023+0000 7f5b2ffff640 1 -- start start 2023-10-11T06:12:17.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b300afe10 0x7f5b301526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b300d0b00 0x7f5b300d2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b301f4e70 0x7f5b300d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b30152c10 con 0x7f5b301f4e70 2023-10-11T06:12:17.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2ffff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b30152e00 con 0x7f5b300afe10 2023-10-11T06:12:17.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2ffff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301e5e20 con 0x7f5b300d0b00 2023-10-11T06:12:17.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b301f4e70 0x7f5b300d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b301f4e70 0x7f5b300d2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53328/0 (socket says 172.21.15.84:53328) 2023-10-11T06:12:17.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.024+0000 7f5b2f7fe640 1 -- 172.21.15.84:0/3944482383 learned_addr learned my addr 172.21.15.84:0/3944482383 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:17.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2effd640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b300afe10 0x7f5b301526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2f7fe640 1 -- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b300d0b00 msgr2=0x7f5b300d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2f7fe640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b300d0b00 0x7f5b300d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2f7fe640 1 -- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b300afe10 msgr2=0x7f5b301526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2f7fe640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b300afe10 0x7f5b301526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2f7fe640 1 -- 172.21.15.84:0/3944482383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200093a0 con 0x7f5b301f4e70 2023-10-11T06:12:17.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2effd640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b300afe10 0x7f5b301526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:17.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2f7fe640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b301f4e70 0x7f5b300d2dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c6d0 tx=0x7f5b2800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b27fff640 1 -- 172.21.15.84:0/3944482383 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800a840 con 0x7f5b301f4e70 2023-10-11T06:12:17.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b27fff640 1 -- 172.21.15.84:0/3944482383 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2800a9a0 con 0x7f5b301f4e70 2023-10-11T06:12:17.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b27fff640 1 -- 172.21.15.84:0/3944482383 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800ad90 con 0x7f5b301f4e70 2023-10-11T06:12:17.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.025+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b301e6100 con 0x7f5b301f4e70 2023-10-11T06:12:17.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.026+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b301e66c0 con 0x7f5b301f4e70 2023-10-11T06:12:17.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.027+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3004e890 con 0x7f5b301f4e70 2023-10-11T06:12:17.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.027+0000 7f5b27fff640 1 -- 172.21.15.84:0/3944482383 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5b28012630 con 0x7f5b301f4e70 2023-10-11T06:12:17.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.028+0000 7f5b27fff640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b1406a860 0x7f5b1406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.028+0000 7f5b27fff640 1 -- 172.21.15.84:0/3944482383 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5b28089760 con 0x7f5b301f4e70 2023-10-11T06:12:17.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.029+0000 7f5b2effd640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b1406a860 0x7f5b1406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.029+0000 7f5b2effd640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b1406a860 0x7f5b1406cd10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b20036000 tx=0x7f5b20008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.033+0000 7f5b27fff640 1 -- 172.21.15.84:0/3944482383 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5b28054990 con 0x7f5b301f4e70 2023-10-11T06:12:17.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.225+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b30152f40 con 0x7f5b301f4e70 2023-10-11T06:12:17.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.228+0000 7f5b27fff640 1 -- 172.21.15.84:0/3944482383 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f5b28057fb0 con 0x7f5b301f4e70 2023-10-11T06:12:17.230 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:17.230 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:17.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b1406a860 msgr2=0x7f5b1406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b1406a860 0x7f5b1406cd10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b20036000 tx=0x7f5b20008030 comp rx=0 tx=0).stop 2023-10-11T06:12:17.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b301f4e70 msgr2=0x7f5b300d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b301f4e70 0x7f5b300d2dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c6d0 tx=0x7f5b2800cb90 comp rx=0 tx=0).stop 2023-10-11T06:12:17.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 shutdown_connections 2023-10-11T06:12:17.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b1406a860 0x7f5b1406cd10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b301f4e70 0x7f5b300d2dd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b300d0b00 0x7f5b300d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 --2- 172.21.15.84:0/3944482383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b300afe10 0x7f5b301526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 >> 172.21.15.84:0/3944482383 conn(0x7f5b3009bb10 msgr2=0x7f5b300ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 shutdown_connections 2023-10-11T06:12:17.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.231+0000 7f5b2ffff640 1 -- 172.21.15.84:0/3944482383 wait complete. 2023-10-11T06:12:17.249 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:17.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:17.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.469+0000 7f82daa51640 1 Processor -- start 2023-10-11T06:12:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.469+0000 7f82daa51640 1 -- start start 2023-10-11T06:12:17.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82daa51640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d415a0b0 0x7f82d40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82daa51640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82d40c8d00 0x7f82d40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82daa51640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40cefc0 con 0x7f82d40ceb60 2023-10-11T06:12:17.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82daa51640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d406e0f0 con 0x7f82d40c8d00 2023-10-11T06:12:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82daa51640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40cbae0 con 0x7f82d415a0b0 2023-10-11T06:12:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82cbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82d40c8d00 0x7f82d40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82cbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82d40c8d00 0x7f82d40ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53342/0 (socket says 172.21.15.84:53342) 2023-10-11T06:12:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.471+0000 7f82cbfff640 1 -- 172.21.15.84:0/3380711949 learned_addr learned my addr 172.21.15.84:0/3380711949 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.472+0000 7f82d3fff640 1 --2- 172.21.15.84:0/3380711949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d415a0b0 0x7f82d40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.472+0000 7f82cbfff640 1 -- 172.21.15.84:0/3380711949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d415a0b0 msgr2=0x7f82d40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.472+0000 7f82cbfff640 1 --2- 172.21.15.84:0/3380711949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d415a0b0 0x7f82d40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.472+0000 7f82cbfff640 1 -- 172.21.15.84:0/3380711949 >> v1:172.21.15.99:6789/0 conn(0x7f82d40ceb60 legacy=0x7f82d406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.472+0000 7f82cbfff640 1 -- 172.21.15.84:0/3380711949 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d4071380 con 0x7f82d40c8d00 2023-10-11T06:12:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.472+0000 7f82d3fff640 1 --2- 172.21.15.84:0/3380711949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d415a0b0 0x7f82d40cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:17.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.473+0000 7f82cbfff640 1 --2- 172.21.15.84:0/3380711949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d40c8d00 0x7f82d40ce620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098a0 tx=0x7f82bc02b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b7030ad57640ae server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.473+0000 7f82d37fe640 1 -- 172.21.15.84:0/3380711949 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc033620 con 0x7f82d40c8d00 2023-10-11T06:12:17.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.473+0000 7f82d37fe640 1 -- 172.21.15.84:0/3380711949 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc033780 con 0x7f82d40c8d00 2023-10-11T06:12:17.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.473+0000 7f82d37fe640 1 -- 172.21.15.84:0/3380711949 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc033b10 con 0x7f82d40c8d00 2023-10-11T06:12:17.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 -- 172.21.15.84:0/3380711949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d40c8d00 msgr2=0x7f82d40ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 --2- 172.21.15.84:0/3380711949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d40c8d00 0x7f82d40ce620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098a0 tx=0x7f82bc02b7d0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 -- 172.21.15.84:0/3380711949 shutdown_connections 2023-10-11T06:12:17.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 --2- 172.21.15.84:0/3380711949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d40c8d00 0x7f82d40ce620 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 --2- 172.21.15.84:0/3380711949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d415a0b0 0x7f82d40cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 -- 172.21.15.84:0/3380711949 >> 172.21.15.84:0/3380711949 conn(0x7f82d4069c70 msgr2=0x7f82d406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:17.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 -- 172.21.15.84:0/3380711949 shutdown_connections 2023-10-11T06:12:17.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.474+0000 7f82daa51640 1 -- 172.21.15.84:0/3380711949 wait complete. 2023-10-11T06:12:17.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.476+0000 7f82daa51640 1 Processor -- start 2023-10-11T06:12:17.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.476+0000 7f82daa51640 1 -- start start 2023-10-11T06:12:17.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82daa51640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d40c8d00 0x7f82d40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82daa51640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82d40ceb60 0x7f82d415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82daa51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d415a0b0 0x7f82d414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82daa51640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d414ea50 con 0x7f82d415a0b0 2023-10-11T06:12:17.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82daa51640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d4157730 con 0x7f82d40ceb60 2023-10-11T06:12:17.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82daa51640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d415a980 con 0x7f82d40c8d00 2023-10-11T06:12:17.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82d8fee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d415a0b0 0x7f82d414d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82d8fee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d415a0b0 0x7f82d414d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53348/0 (socket says 172.21.15.84:53348) 2023-10-11T06:12:17.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82d8fee640 1 -- 172.21.15.84:0/1968577902 learned_addr learned my addr 172.21.15.84:0/1968577902 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:17.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.477+0000 7f82d3fff640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d40c8d00 0x7f82d40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82cbfff640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82d40ceb60 0x7f82d415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d8fee640 1 -- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d40c8d00 msgr2=0x7f82d40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d8fee640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d40c8d00 0x7f82d40b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d8fee640 1 -- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82d40ceb60 msgr2=0x7f82d415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d8fee640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82d40ceb60 0x7f82d415ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d8fee640 1 -- 172.21.15.84:0/1968577902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc0094d0 con 0x7f82d415a0b0 2023-10-11T06:12:17.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82cbfff640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82d40ceb60 0x7f82d415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:17.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d8fee640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d415a0b0 0x7f82d414d8e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c780 tx=0x7f82cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d17fa640 1 -- 172.21.15.84:0/1968577902 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc004b60 con 0x7f82d415a0b0 2023-10-11T06:12:17.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d17fa640 1 -- 172.21.15.84:0/1968577902 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc00ce90 con 0x7f82d415a0b0 2023-10-11T06:12:17.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d4159240 con 0x7f82d415a0b0 2023-10-11T06:12:17.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82d17fa640 1 -- 172.21.15.84:0/1968577902 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc00a850 con 0x7f82d415a0b0 2023-10-11T06:12:17.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.478+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d4159800 con 0x7f82d415a0b0 2023-10-11T06:12:17.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.479+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d41578e0 con 0x7f82d415a0b0 2023-10-11T06:12:17.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.480+0000 7f82d17fa640 1 -- 172.21.15.84:0/1968577902 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f82cc00a9b0 con 0x7f82d415a0b0 2023-10-11T06:12:17.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.481+0000 7f82d17fa640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f82b406a830 0x7f82b406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.482+0000 7f82d17fa640 1 -- 172.21.15.84:0/1968577902 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f82cc0892c0 con 0x7f82d415a0b0 2023-10-11T06:12:17.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.482+0000 7f82d3fff640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f82b406a830 0x7f82b406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.482+0000 7f82d3fff640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f82b406a830 0x7f82b406cce0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82c400a950 tx=0x7f82c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.486+0000 7f82d17fa640 1 -- 172.21.15.84:0/1968577902 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f82d41578e0 con 0x7f82d415a0b0 2023-10-11T06:12:17.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.673+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82d41578e0 con 0x7f82d415a0b0 2023-10-11T06:12:17.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.674+0000 7f82d17fa640 1 -- 172.21.15.84:0/1968577902 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82d41578e0 con 0x7f82d415a0b0 2023-10-11T06:12:17.676 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f82b406a830 msgr2=0x7f82b406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f82b406a830 0x7f82b406cce0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82c400a950 tx=0x7f82c4008040 comp rx=0 tx=0).stop 2023-10-11T06:12:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d415a0b0 msgr2=0x7f82d414d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d415a0b0 0x7f82d414d8e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c780 tx=0x7f82cc00cc40 comp rx=0 tx=0).stop 2023-10-11T06:12:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 shutdown_connections 2023-10-11T06:12:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f82b406a830 0x7f82b406cce0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82d415a0b0 0x7f82d414d8e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82d40ceb60 0x7f82d415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 --2- 172.21.15.84:0/1968577902 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82d40c8d00 0x7f82d40b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 >> 172.21.15.84:0/1968577902 conn(0x7f82d4069c70 msgr2=0x7f82d40c35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 shutdown_connections 2023-10-11T06:12:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.677+0000 7f82daa51640 1 -- 172.21.15.84:0/1968577902 wait complete. 2023-10-11T06:12:17.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:17.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.905+0000 7fca6d705640 1 Processor -- start 2023-10-11T06:12:17.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.905+0000 7fca6d705640 1 -- start start 2023-10-11T06:12:17.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca6d705640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca68158e20 0x7fca680cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca6d705640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fca680cc890 0x7fca680cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca6d705640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca680cb8e0 con 0x7fca681591f0 2023-10-11T06:12:17.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca6d705640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6815cd40 con 0x7fca68158e20 2023-10-11T06:12:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca6d705640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca680ce2c0 con 0x7fca680cc890 2023-10-11T06:12:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca67fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca68158e20 0x7fca680cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca67fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fca68158e20 0x7fca680cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53364/0 (socket says 172.21.15.84:53364) 2023-10-11T06:12:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.906+0000 7fca67fff640 1 -- 172.21.15.84:0/2351419165 learned_addr learned my addr 172.21.15.84:0/2351419165 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:17.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.907+0000 7fca67fff640 1 -- 172.21.15.84:0/2351419165 >> v2:172.21.15.99:3300/0 conn(0x7fca680cc890 msgr2=0x7fca680cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.907+0000 7fca67fff640 1 --2- 172.21.15.84:0/2351419165 >> v2:172.21.15.99:3300/0 conn(0x7fca680cc890 0x7fca680cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.907+0000 7fca67fff640 1 -- 172.21.15.84:0/2351419165 >> v1:172.21.15.99:6789/0 conn(0x7fca681591f0 legacy=0x7fca68069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:17.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.907+0000 7fca67fff640 1 -- 172.21.15.84:0/2351419165 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6815cfe0 con 0x7fca68158e20 2023-10-11T06:12:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.908+0000 7fca67fff640 1 --2- 172.21.15.84:0/2351419165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca680cae60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fca680c55b0 tx=0x7fca5802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac52795004109f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.908+0000 7fca677fe640 1 -- 172.21.15.84:0/2351419165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca580317e0 con 0x7fca68158e20 2023-10-11T06:12:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.908+0000 7fca677fe640 1 -- 172.21.15.84:0/2351419165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca58031940 con 0x7fca68158e20 2023-10-11T06:12:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.908+0000 7fca677fe640 1 -- 172.21.15.84:0/2351419165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58031cd0 con 0x7fca68158e20 2023-10-11T06:12:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 -- 172.21.15.84:0/2351419165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 msgr2=0x7fca680cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 --2- 172.21.15.84:0/2351419165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca680cae60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fca680c55b0 tx=0x7fca5802b880 comp rx=0 tx=0).stop 2023-10-11T06:12:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 -- 172.21.15.84:0/2351419165 shutdown_connections 2023-10-11T06:12:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 --2- 172.21.15.84:0/2351419165 >> v2:172.21.15.99:3300/0 conn(0x7fca680cc890 0x7fca680cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 --2- 172.21.15.84:0/2351419165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca680cae60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 -- 172.21.15.84:0/2351419165 >> 172.21.15.84:0/2351419165 conn(0x7fca6806bfb0 msgr2=0x7fca6806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:17.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 -- 172.21.15.84:0/2351419165 shutdown_connections 2023-10-11T06:12:17.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.909+0000 7fca6d705640 1 -- 172.21.15.84:0/2351419165 wait complete. 2023-10-11T06:12:17.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.911+0000 7fca6d705640 1 Processor -- start 2023-10-11T06:12:17.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.911+0000 7fca6d705640 1 -- start start 2023-10-11T06:12:17.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca6d705640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fca680cc890 0x7fca681f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca6d705640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca6814cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca6d705640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca681591f0 0x7fca6814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca6d705640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca680c26c0 con 0x7fca68158e20 2023-10-11T06:12:17.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca6d705640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6814d010 con 0x7fca681591f0 2023-10-11T06:12:17.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca6d705640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca68150590 con 0x7fca680cc890 2023-10-11T06:12:17.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca5ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca6814cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca5ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca6814cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53374/0 (socket says 172.21.15.84:53374) 2023-10-11T06:12:17.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca5ffff640 1 -- 172.21.15.84:0/3212987178 learned_addr learned my addr 172.21.15.84:0/3212987178 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:17.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca67fff640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fca680cc890 0x7fca681f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.912+0000 7fca6cf04640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca681591f0 0x7fca6814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca5ffff640 1 -- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fca680cc890 msgr2=0x7fca681f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca5ffff640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fca680cc890 0x7fca681f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca5ffff640 1 -- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca681591f0 msgr2=0x7fca6814f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:17.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca5ffff640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca681591f0 0x7fca6814f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:17.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca5ffff640 1 -- 172.21.15.84:0/3212987178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580094d0 con 0x7fca68158e20 2023-10-11T06:12:17.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca6cf04640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca681591f0 0x7fca6814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:17.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca5ffff640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca6814cad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca5000b4d0 tx=0x7fca5000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca657fa640 1 -- 172.21.15.84:0/3212987178 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50010870 con 0x7fca68158e20 2023-10-11T06:12:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca657fa640 1 -- 172.21.15.84:0/3212987178 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca500109d0 con 0x7fca68158e20 2023-10-11T06:12:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca657fa640 1 -- 172.21.15.84:0/3212987178 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50010d80 con 0x7fca68158e20 2023-10-11T06:12:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6814fa50 con 0x7fca68158e20 2023-10-11T06:12:17.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.913+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6814ffd0 con 0x7fca68158e20 2023-10-11T06:12:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.914+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca680c2800 con 0x7fca68158e20 2023-10-11T06:12:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.915+0000 7fca657fa640 1 -- 172.21.15.84:0/3212987178 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fca5000f3c0 con 0x7fca68158e20 2023-10-11T06:12:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.916+0000 7fca657fa640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fca4806a860 0x7fca4806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.916+0000 7fca657fa640 1 -- 172.21.15.84:0/3212987178 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fca50088f90 con 0x7fca68158e20 2023-10-11T06:12:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.917+0000 7fca67fff640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fca4806a860 0x7fca4806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:17.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.917+0000 7fca67fff640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fca4806a860 0x7fca4806cd10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca58036000 tx=0x7fca5803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:17.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:17.920+0000 7fca657fa640 1 -- 172.21.15.84:0/3212987178 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fca680c2800 con 0x7fca68158e20 2023-10-11T06:12:18.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.114+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fca680c2800 con 0x7fca68158e20 2023-10-11T06:12:18.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.115+0000 7fca657fa640 1 -- 172.21.15.84:0/3212987178 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fca5001e020 con 0x7fca68158e20 2023-10-11T06:12:18.117 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:18.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fca4806a860 msgr2=0x7fca4806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fca4806a860 0x7fca4806cd10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca58036000 tx=0x7fca5803d040 comp rx=0 tx=0).stop 2023-10-11T06:12:18.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 msgr2=0x7fca6814cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca6814cad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca5000b4d0 tx=0x7fca5000b990 comp rx=0 tx=0).stop 2023-10-11T06:12:18.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 shutdown_connections 2023-10-11T06:12:18.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fca4806a860 0x7fca4806cd10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca681591f0 0x7fca6814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca68158e20 0x7fca6814cad0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 --2- 172.21.15.84:0/3212987178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fca680cc890 0x7fca681f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 >> 172.21.15.84:0/3212987178 conn(0x7fca6806bfb0 msgr2=0x7fca68070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:18.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 shutdown_connections 2023-10-11T06:12:18.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.118+0000 7fca6d705640 1 -- 172.21.15.84:0/3212987178 wait complete. 2023-10-11T06:12:18.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:18.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.351+0000 7ff0df49d640 1 Processor -- start 2023-10-11T06:12:18.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.352+0000 7ff0df49d640 1 -- start start 2023-10-11T06:12:18.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0df49d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff0d815f7e0 0x7ff0d80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0df49d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0d80d31d0 0x7ff0d80d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0df49d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d80cf220 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0df49d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d80cf500 con 0x7ff0d815f7e0 2023-10-11T06:12:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0df49d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d80cf7e0 con 0x7ff0d80d31d0 2023-10-11T06:12:18.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0dd239640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff0d815f7e0 0x7ff0d80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0dd239640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff0d815f7e0 0x7ff0d80cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53388/0 (socket says 172.21.15.84:53388) 2023-10-11T06:12:18.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.353+0000 7ff0dd239640 1 -- 172.21.15.84:0/1824856511 learned_addr learned my addr 172.21.15.84:0/1824856511 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:18.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.354+0000 7ff0dda3a640 1 --2- 172.21.15.84:0/1824856511 >> v2:172.21.15.99:3300/0 conn(0x7ff0d80d31d0 0x7ff0d80d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.354+0000 7ff0dd239640 1 -- 172.21.15.84:0/1824856511 >> v2:172.21.15.99:3300/0 conn(0x7ff0d80d31d0 msgr2=0x7ff0d80d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:18.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.354+0000 7ff0dd239640 1 --2- 172.21.15.84:0/1824856511 >> v2:172.21.15.99:3300/0 conn(0x7ff0d80d31d0 0x7ff0d80d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.354+0000 7ff0dd239640 1 -- 172.21.15.84:0/1824856511 >> v1:172.21.15.99:6789/0 conn(0x7ff0d80cd6c0 legacy=0x7ff0d80d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.354+0000 7ff0dd239640 1 -- 172.21.15.84:0/1824856511 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d80cfac0 con 0x7ff0d815f7e0 2023-10-11T06:12:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.355+0000 7ff0dd239640 1 --2- 172.21.15.84:0/1824856511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d815f7e0 0x7ff0d80cd180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80b2310 tx=0x7ff0c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f745c4402e1390 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.355+0000 7ff0d7fff640 1 -- 172.21.15.84:0/1824856511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c802ca80 con 0x7ff0d815f7e0 2023-10-11T06:12:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0d7fff640 1 -- 172.21.15.84:0/1824856511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c802cbe0 con 0x7ff0d815f7e0 2023-10-11T06:12:18.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0d7fff640 1 -- 172.21.15.84:0/1824856511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80343f0 con 0x7ff0d815f7e0 2023-10-11T06:12:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 -- 172.21.15.84:0/1824856511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d815f7e0 msgr2=0x7ff0d80cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 --2- 172.21.15.84:0/1824856511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d815f7e0 0x7ff0d80cd180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80b2310 tx=0x7ff0c802bee0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 -- 172.21.15.84:0/1824856511 shutdown_connections 2023-10-11T06:12:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 --2- 172.21.15.84:0/1824856511 >> v2:172.21.15.99:3300/0 conn(0x7ff0d80d31d0 0x7ff0d80d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 --2- 172.21.15.84:0/1824856511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d815f7e0 0x7ff0d80cd180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 -- 172.21.15.84:0/1824856511 >> 172.21.15.84:0/1824856511 conn(0x7ff0d80713a0 msgr2=0x7ff0d80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 -- 172.21.15.84:0/1824856511 shutdown_connections 2023-10-11T06:12:18.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.356+0000 7ff0df49d640 1 -- 172.21.15.84:0/1824856511 wait complete. 2023-10-11T06:12:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.358+0000 7ff0df49d640 1 Processor -- start 2023-10-11T06:12:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.358+0000 7ff0df49d640 1 -- start start 2023-10-11T06:12:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0df49d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d80cd6c0 0x7ff0d80b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0df49d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0d80d31d0 0x7ff0d80b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0df49d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d815f7e0 0x7ff0d815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0df49d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80b6010 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0df49d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80b5a00 con 0x7ff0d815f7e0 2023-10-11T06:12:18.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0df49d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d815e180 con 0x7ff0d80d31d0 2023-10-11T06:12:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0dd239640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d80cd6c0 0x7ff0d80b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0dd239640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d80cd6c0 0x7ff0d80b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53392/0 (socket says 172.21.15.84:53392) 2023-10-11T06:12:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.359+0000 7ff0dd239640 1 -- 172.21.15.84:0/2090672665 learned_addr learned my addr 172.21.15.84:0/2090672665 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.360+0000 7ff0dd239640 1 -- 172.21.15.84:0/2090672665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0d80d31d0 msgr2=0x7ff0d80b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.360+0000 7ff0dd239640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0d80d31d0 0x7ff0d80b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.360+0000 7ff0dd239640 1 -- 172.21.15.84:0/2090672665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d815f7e0 msgr2=0x7ff0d815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.360+0000 7ff0dd239640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d815f7e0 0x7ff0d815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.360+0000 7ff0dd239640 1 -- 172.21.15.84:0/2090672665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8009b30 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.360+0000 7ff0dd239640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d80cd6c0 0x7ff0d80b8370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8000f80 tx=0x7ff0c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.360+0000 7ff0d5ffb640 1 -- 172.21.15.84:0/2090672665 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c803e450 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.361+0000 7ff0d5ffb640 1 -- 172.21.15.84:0/2090672665 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c803e5f0 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.361+0000 7ff0d5ffb640 1 -- 172.21.15.84:0/2090672665 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c803e9a0 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.361+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d80b5ca0 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.361+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d80b69e0 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.362+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d80b6ba0 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.363+0000 7ff0d5ffb640 1 -- 172.21.15.84:0/2090672665 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff0c802cd50 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.364+0000 7ff0d5ffb640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff0c406a830 0x7ff0c406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.364+0000 7ff0d5ffb640 1 -- 172.21.15.84:0/2090672665 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff0c80abc40 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.364+0000 7ff0dca38640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff0c406a830 0x7ff0c406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.365+0000 7ff0dca38640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff0c406a830 0x7ff0c406cce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc006fd0 tx=0x7ff0cc006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:18.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.368+0000 7ff0d5ffb640 1 -- 172.21.15.84:0/2090672665 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff0c8076e80 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.556+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0d80b6ba0 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.557+0000 7ff0d5ffb640 1 -- 172.21.15.84:0/2090672665 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0c807a4a0 con 0x7ff0d80cd6c0 2023-10-11T06:12:18.558 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:18.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff0c406a830 msgr2=0x7ff0c406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff0c406a830 0x7ff0c406cce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc006fd0 tx=0x7ff0cc006bc0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d80cd6c0 msgr2=0x7ff0d80b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d80cd6c0 0x7ff0d80b8370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8000f80 tx=0x7ff0c8004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:18.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 shutdown_connections 2023-10-11T06:12:18.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff0c406a830 0x7ff0c406cce0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d815f7e0 0x7ff0d815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0d80d31d0 0x7ff0d80b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 --2- 172.21.15.84:0/2090672665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0d80cd6c0 0x7ff0d80b8370 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 >> 172.21.15.84:0/2090672665 conn(0x7ff0d80713a0 msgr2=0x7ff0d80b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:18.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 shutdown_connections 2023-10-11T06:12:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.560+0000 7ff0df49d640 1 -- 172.21.15.84:0/2090672665 wait complete. 2023-10-11T06:12:18.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:18.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.796+0000 7f2919685640 1 Processor -- start 2023-10-11T06:12:18.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.796+0000 7f2919685640 1 -- start start 2023-10-11T06:12:18.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2919685640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2919685640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29140b06e0 0x7f291415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2919685640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29140cca10 con 0x7f2914158980 2023-10-11T06:12:18.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2919685640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291415d330 con 0x7f29140b06e0 2023-10-11T06:12:18.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2919685640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29140c92a0 con 0x7f29141585b0 2023-10-11T06:12:18.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2918e84640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29140b06e0 0x7f291415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2918e84640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29140b06e0 0x7f291415cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53402/0 (socket says 172.21.15.84:53402) 2023-10-11T06:12:18.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2918e84640 1 -- 172.21.15.84:0/4034659183 learned_addr learned my addr 172.21.15.84:0/4034659183 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:18.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.798+0000 7f2913fff640 1 --2- 172.21.15.84:0/4034659183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.799+0000 7f2918e84640 1 -- 172.21.15.84:0/4034659183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 msgr2=0x7f29140c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:18.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.799+0000 7f2918e84640 1 --2- 172.21.15.84:0/4034659183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.799+0000 7f2918e84640 1 -- 172.21.15.84:0/4034659183 >> v1:172.21.15.99:6789/0 conn(0x7f2914158980 legacy=0x7f29140affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:18.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.799+0000 7f2918e84640 1 -- 172.21.15.84:0/4034659183 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29140c9580 con 0x7f29140b06e0 2023-10-11T06:12:18.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.800+0000 7f2918e84640 1 --2- 172.21.15.84:0/4034659183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291415cdf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f290c009e50 tx=0x7f290c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19aa797ca772711d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:18.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.800+0000 7f29137fe640 1 -- 172.21.15.84:0/4034659183 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c02ca70 con 0x7f29140b06e0 2023-10-11T06:12:18.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.800+0000 7f29137fe640 1 -- 172.21.15.84:0/4034659183 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c02cbd0 con 0x7f29140b06e0 2023-10-11T06:12:18.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.800+0000 7f29137fe640 1 -- 172.21.15.84:0/4034659183 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0343f0 con 0x7f29140b06e0 2023-10-11T06:12:18.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 -- 172.21.15.84:0/4034659183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 msgr2=0x7f291415cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 --2- 172.21.15.84:0/4034659183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291415cdf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f290c009e50 tx=0x7f290c02bed0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 -- 172.21.15.84:0/4034659183 shutdown_connections 2023-10-11T06:12:18.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 --2- 172.21.15.84:0/4034659183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291415cdf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 --2- 172.21.15.84:0/4034659183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 -- 172.21.15.84:0/4034659183 >> 172.21.15.84:0/4034659183 conn(0x7f291406bfb0 msgr2=0x7f291406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:18.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 -- 172.21.15.84:0/4034659183 shutdown_connections 2023-10-11T06:12:18.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.801+0000 7f2919685640 1 -- 172.21.15.84:0/4034659183 wait complete. 2023-10-11T06:12:18.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.803+0000 7f2919685640 1 Processor -- start 2023-10-11T06:12:18.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.803+0000 7f2919685640 1 -- start start 2023-10-11T06:12:18.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2919685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2919685640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2919685640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2914158980 0x7f2914070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2919685640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29140b3680 con 0x7f29140b06e0 2023-10-11T06:12:18.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2919685640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141647f0 con 0x7f2914158980 2023-10-11T06:12:18.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2919685640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29140b38b0 con 0x7f29141585b0 2023-10-11T06:12:18.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2913fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2913fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53414/0 (socket says 172.21.15.84:53414) 2023-10-11T06:12:18.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2913fff640 1 -- 172.21.15.84:0/4022284011 learned_addr learned my addr 172.21.15.84:0/4022284011 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:18.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f290b7fe640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.804+0000 7f2918e84640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2914158980 0x7f2914070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2913fff640 1 -- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 msgr2=0x7f29140708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2913fff640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2913fff640 1 -- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2914158980 msgr2=0x7f2914070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:18.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2913fff640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2914158980 0x7f2914070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:18.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2913fff640 1 -- 172.21.15.84:0/4022284011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c009b30 con 0x7f29140b06e0 2023-10-11T06:12:18.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2918e84640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2914158980 0x7f2914070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:18.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2913fff640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291414fc90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f290400d800 tx=0x7f290400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:18.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f29117fa640 1 -- 172.21.15.84:0/4022284011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400b6c0 con 0x7f29140b06e0 2023-10-11T06:12:18.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f29117fa640 1 -- 172.21.15.84:0/4022284011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290400b820 con 0x7f29140b06e0 2023-10-11T06:12:18.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f29117fa640 1 -- 172.21.15.84:0/4022284011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400bbb0 con 0x7f29140b06e0 2023-10-11T06:12:18.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29140b3b90 con 0x7f29140b06e0 2023-10-11T06:12:18.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.805+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29140b4150 con 0x7f29140b06e0 2023-10-11T06:12:18.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.807+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291404e890 con 0x7f29140b06e0 2023-10-11T06:12:18.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.807+0000 7f29117fa640 1 -- 172.21.15.84:0/4022284011 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f290400bd10 con 0x7f29140b06e0 2023-10-11T06:12:18.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.809+0000 7f29117fa640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f28f406a830 0x7f28f406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:18.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.809+0000 7f29117fa640 1 -- 172.21.15.84:0/4022284011 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2904089530 con 0x7f29140b06e0 2023-10-11T06:12:18.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.809+0000 7f290b7fe640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f28f406a830 0x7f28f406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:18.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.810+0000 7f290b7fe640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f28f406a830 0x7f28f406cce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28fc006fd0 tx=0x7f28fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:18.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:18.813+0000 7f29117fa640 1 -- 172.21.15.84:0/4022284011 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2904054760 con 0x7f29140b06e0 2023-10-11T06:12:19.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.002+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29140b44d0 con 0x7f29140b06e0 2023-10-11T06:12:19.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.002+0000 7f29117fa640 1 -- 172.21.15.84:0/4022284011 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2904057d80 con 0x7f29140b06e0 2023-10-11T06:12:19.004 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:19.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f28f406a830 msgr2=0x7f28f406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f28f406a830 0x7f28f406cce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28fc006fd0 tx=0x7f28fc008040 comp rx=0 tx=0).stop 2023-10-11T06:12:19.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 msgr2=0x7f291414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291414fc90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f290400d800 tx=0x7f290400dcc0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 shutdown_connections 2023-10-11T06:12:19.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f28f406a830 0x7f28f406cce0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2914158980 0x7f2914070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29141585b0 0x7f29140708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 --2- 172.21.15.84:0/4022284011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29140b06e0 0x7f291414fc90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 >> 172.21.15.84:0/4022284011 conn(0x7f291406bfb0 msgr2=0x7f29140ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:19.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 shutdown_connections 2023-10-11T06:12:19.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.005+0000 7f2919685640 1 -- 172.21.15.84:0/4022284011 wait complete. 2023-10-11T06:12:19.023 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.234+0000 7f8cb43fd640 1 Processor -- start 2023-10-11T06:12:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.234+0000 7f8cb43fd640 1 -- start start 2023-10-11T06:12:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.235+0000 7f8cb43fd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 0x7f8cac0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.235+0000 7f8cb43fd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac1593f0 con 0x7f8cac0ce0d0 2023-10-11T06:12:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.235+0000 7f8cb43fd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac1596c0 con 0x7f8cac0cd300 2023-10-11T06:12:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.235+0000 7f8cb43fd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac15a340 con 0x7f8cac0adb90 2023-10-11T06:12:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb2199640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8cac0ce0d0 0x7f8cac0c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37106/0 (socket says 172.21.15.84:37106) 2023-10-11T06:12:19.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb2199640 1 -- 172.21.15.84:0/934662661 learned_addr learned my addr 172.21.15.84:0/934662661 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:19.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1998640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 0x7f8cac0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836694691 0 0) 0x7f8cac1593f0 con 0x7f8cac0ce0d0 2023-10-11T06:12:19.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c98003520 con 0x7f8cac0ce0d0 2023-10-11T06:12:19.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 577789839 0 0) 0x7f8c98003520 con 0x7f8cac0ce0d0 2023-10-11T06:12:19.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 msgr2=0x7f8cac0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:19.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 0x7f8cac0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 >> v1:172.21.15.99:6789/0 conn(0x7f8cac0cd300 legacy=0x7f8cac15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac1599a0 con 0x7f8cac0ce0d0 2023-10-11T06:12:19.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.236+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f8ca80017f0 con 0x7f8cac0ce0d0 2023-10-11T06:12:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 >> v1:172.21.15.84:6789/0 conn(0x7f8cac0ce0d0 legacy=0x7f8cac0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1197640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c980039a0 0x7f8c98003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1197640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 0x7f8c980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1197640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8c98005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac15a340 con 0x7f8c98004d00 2023-10-11T06:12:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9800ba60 con 0x7f8c980039a0 2023-10-11T06:12:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9800de30 con 0x7f8cac0adb90 2023-10-11T06:12:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb299a640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8c98005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb2199640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c980039a0 0x7f8c98003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb1998640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 0x7f8c980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb299a640 1 -- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 msgr2=0x7f8c980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb299a640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0adb90 0x7f8c980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb299a640 1 -- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c980039a0 msgr2=0x7f8c98003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.237+0000 7f8cb299a640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c980039a0 0x7f8c98003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.238+0000 7f8cb299a640 1 -- 172.21.15.84:0/934662661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98003520 con 0x7f8c98004d00 2023-10-11T06:12:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb299a640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8c98005170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4001e50 tx=0x7f8c980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0957a25c8ba276e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4003b70 con 0x7f8c98004d00 2023-10-11T06:12:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4003cd0 con 0x7f8c98004d00 2023-10-11T06:12:19.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb1197640 1 -- 172.21.15.84:0/934662661 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4005310 con 0x7f8c98004d00 2023-10-11T06:12:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb43fd640 1 -- 172.21.15.84:0/934662661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 msgr2=0x7f8c98005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb43fd640 1 --2- 172.21.15.84:0/934662661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8c98005170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4001e50 tx=0x7f8c980057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb299a640 1 -- 172.21.15.84:0/934662661 reap_dead start 2023-10-11T06:12:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb43fd640 1 -- 172.21.15.84:0/934662661 shutdown_connections 2023-10-11T06:12:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.239+0000 7f8cb43fd640 1 -- 172.21.15.84:0/934662661 >> 172.21.15.84:0/934662661 conn(0x7f8cac069c70 msgr2=0x7f8cac06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:19.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.240+0000 7f8cb43fd640 1 -- 172.21.15.84:0/934662661 shutdown_connections 2023-10-11T06:12:19.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.240+0000 7f8cb43fd640 1 -- 172.21.15.84:0/934662661 wait complete. 2023-10-11T06:12:19.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.241+0000 7f8cb43fd640 1 Processor -- start 2023-10-11T06:12:19.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.241+0000 7f8cb43fd640 1 -- start start 2023-10-11T06:12:19.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.242+0000 7f8cb43fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8cac0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.242+0000 7f8cb43fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cac151460 0x7f8cac0afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.242+0000 7f8cb43fd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0ae060 0x7f8cac0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.242+0000 7f8cb43fd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0b0290 con 0x7f8c98004d00 2023-10-11T06:12:19.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.242+0000 7f8cb43fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0abfd0 con 0x7f8cac151460 2023-10-11T06:12:19.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.242+0000 7f8cb43fd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac151bb0 con 0x7f8cac0ae060 2023-10-11T06:12:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8cac0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8cac0b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50596/0 (socket says 172.21.15.84:50596) 2023-10-11T06:12:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 -- 172.21.15.84:0/709841071 learned_addr learned my addr 172.21.15.84:0/709841071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb299a640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0ae060 0x7f8cac0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb1998640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cac151460 0x7f8cac0afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 -- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0ae060 msgr2=0x7f8cac0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0ae060 0x7f8cac0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 -- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cac151460 msgr2=0x7f8cac0afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cac151460 0x7f8cac0afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 -- 172.21.15.84:0/709841071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac1518d0 con 0x7f8c98004d00 2023-10-11T06:12:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.243+0000 7f8cb2199640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8cac0b29e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e7b0 tx=0x7f8ca800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.244+0000 7f8ca2ffd640 1 -- 172.21.15.84:0/709841071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800ee60 con 0x7f8c98004d00 2023-10-11T06:12:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.244+0000 7f8ca2ffd640 1 -- 172.21.15.84:0/709841071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca80072e0 con 0x7f8c98004d00 2023-10-11T06:12:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.244+0000 7f8ca2ffd640 1 -- 172.21.15.84:0/709841071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80057a0 con 0x7f8c98004d00 2023-10-11T06:12:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.244+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac0ab980 con 0x7f8c98004d00 2023-10-11T06:12:19.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.244+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0b0c50 con 0x7f8c98004d00 2023-10-11T06:12:19.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.245+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac0b0de0 con 0x7f8c98004d00 2023-10-11T06:12:19.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.246+0000 7f8ca2ffd640 1 -- 172.21.15.84:0/709841071 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8ca8005900 con 0x7f8c98004d00 2023-10-11T06:12:19.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.247+0000 7f8ca2ffd640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8c9006a830 0x7f8c9006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.247+0000 7f8ca2ffd640 1 -- 172.21.15.84:0/709841071 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8ca80890f0 con 0x7f8c98004d00 2023-10-11T06:12:19.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.247+0000 7f8cb1998640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8c9006a830 0x7f8c9006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.248+0000 7f8cb1998640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8c9006a830 0x7f8c9006cce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c8bb0 tx=0x7f8c9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:19.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.251+0000 7f8ca2ffd640 1 -- 172.21.15.84:0/709841071 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8ca80542a0 con 0x7f8c98004d00 2023-10-11T06:12:19.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.451+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8cac073ed0 con 0x7f8c98004d00 2023-10-11T06:12:19.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.452+0000 7f8ca2ffd640 1 -- 172.21.15.84:0/709841071 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8ca80578c0 con 0x7f8c98004d00 2023-10-11T06:12:19.454 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:19.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8c9006a830 msgr2=0x7f8c9006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8c9006a830 0x7f8c9006cce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c8bb0 tx=0x7f8c9c008040 comp rx=0 tx=0).stop 2023-10-11T06:12:19.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 msgr2=0x7f8cac0b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8cac0b29e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800e7b0 tx=0x7f8ca800ec70 comp rx=0 tx=0).stop 2023-10-11T06:12:19.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 shutdown_connections 2023-10-11T06:12:19.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8c9006a830 0x7f8c9006cce0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cac0ae060 0x7f8cac0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cac151460 0x7f8cac0afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 --2- 172.21.15.84:0/709841071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c98004d00 0x7f8cac0b29e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 >> 172.21.15.84:0/709841071 conn(0x7f8cac069c70 msgr2=0x7f8cac0c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:19.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 shutdown_connections 2023-10-11T06:12:19.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.455+0000 7f8cb43fd640 1 -- 172.21.15.84:0/709841071 wait complete. 2023-10-11T06:12:19.475 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:12:19.476 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:19.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:19.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.758+0000 7fafe6b8b640 1 Processor -- start 2023-10-11T06:12:19.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.758+0000 7fafe6b8b640 1 -- start start 2023-10-11T06:12:19.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.759+0000 7fafe6b8b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafe00ce0d0 0x7fafe00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafe6b8b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe01593f0 con 0x7fafe00adb90 2023-10-11T06:12:19.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafe6b8b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe01596c0 con 0x7fafe00cd300 2023-10-11T06:12:19.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafe6b8b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe015a340 con 0x7fafe00ce0d0 2023-10-11T06:12:19.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafdffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fafe00adb90 0x7fafe00caee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37108/0 (socket says 172.21.15.84:37108) 2023-10-11T06:12:19.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafdffff640 1 -- 172.21.15.84:0/1525024508 learned_addr learned my addr 172.21.15.84:0/1525024508 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:19.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafe4927640 1 --2- 172.21.15.84:0/1525024508 >> v2:172.21.15.99:3300/0 conn(0x7fafe00ce0d0 0x7fafe00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465340826 0 0) 0x7fafe01593f0 con 0x7fafe00adb90 2023-10-11T06:12:19.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.760+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc8003520 con 0x7fafe00adb90 2023-10-11T06:12:19.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3267701682 0 0) 0x7fafc8003520 con 0x7fafe00adb90 2023-10-11T06:12:19.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 >> v2:172.21.15.99:3300/0 conn(0x7fafe00ce0d0 msgr2=0x7fafe00c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:19.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 --2- 172.21.15.84:0/1525024508 >> v2:172.21.15.99:3300/0 conn(0x7fafe00ce0d0 0x7fafe00c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 >> v1:172.21.15.99:6789/0 conn(0x7fafe00cd300 legacy=0x7fafe015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe01599a0 con 0x7fafe00adb90 2023-10-11T06:12:19.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fafd4001670 con 0x7fafe00adb90 2023-10-11T06:12:19.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 >> v1:172.21.15.84:6789/0 conn(0x7fafe00adb90 legacy=0x7fafe00caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc8003970 0x7fafc8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc80047c0 0x7fafc8004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe015a340 con 0x7fafc8005170 2023-10-11T06:12:19.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc800c100 con 0x7fafc80047c0 2023-10-11T06:12:19.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc800c2c0 con 0x7fafc8003970 2023-10-11T06:12:19.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafe5128640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.761+0000 7fafdffff640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc80047c0 0x7fafc8004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.762+0000 7fafe4927640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc8003970 0x7fafc8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.762+0000 7fafe5128640 1 -- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc8003970 msgr2=0x7fafc8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:19.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.762+0000 7fafe5128640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc8003970 0x7fafc8003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.762+0000 7fafe5128640 1 -- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc80047c0 msgr2=0x7fafc8004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:19.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.762+0000 7fafe5128640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc80047c0 0x7fafc8004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.762+0000 7fafe5128640 1 -- 172.21.15.84:0/1525024508 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8003520 con 0x7fafc8005170 2023-10-11T06:12:19.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.762+0000 7fafe5128640 1 -- 172.21.15.84:0/1525024508 reap_dead start 2023-10-11T06:12:19.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.763+0000 7fafe5128640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafc800b980 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fafd8001720 tx=0x7fafc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd81a5969f6f0f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:19.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.763+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80052d0 con 0x7fafc8005170 2023-10-11T06:12:19.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.763+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd802e050 con 0x7fafc8005170 2023-10-11T06:12:19.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.763+0000 7fafdf7fe640 1 -- 172.21.15.84:0/1525024508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd80035f0 con 0x7fafc8005170 2023-10-11T06:12:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.763+0000 7fafe6b8b640 1 -- 172.21.15.84:0/1525024508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 msgr2=0x7fafc800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.763+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafc800b980 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fafd8001720 tx=0x7fafc80057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.764+0000 7fafe6b8b640 1 -- 172.21.15.84:0/1525024508 shutdown_connections 2023-10-11T06:12:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.764+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/1525024508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafc800b980 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.764+0000 7fafe6b8b640 1 -- 172.21.15.84:0/1525024508 >> 172.21.15.84:0/1525024508 conn(0x7fafe0069c70 msgr2=0x7fafe006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.764+0000 7fafe6b8b640 1 -- 172.21.15.84:0/1525024508 shutdown_connections 2023-10-11T06:12:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.764+0000 7fafe6b8b640 1 -- 172.21.15.84:0/1525024508 wait complete. 2023-10-11T06:12:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.765+0000 7fafe6b8b640 1 Processor -- start 2023-10-11T06:12:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.765+0000 7fafe6b8b640 1 -- start start 2023-10-11T06:12:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.766+0000 7fafe6b8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafe01ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.766+0000 7fafe6b8b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafe00c7f70 0x7fafe00b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.766+0000 7fafe6b8b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafe014d1a0 0x7fafe00b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.766+0000 7fafe6b8b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe01ee8c0 con 0x7fafc8005170 2023-10-11T06:12:19.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.766+0000 7fafe6b8b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe00afbd0 con 0x7fafe014d1a0 2023-10-11T06:12:19.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.766+0000 7fafe6b8b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe00b2780 con 0x7fafe00c7f70 2023-10-11T06:12:19.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafe01ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafe01ee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50614/0 (socket says 172.21.15.84:50614) 2023-10-11T06:12:19.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 -- 172.21.15.84:0/834473914 learned_addr learned my addr 172.21.15.84:0/834473914 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:19.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe5128640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafe014d1a0 0x7fafe00b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 -- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafe00c7f70 msgr2=0x7fafe00b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafdffff640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafe00c7f70 0x7fafe00b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafe00c7f70 0x7fafe00b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 -- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafe014d1a0 msgr2=0x7fafe00b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafe014d1a0 0x7fafe00b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 -- 172.21.15.84:0/834473914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe00c84a0 con 0x7fafc8005170 2023-10-11T06:12:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe5128640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafe014d1a0 0x7fafe00b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.767+0000 7fafe4927640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafe01ee380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007f50 tx=0x7fafd000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.768+0000 7fafdd7fa640 1 -- 172.21.15.84:0/834473914 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd00145f0 con 0x7fafc8005170 2023-10-11T06:12:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.768+0000 7fafdd7fa640 1 -- 172.21.15.84:0/834473914 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd0014750 con 0x7fafc8005170 2023-10-11T06:12:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.768+0000 7fafdd7fa640 1 -- 172.21.15.84:0/834473914 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0014ae0 con 0x7fafc8005170 2023-10-11T06:12:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.768+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe00af580 con 0x7fafc8005170 2023-10-11T06:12:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.768+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe00b05a0 con 0x7fafc8005170 2023-10-11T06:12:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.769+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe0073ed0 con 0x7fafc8005170 2023-10-11T06:12:19.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.770+0000 7fafdd7fa640 1 -- 172.21.15.84:0/834473914 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fafd0014c40 con 0x7fafc8005170 2023-10-11T06:12:19.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.771+0000 7fafdd7fa640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafcc06a830 0x7fafcc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:19.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.771+0000 7fafdd7fa640 1 -- 172.21.15.84:0/834473914 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fafd0089dc0 con 0x7fafc8005170 2023-10-11T06:12:19.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.771+0000 7fafdffff640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafcc06a830 0x7fafcc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:19.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.772+0000 7fafdffff640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafcc06a830 0x7fafcc06cce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fafd4001200 tx=0x7fafd4000f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:19.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.775+0000 7fafdd7fa640 1 -- 172.21.15.84:0/834473914 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fafd0054ff0 con 0x7fafc8005170 2023-10-11T06:12:19.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.965+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafe0158430 con 0x7fafc8005170 2023-10-11T06:12:19.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.968+0000 7fafdd7fa640 1 -- 172.21.15.84:0/834473914 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fafd0058610 con 0x7fafc8005170 2023-10-11T06:12:19.970 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:19.970 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:19.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafcc06a830 msgr2=0x7fafcc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafcc06a830 0x7fafcc06cce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fafd4001200 tx=0x7fafd4000f30 comp rx=0 tx=0).stop 2023-10-11T06:12:19.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 msgr2=0x7fafe01ee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:19.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafe01ee380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007f50 tx=0x7fafd000f5e0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 shutdown_connections 2023-10-11T06:12:19.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafcc06a830 0x7fafcc06cce0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafe014d1a0 0x7fafe00b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafe00c7f70 0x7fafe00b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 --2- 172.21.15.84:0/834473914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc8005170 0x7fafe01ee380 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:19.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 >> 172.21.15.84:0/834473914 conn(0x7fafe0069c70 msgr2=0x7fafe00cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:19.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 shutdown_connections 2023-10-11T06:12:19.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:19.971+0000 7fafe6b8b640 1 -- 172.21.15.84:0/834473914 wait complete. 2023-10-11T06:12:19.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:19.991 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:20.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.210+0000 7f4538ab9640 1 Processor -- start 2023-10-11T06:12:20.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.211+0000 7f4538ab9640 1 -- start start 2023-10-11T06:12:20.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.212+0000 7f4538ab9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 0x7f45340cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.212+0000 7f4538ab9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45340cd6c0 0x7f45340d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.212+0000 7f4538ab9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45340d3460 con 0x7f45340d3000 2023-10-11T06:12:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.212+0000 7f4538ab9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340cf220 con 0x7f45340cd6c0 2023-10-11T06:12:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.212+0000 7f4538ab9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340cf500 con 0x7f453415f7e0 2023-10-11T06:12:20.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.212+0000 7f4532d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f45340d3000 0x7f453415fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37122/0 (socket says 172.21.15.84:37122) 2023-10-11T06:12:20.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.212+0000 7f4532d9d640 1 -- 172.21.15.84:0/2262235630 learned_addr learned my addr 172.21.15.84:0/2262235630 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:20.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f452bfff640 1 --2- 172.21.15.84:0/2262235630 >> v2:172.21.15.99:3300/0 conn(0x7f45340cd6c0 0x7f45340d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f453259c640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 0x7f45340cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874487012 0 0) 0x7f45340d3460 con 0x7f45340d3000 2023-10-11T06:12:20.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4520003520 con 0x7f45340d3000 2023-10-11T06:12:20.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3154598741 0 0) 0x7f4520003520 con 0x7f45340d3000 2023-10-11T06:12:20.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 msgr2=0x7f45340cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:20.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 0x7f45340cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 >> v2:172.21.15.99:3300/0 conn(0x7f45340cd6c0 msgr2=0x7f45340d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:20.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 --2- 172.21.15.84:0/2262235630 >> v2:172.21.15.99:3300/0 conn(0x7f45340cd6c0 0x7f45340d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340cf7e0 con 0x7f45340d3000 2023-10-11T06:12:20.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f452c0019f0 con 0x7f45340d3000 2023-10-11T06:12:20.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.213+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 >> v1:172.21.15.84:6789/0 conn(0x7f45340d3000 legacy=0x7f453415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4531d9b640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4520003970 0x7f4520003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4531d9b640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 0x7f45200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4531d9b640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4520004d00 0x7f4520005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340cf220 con 0x7f4520004d00 2023-10-11T06:12:20.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340cf500 con 0x7f4520003970 2023-10-11T06:12:20.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452000b980 con 0x7f453415f7e0 2023-10-11T06:12:20.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4532d9d640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4520004d00 0x7f4520005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f452bfff640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 0x7f45200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4532d9d640 1 -- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 msgr2=0x7f45200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:20.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4532d9d640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f453415f7e0 0x7f45200047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f453259c640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4520003970 0x7f4520003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4532d9d640 1 -- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4520003970 msgr2=0x7f4520003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:20.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4532d9d640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4520003970 0x7f4520003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.214+0000 7f4532d9d640 1 -- 172.21.15.84:0/2262235630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520003520 con 0x7f4520004d00 2023-10-11T06:12:20.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.215+0000 7f4532d9d640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4520004d00 0x7f4520005170 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f452c00a860 tx=0x7f45200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41944eefd970003 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:20.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c034630 con 0x7f4520004d00 2023-10-11T06:12:20.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c02f070 con 0x7f4520004d00 2023-10-11T06:12:20.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4531d9b640 1 -- 172.21.15.84:0/2262235630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c0349f0 con 0x7f4520004d00 2023-10-11T06:12:20.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4538ab9640 1 -- 172.21.15.84:0/2262235630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4520004d00 msgr2=0x7f4520005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4538ab9640 1 --2- 172.21.15.84:0/2262235630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4520004d00 0x7f4520005170 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f452c00a860 tx=0x7f45200057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4532d9d640 1 -- 172.21.15.84:0/2262235630 reap_dead start 2023-10-11T06:12:20.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4538ab9640 1 -- 172.21.15.84:0/2262235630 shutdown_connections 2023-10-11T06:12:20.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.216+0000 7f4538ab9640 1 -- 172.21.15.84:0/2262235630 >> 172.21.15.84:0/2262235630 conn(0x7f45340713a0 msgr2=0x7f45340717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:20.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.217+0000 7f4538ab9640 1 -- 172.21.15.84:0/2262235630 shutdown_connections 2023-10-11T06:12:20.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.217+0000 7f4538ab9640 1 -- 172.21.15.84:0/2262235630 wait complete. 2023-10-11T06:12:20.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.218+0000 7f4538ab9640 1 Processor -- start 2023-10-11T06:12:20.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.218+0000 7f4538ab9640 1 -- start start 2023-10-11T06:12:20.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.219+0000 7f4538ab9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4520004d00 0x7f45340c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.219+0000 7f4538ab9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453415e910 0x7f4534153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.219+0000 7f4538ab9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4534158910 0x7f4534158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.219+0000 7f4538ab9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45341592c0 con 0x7f4534158910 2023-10-11T06:12:20.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.219+0000 7f4538ab9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340c7b60 con 0x7f453415e910 2023-10-11T06:12:20.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.219+0000 7f4538ab9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340c6920 con 0x7f4520004d00 2023-10-11T06:12:20.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4534158910 0x7f4534158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4534158910 0x7f4534158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50642/0 (socket says 172.21.15.84:50642) 2023-10-11T06:12:20.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 -- 172.21.15.84:0/1237000698 learned_addr learned my addr 172.21.15.84:0/1237000698 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:20.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f452bfff640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453415e910 0x7f4534153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f453259c640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4520004d00 0x7f45340c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 -- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4520004d00 msgr2=0x7f45340c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4520004d00 0x7f45340c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 -- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453415e910 msgr2=0x7f4534153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453415e910 0x7f4534153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.220+0000 7f4532d9d640 1 -- 172.21.15.84:0/1237000698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340c6fe0 con 0x7f4534158910 2023-10-11T06:12:20.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.221+0000 7f4532d9d640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4534158910 0x7f4534158d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f452c00a860 tx=0x7f452c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:20.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.221+0000 7f452affd640 1 -- 172.21.15.84:0/1237000698 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c034840 con 0x7f4534158910 2023-10-11T06:12:20.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.221+0000 7f452affd640 1 -- 172.21.15.84:0/1237000698 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c0349a0 con 0x7f4534158910 2023-10-11T06:12:20.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.221+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45340c72c0 con 0x7f4534158910 2023-10-11T06:12:20.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.221+0000 7f452affd640 1 -- 172.21.15.84:0/1237000698 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c034d10 con 0x7f4534158910 2023-10-11T06:12:20.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.221+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45340c7880 con 0x7f4534158910 2023-10-11T06:12:20.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.222+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4534073ed0 con 0x7f4534158910 2023-10-11T06:12:20.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.223+0000 7f452affd640 1 -- 172.21.15.84:0/1237000698 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f452c031030 con 0x7f4534158910 2023-10-11T06:12:20.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.224+0000 7f452affd640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f451406a860 0x7f451406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.225+0000 7f452affd640 1 -- 172.21.15.84:0/1237000698 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f452c0aa940 con 0x7f4534158910 2023-10-11T06:12:20.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.225+0000 7f453259c640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f451406a860 0x7f451406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.225+0000 7f453259c640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f451406a860 0x7f451406cd10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4524009910 tx=0x7f4524008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:20.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.229+0000 7f452affd640 1 -- 172.21.15.84:0/1237000698 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f452c075ae0 con 0x7f4534158910 2023-10-11T06:12:20.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.417+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45340b5030 con 0x7f4534158910 2023-10-11T06:12:20.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.418+0000 7f452affd640 1 -- 172.21.15.84:0/1237000698 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f452c0450a0 con 0x7f4534158910 2023-10-11T06:12:20.419 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f451406a860 msgr2=0x7f451406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f451406a860 0x7f451406cd10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4524009910 tx=0x7f4524008040 comp rx=0 tx=0).stop 2023-10-11T06:12:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4534158910 msgr2=0x7f4534158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4534158910 0x7f4534158d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f452c00a860 tx=0x7f452c001d80 comp rx=0 tx=0).stop 2023-10-11T06:12:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 shutdown_connections 2023-10-11T06:12:20.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f451406a860 0x7f451406cd10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4534158910 0x7f4534158d80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453415e910 0x7f4534153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.420+0000 7f4538ab9640 1 --2- 172.21.15.84:0/1237000698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4520004d00 0x7f45340c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.421+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 >> 172.21.15.84:0/1237000698 conn(0x7f45340713a0 msgr2=0x7f453406ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:20.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.421+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 shutdown_connections 2023-10-11T06:12:20.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.421+0000 7f4538ab9640 1 -- 172.21.15.84:0/1237000698 wait complete. 2023-10-11T06:12:20.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:20.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.657+0000 7f78e8e5f640 1 Processor -- start 2023-10-11T06:12:20.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.657+0000 7f78e8e5f640 1 -- start start 2023-10-11T06:12:20.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e8e5f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78e414ff30 0x7f78e40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e8e5f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f78e40ce510 0x7f78e40ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e8e5f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e40c9b90 con 0x7f78e4150300 2023-10-11T06:12:20.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e8e5f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e415da10 con 0x7f78e414ff30 2023-10-11T06:12:20.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e8e5f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e40cc990 con 0x7f78e40ce510 2023-10-11T06:12:20.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78e414ff30 0x7f78e40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78e414ff30 0x7f78e40c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50658/0 (socket says 172.21.15.84:50658) 2023-10-11T06:12:20.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.659+0000 7f78e259c640 1 -- 172.21.15.84:0/1213831178 learned_addr learned my addr 172.21.15.84:0/1213831178 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:20.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.660+0000 7f78e259c640 1 -- 172.21.15.84:0/1213831178 >> v2:172.21.15.99:3300/0 conn(0x7f78e40ce510 msgr2=0x7f78e40ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:20.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.660+0000 7f78e259c640 1 --2- 172.21.15.84:0/1213831178 >> v2:172.21.15.99:3300/0 conn(0x7f78e40ce510 0x7f78e40ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.660+0000 7f78e259c640 1 -- 172.21.15.84:0/1213831178 >> v1:172.21.15.99:6789/0 conn(0x7f78e4150300 legacy=0x7f78e40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:20.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.660+0000 7f78e259c640 1 -- 172.21.15.84:0/1213831178 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e415ce00 con 0x7f78e414ff30 2023-10-11T06:12:20.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.661+0000 7f78e259c640 1 --2- 172.21.15.84:0/1213831178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e414ff30 0x7f78e40c9650 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f78e406e3f0 tx=0x7f78d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2dcc5a12d2e77d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:20.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.661+0000 7f78e159a640 1 -- 172.21.15.84:0/1213831178 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d40317e0 con 0x7f78e414ff30 2023-10-11T06:12:20.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.661+0000 7f78e159a640 1 -- 172.21.15.84:0/1213831178 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d4031940 con 0x7f78e414ff30 2023-10-11T06:12:20.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.661+0000 7f78e159a640 1 -- 172.21.15.84:0/1213831178 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4031cd0 con 0x7f78e414ff30 2023-10-11T06:12:20.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 -- 172.21.15.84:0/1213831178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e414ff30 msgr2=0x7f78e40c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/1213831178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e414ff30 0x7f78e40c9650 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f78e406e3f0 tx=0x7f78d402b880 comp rx=0 tx=0).stop 2023-10-11T06:12:20.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 -- 172.21.15.84:0/1213831178 shutdown_connections 2023-10-11T06:12:20.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/1213831178 >> v2:172.21.15.99:3300/0 conn(0x7f78e40ce510 0x7f78e40ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/1213831178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e414ff30 0x7f78e40c9650 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 -- 172.21.15.84:0/1213831178 >> 172.21.15.84:0/1213831178 conn(0x7f78e4069c70 msgr2=0x7f78e406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:20.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 -- 172.21.15.84:0/1213831178 shutdown_connections 2023-10-11T06:12:20.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.662+0000 7f78e8e5f640 1 -- 172.21.15.84:0/1213831178 wait complete. 2023-10-11T06:12:20.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.664+0000 7f78e8e5f640 1 Processor -- start 2023-10-11T06:12:20.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.664+0000 7f78e8e5f640 1 -- start start 2023-10-11T06:12:20.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e8e5f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78e40ce510 0x7f78e41477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e8e5f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78e414ff30 0x7f78e414ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e8e5f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e4150300 0x7f78e4140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e8e5f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e414e2f0 con 0x7f78e4150300 2023-10-11T06:12:20.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e8e5f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e4140950 con 0x7f78e414ff30 2023-10-11T06:12:20.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e8e5f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e414ed50 con 0x7f78e40ce510 2023-10-11T06:12:20.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e4150300 0x7f78e4140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e4150300 0x7f78e4140410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50666/0 (socket says 172.21.15.84:50666) 2023-10-11T06:12:20.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.665+0000 7f78e2d9d640 1 -- 172.21.15.84:0/163747268 learned_addr learned my addr 172.21.15.84:0/163747268 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:20.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e259c640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78e40ce510 0x7f78e41477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e2d9d640 1 -- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78e40ce510 msgr2=0x7f78e41477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e2d9d640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78e40ce510 0x7f78e41477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e1d9b640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78e414ff30 0x7f78e414ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e2d9d640 1 -- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78e414ff30 msgr2=0x7f78e414ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e2d9d640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78e414ff30 0x7f78e414ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e2d9d640 1 -- 172.21.15.84:0/163747268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d40094d0 con 0x7f78e4150300 2023-10-11T06:12:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e259c640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78e40ce510 0x7f78e41477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e2d9d640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e4150300 0x7f78e4140410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00b970 tx=0x7f78dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:20.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78d2ffd640 1 -- 172.21.15.84:0/163747268 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00cb40 con 0x7f78e4150300 2023-10-11T06:12:20.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78d2ffd640 1 -- 172.21.15.84:0/163747268 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc00ccc0 con 0x7f78e4150300 2023-10-11T06:12:20.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78d2ffd640 1 -- 172.21.15.84:0/163747268 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc014680 con 0x7f78e4150300 2023-10-11T06:12:20.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.666+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e4167a10 con 0x7f78e4150300 2023-10-11T06:12:20.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.667+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e414e9e0 con 0x7f78e4150300 2023-10-11T06:12:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.668+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e41461d0 con 0x7f78e4150300 2023-10-11T06:12:20.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.668+0000 7f78d2ffd640 1 -- 172.21.15.84:0/163747268 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f78dc01d3b0 con 0x7f78e4150300 2023-10-11T06:12:20.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.669+0000 7f78d2ffd640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78c406a830 0x7f78c406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:20.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.670+0000 7f78d2ffd640 1 -- 172.21.15.84:0/163747268 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f78dc088fe0 con 0x7f78e4150300 2023-10-11T06:12:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.670+0000 7f78e259c640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78c406a830 0x7f78c406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.670+0000 7f78e259c640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78c406a830 0x7f78c406cce0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f78d4036000 tx=0x7f78d403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.674+0000 7f78d2ffd640 1 -- 172.21.15.84:0/163747268 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f78dc054180 con 0x7f78e4150300 2023-10-11T06:12:20.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.865+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f78e406ab90 con 0x7f78e4150300 2023-10-11T06:12:20.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.866+0000 7f78d2ffd640 1 -- 172.21.15.84:0/163747268 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f78dc019020 con 0x7f78e4150300 2023-10-11T06:12:20.867 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:20.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.868+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78c406a830 msgr2=0x7f78c406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.868+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78c406a830 0x7f78c406cce0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f78d4036000 tx=0x7f78d403d040 comp rx=0 tx=0).stop 2023-10-11T06:12:20.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.868+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e4150300 msgr2=0x7f78e4140410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:20.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.868+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e4150300 0x7f78e4140410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00b970 tx=0x7f78dc00be30 comp rx=0 tx=0).stop 2023-10-11T06:12:20.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 shutdown_connections 2023-10-11T06:12:20.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78c406a830 0x7f78c406cce0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78e4150300 0x7f78e4140410 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78e414ff30 0x7f78e414ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 --2- 172.21.15.84:0/163747268 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78e40ce510 0x7f78e41477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:20.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 >> 172.21.15.84:0/163747268 conn(0x7f78e4069c70 msgr2=0x7f78e4148650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:20.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 shutdown_connections 2023-10-11T06:12:20.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:20.869+0000 7f78e8e5f640 1 -- 172.21.15.84:0/163747268 wait complete. 2023-10-11T06:12:20.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.107+0000 7f96eb02d640 1 Processor -- start 2023-10-11T06:12:21.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.107+0000 7f96eb02d640 1 -- start start 2023-10-11T06:12:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.108+0000 7f96eb02d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96e4154570 0x7f96e415b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96eb02d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96e40affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96eb02d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e40ccb60 con 0x7f96e4154940 2023-10-11T06:12:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96eb02d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e415b680 con 0x7f96e4154570 2023-10-11T06:12:21.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96eb02d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e4070420 con 0x7f96e406fee0 2023-10-11T06:12:21.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96e982a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f96e4154940 0x7f96e40cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37126/0 (socket says 172.21.15.84:37126) 2023-10-11T06:12:21.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96e982a640 1 -- 172.21.15.84:0/1938725958 learned_addr learned my addr 172.21.15.84:0/1938725958 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96ea02b640 1 --2- 172.21.15.84:0/1938725958 >> v2:172.21.15.99:3300/0 conn(0x7f96e4154570 0x7f96e415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96ea82c640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96e40affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297295863 0 0) 0x7f96e40ccb60 con 0x7f96e4154940 2023-10-11T06:12:21.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.109+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d0003520 con 0x7f96e4154940 2023-10-11T06:12:21.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2927932371 0 0) 0x7f96d0003520 con 0x7f96e4154940 2023-10-11T06:12:21.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 msgr2=0x7f96e40affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96e40affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 >> v2:172.21.15.99:3300/0 conn(0x7f96e4154570 msgr2=0x7f96e415b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 --2- 172.21.15.84:0/1938725958 >> v2:172.21.15.99:3300/0 conn(0x7f96e4154570 0x7f96e415b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e4071e00 con 0x7f96e4154940 2023-10-11T06:12:21.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f96d4001690 con 0x7f96e4154940 2023-10-11T06:12:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96ea82c640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96e40affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 >> v1:172.21.15.84:6789/0 conn(0x7f96e4154940 legacy=0x7f96e40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0003970 0x7f96d0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d0004d00 0x7f96d0005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e415b680 con 0x7f96d0004d00 2023-10-11T06:12:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e4070420 con 0x7f96d0003970 2023-10-11T06:12:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d000b980 con 0x7f96e406fee0 2023-10-11T06:12:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96ea82c640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d0004d00 0x7f96d0005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.110+0000 7f96ea02b640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0003970 0x7f96d0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.111+0000 7f96e982a640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.111+0000 7f96ea82c640 1 -- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 msgr2=0x7f96d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.111+0000 7f96ea82c640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.111+0000 7f96ea82c640 1 -- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0003970 msgr2=0x7f96d0003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.111+0000 7f96ea82c640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0003970 0x7f96d0003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.111+0000 7f96ea82c640 1 -- 172.21.15.84:0/1938725958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0003520 con 0x7f96d0004d00 2023-10-11T06:12:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.111+0000 7f96e982a640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e406fee0 0x7f96d00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.112+0000 7f96ea82c640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d0004d00 0x7f96d0005170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96dc007410 tx=0x7f96d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d263e7c5f091b39 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.112+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc02cd30 con 0x7f96d0004d00 2023-10-11T06:12:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.112+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc02ce90 con 0x7f96d0004d00 2023-10-11T06:12:21.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.112+0000 7f96e9029640 1 -- 172.21.15.84:0/1938725958 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc034640 con 0x7f96d0004d00 2023-10-11T06:12:21.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.112+0000 7f96eb02d640 1 -- 172.21.15.84:0/1938725958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d0004d00 msgr2=0x7f96d0005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.112+0000 7f96eb02d640 1 --2- 172.21.15.84:0/1938725958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96d0004d00 0x7f96d0005170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96dc007410 tx=0x7f96d00057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.112+0000 7f96ea82c640 1 -- 172.21.15.84:0/1938725958 reap_dead start 2023-10-11T06:12:21.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.113+0000 7f96eb02d640 1 -- 172.21.15.84:0/1938725958 shutdown_connections 2023-10-11T06:12:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.113+0000 7f96eb02d640 1 -- 172.21.15.84:0/1938725958 >> 172.21.15.84:0/1938725958 conn(0x7f96e406bfb0 msgr2=0x7f96e406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.113+0000 7f96eb02d640 1 -- 172.21.15.84:0/1938725958 shutdown_connections 2023-10-11T06:12:21.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.113+0000 7f96eb02d640 1 -- 172.21.15.84:0/1938725958 wait complete. 2023-10-11T06:12:21.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.114+0000 7f96eb02d640 1 Processor -- start 2023-10-11T06:12:21.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.115+0000 7f96eb02d640 1 -- start start 2023-10-11T06:12:21.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96eb02d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0004d00 0x7f96e41e3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96eb02d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e4150c20 0x7f96e40c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96eb02d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e40c2580 0x7f96e40c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96eb02d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e41e3dd0 con 0x7f96e4150c20 2023-10-11T06:12:21.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96eb02d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e41513c0 con 0x7f96d0004d00 2023-10-11T06:12:21.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96eb02d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e41e4010 con 0x7f96e40c2580 2023-10-11T06:12:21.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e4150c20 0x7f96e40c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e4150c20 0x7f96e40c2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50682/0 (socket says 172.21.15.84:50682) 2023-10-11T06:12:21.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 -- 172.21.15.84:0/3358642843 learned_addr learned my addr 172.21.15.84:0/3358642843 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:21.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96ea02b640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0004d00 0x7f96e41e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96ea82c640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e40c2580 0x7f96e40c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 -- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e40c2580 msgr2=0x7f96e40c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e40c2580 0x7f96e40c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 -- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0004d00 msgr2=0x7f96e41e3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0004d00 0x7f96e41e3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.116+0000 7f96e982a640 1 -- 172.21.15.84:0/3358642843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e41e42f0 con 0x7f96e4150c20 2023-10-11T06:12:21.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.117+0000 7f96e982a640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e4150c20 0x7f96e40c2040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f96d400d8e0 tx=0x7f96d400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:21.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.117+0000 7f96daffd640 1 -- 172.21.15.84:0/3358642843 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d40074e0 con 0x7f96e4150c20 2023-10-11T06:12:21.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.117+0000 7f96daffd640 1 -- 172.21.15.84:0/3358642843 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4007640 con 0x7f96e4150c20 2023-10-11T06:12:21.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.117+0000 7f96daffd640 1 -- 172.21.15.84:0/3358642843 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4004a00 con 0x7f96e4150c20 2023-10-11T06:12:21.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.117+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e41e45d0 con 0x7f96e4150c20 2023-10-11T06:12:21.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.117+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e4151090 con 0x7f96e4150c20 2023-10-11T06:12:21.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.118+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e404e890 con 0x7f96e4150c20 2023-10-11T06:12:21.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.119+0000 7f96daffd640 1 -- 172.21.15.84:0/3358642843 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f96d4004b60 con 0x7f96e4150c20 2023-10-11T06:12:21.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.120+0000 7f96daffd640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f96c806a830 0x7f96c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.120+0000 7f96daffd640 1 -- 172.21.15.84:0/3358642843 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f96d4088b30 con 0x7f96e4150c20 2023-10-11T06:12:21.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.120+0000 7f96ea02b640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f96c806a830 0x7f96c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.121+0000 7f96ea02b640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f96c806a830 0x7f96c806cce0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009a10 tx=0x7f96e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:21.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.125+0000 7f96daffd640 1 -- 172.21.15.84:0/3358642843 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f96d4053d80 con 0x7f96e4150c20 2023-10-11T06:12:21.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.311+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96e406a180 con 0x7f96e4150c20 2023-10-11T06:12:21.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.313+0000 7f96daffd640 1 -- 172.21.15.84:0/3358642843 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96d40190a0 con 0x7f96e4150c20 2023-10-11T06:12:21.314 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:21.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f96c806a830 msgr2=0x7f96c806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f96c806a830 0x7f96c806cce0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009a10 tx=0x7f96e0008040 comp rx=0 tx=0).stop 2023-10-11T06:12:21.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e4150c20 msgr2=0x7f96e40c2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e4150c20 0x7f96e40c2040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f96d400d8e0 tx=0x7f96d400dda0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 shutdown_connections 2023-10-11T06:12:21.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f96c806a830 0x7f96c806cce0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96e40c2580 0x7f96e40c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e4150c20 0x7f96e40c2040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 --2- 172.21.15.84:0/3358642843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96d0004d00 0x7f96e41e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.315+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 >> 172.21.15.84:0/3358642843 conn(0x7f96e406bfb0 msgr2=0x7f96e415ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:21.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.316+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 shutdown_connections 2023-10-11T06:12:21.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.316+0000 7f96eb02d640 1 -- 172.21.15.84:0/3358642843 wait complete. 2023-10-11T06:12:21.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:21.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.557+0000 7f0698a10640 1 Processor -- start 2023-10-11T06:12:21.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.557+0000 7f0698a10640 1 -- start start 2023-10-11T06:12:21.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0698a10640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f06940d2ac0 0x7f06940d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0698a10640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06940cf220 con 0x7f069415f7e0 2023-10-11T06:12:21.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0698a10640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06940cf500 con 0x7f06940d0940 2023-10-11T06:12:21.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0698a10640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06940d3480 con 0x7f06940d2ac0 2023-10-11T06:12:21.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f069259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f069415f7e0 0x7f06940cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37138/0 (socket says 172.21.15.84:37138) 2023-10-11T06:12:21.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0689d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f06940d2ac0 0x7f06940d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f069259c640 1 -- 172.21.15.84:0/548990328 learned_addr learned my addr 172.21.15.84:0/548990328 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:21.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0689d9b640 1 -- 172.21.15.84:0/548990328 >> v1:172.21.15.99:6789/0 conn(0x7f06940d0940 legacy=0x7f06940d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:21.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0689d9b640 1 -- 172.21.15.84:0/548990328 >> v1:172.21.15.84:6789/0 conn(0x7f069415f7e0 legacy=0x7f06940cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.559+0000 7f0689d9b640 1 -- 172.21.15.84:0/548990328 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06940cf7e0 con 0x7f06940d2ac0 2023-10-11T06:12:21.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.561+0000 7f0689d9b640 1 --2- 172.21.15.84:0/548990328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06940d2ac0 0x7f06940d2f40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f067c0097a0 tx=0x7f067c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e7a03027f1c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:21.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.561+0000 7f0691d9b640 1 -- 172.21.15.84:0/548990328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c031830 con 0x7f06940d2ac0 2023-10-11T06:12:21.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.561+0000 7f0691d9b640 1 -- 172.21.15.84:0/548990328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c031990 con 0x7f06940d2ac0 2023-10-11T06:12:21.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.561+0000 7f0691d9b640 1 -- 172.21.15.84:0/548990328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c031d20 con 0x7f06940d2ac0 2023-10-11T06:12:21.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.561+0000 7f0698a10640 1 -- 172.21.15.84:0/548990328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06940d2ac0 msgr2=0x7f06940d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.561+0000 7f0698a10640 1 --2- 172.21.15.84:0/548990328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06940d2ac0 0x7f06940d2f40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f067c0097a0 tx=0x7f067c02b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.562+0000 7f0698a10640 1 -- 172.21.15.84:0/548990328 shutdown_connections 2023-10-11T06:12:21.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.562+0000 7f0698a10640 1 --2- 172.21.15.84:0/548990328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06940d2ac0 0x7f06940d2f40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.562+0000 7f0698a10640 1 -- 172.21.15.84:0/548990328 >> 172.21.15.84:0/548990328 conn(0x7f06940713a0 msgr2=0x7f06940717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:21.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.562+0000 7f0698a10640 1 -- 172.21.15.84:0/548990328 shutdown_connections 2023-10-11T06:12:21.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.562+0000 7f0698a10640 1 -- 172.21.15.84:0/548990328 wait complete. 2023-10-11T06:12:21.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.563+0000 7f0698a10640 1 Processor -- start 2023-10-11T06:12:21.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.564+0000 7f0698a10640 1 -- start start 2023-10-11T06:12:21.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.564+0000 7f0698a10640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06940d0940 0x7f06940c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.564+0000 7f0698a10640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06940d2ac0 0x7f0694153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.564+0000 7f0698a10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069415f7e0 0x7f06940ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.564+0000 7f0698a10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069415b100 con 0x7f069415f7e0 2023-10-11T06:12:21.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.564+0000 7f0698a10640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069415ab20 con 0x7f06940d2ac0 2023-10-11T06:12:21.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.564+0000 7f0698a10640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06940cac70 con 0x7f06940d0940 2023-10-11T06:12:21.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069415f7e0 0x7f06940ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069415f7e0 0x7f06940ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50692/0 (socket says 172.21.15.84:50692) 2023-10-11T06:12:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 -- 172.21.15.84:0/2489333691 learned_addr learned my addr 172.21.15.84:0/2489333691 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f069259c640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06940d0940 0x7f06940c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0689d9b640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06940d2ac0 0x7f0694153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 -- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06940d0940 msgr2=0x7f06940c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06940d0940 0x7f06940c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 -- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06940d2ac0 msgr2=0x7f0694153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06940d2ac0 0x7f0694153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f0692d9d640 1 -- 172.21.15.84:0/2489333691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009400 con 0x7f069415f7e0 2023-10-11T06:12:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.565+0000 7f069259c640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06940d0940 0x7f06940c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:21.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.566+0000 7f0692d9d640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069415f7e0 0x7f06940ca730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c7b0 tx=0x7f068c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.566+0000 7f068b7fe640 1 -- 172.21.15.84:0/2489333691 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c004b60 con 0x7f069415f7e0 2023-10-11T06:12:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.566+0000 7f068b7fe640 1 -- 172.21.15.84:0/2489333691 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c004cc0 con 0x7f069415f7e0 2023-10-11T06:12:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.566+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06940caf10 con 0x7f069415f7e0 2023-10-11T06:12:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.566+0000 7f068b7fe640 1 -- 172.21.15.84:0/2489333691 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00aa90 con 0x7f069415f7e0 2023-10-11T06:12:21.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.566+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06940cb1f0 con 0x7f069415f7e0 2023-10-11T06:12:21.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.567+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069415b2b0 con 0x7f069415f7e0 2023-10-11T06:12:21.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.568+0000 7f068b7fe640 1 -- 172.21.15.84:0/2489333691 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f068c00abf0 con 0x7f069415f7e0 2023-10-11T06:12:21.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.569+0000 7f068b7fe640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f067406a830 0x7f067406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:21.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.569+0000 7f068b7fe640 1 -- 172.21.15.84:0/2489333691 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f068c0891a0 con 0x7f069415f7e0 2023-10-11T06:12:21.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.569+0000 7f069259c640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f067406a830 0x7f067406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:21.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.570+0000 7f069259c640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f067406a830 0x7f067406cce0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0684005880 tx=0x7f0684009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:21.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.573+0000 7f068b7fe640 1 -- 172.21.15.84:0/2489333691 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f069415b2b0 con 0x7f069415f7e0 2023-10-11T06:12:21.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.760+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f069415b2b0 con 0x7f069415f7e0 2023-10-11T06:12:21.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.761+0000 7f068b7fe640 1 -- 172.21.15.84:0/2489333691 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f069415b2b0 con 0x7f069415f7e0 2023-10-11T06:12:21.763 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f067406a830 msgr2=0x7f067406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f067406a830 0x7f067406cce0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0684005880 tx=0x7f0684009040 comp rx=0 tx=0).stop 2023-10-11T06:12:21.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069415f7e0 msgr2=0x7f06940ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:21.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069415f7e0 0x7f06940ca730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c7b0 tx=0x7f068c00cc70 comp rx=0 tx=0).stop 2023-10-11T06:12:21.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 shutdown_connections 2023-10-11T06:12:21.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f067406a830 0x7f067406cce0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f069415f7e0 0x7f06940ca730 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06940d2ac0 0x7f0694153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 --2- 172.21.15.84:0/2489333691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06940d0940 0x7f06940c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:21.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 >> 172.21.15.84:0/2489333691 conn(0x7f06940713a0 msgr2=0x7f0694160590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:21.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 shutdown_connections 2023-10-11T06:12:21.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.764+0000 7f0698a10640 1 -- 172.21.15.84:0/2489333691 wait complete. 2023-10-11T06:12:21.784 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:21.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.998+0000 7f5d4dcf7640 1 Processor -- start 2023-10-11T06:12:21.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.998+0000 7f5d4dcf7640 1 -- start start 2023-10-11T06:12:22.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.999+0000 7f5d4dcf7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d4815d4f0 0x7f5d480cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.999+0000 7f5d4dcf7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d480ca300 0x7f5d480c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:21.999+0000 7f5d4dcf7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d48071a90 con 0x7f5d480c9190 2023-10-11T06:12:22.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d4dcf7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d48072310 con 0x7f5d480ca300 2023-10-11T06:12:22.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d4dcf7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48070280 con 0x7f5d4815d4f0 2023-10-11T06:12:22.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d46ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d480ca300 0x7f5d480c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d46ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d480ca300 0x7f5d480c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50706/0 (socket says 172.21.15.84:50706) 2023-10-11T06:12:22.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d46ffd640 1 -- 172.21.15.84:0/4184844162 learned_addr learned my addr 172.21.15.84:0/4184844162 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:22.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d477fe640 1 --2- 172.21.15.84:0/4184844162 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d4815d4f0 0x7f5d480cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d46ffd640 1 -- 172.21.15.84:0/4184844162 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d4815d4f0 msgr2=0x7f5d480cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:22.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d46ffd640 1 --2- 172.21.15.84:0/4184844162 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d4815d4f0 0x7f5d480cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d46ffd640 1 -- 172.21.15.84:0/4184844162 >> v1:172.21.15.84:6789/0 conn(0x7f5d480c9190 legacy=0x7f5d4815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d46ffd640 1 -- 172.21.15.84:0/4184844162 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48071cc0 con 0x7f5d480ca300 2023-10-11T06:12:22.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.000+0000 7f5d467fc640 1 -- 172.21.15.84:0/4184844162 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651817585 0 0) 0x7f5d48071a90 con 0x7f5d480c9190 2023-10-11T06:12:22.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.001+0000 7f5d46ffd640 1 --2- 172.21.15.84:0/4184844162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d480c8c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009e50 tx=0x7f5d3002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d7e448563c90a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:22.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d467fc640 1 -- 172.21.15.84:0/4184844162 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3002cb00 con 0x7f5d480ca300 2023-10-11T06:12:22.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d467fc640 1 -- 172.21.15.84:0/4184844162 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3002cc60 con 0x7f5d480ca300 2023-10-11T06:12:22.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d467fc640 1 -- 172.21.15.84:0/4184844162 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d300343f0 con 0x7f5d480ca300 2023-10-11T06:12:22.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/4184844162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 msgr2=0x7f5d480c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/4184844162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d480c8c50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009e50 tx=0x7f5d3002bf60 comp rx=0 tx=0).stop 2023-10-11T06:12:22.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/4184844162 shutdown_connections 2023-10-11T06:12:22.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/4184844162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d480c8c50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/4184844162 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d4815d4f0 0x7f5d480cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.002+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/4184844162 >> 172.21.15.84:0/4184844162 conn(0x7f5d48069c70 msgr2=0x7f5d4806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:22.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.003+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/4184844162 shutdown_connections 2023-10-11T06:12:22.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.003+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/4184844162 wait complete. 2023-10-11T06:12:22.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.004+0000 7f5d4dcf7640 1 Processor -- start 2023-10-11T06:12:22.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.004+0000 7f5d4dcf7640 1 -- start start 2023-10-11T06:12:22.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.005+0000 7f5d4dcf7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d480c9190 0x7f5d481f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.005+0000 7f5d4dcf7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d48154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.005+0000 7f5d4dcf7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4815d4f0 0x7f5d480b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.005+0000 7f5d4dcf7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480cd630 con 0x7f5d480ca300 2023-10-11T06:12:22.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.005+0000 7f5d4dcf7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d481f2b60 con 0x7f5d4815d4f0 2023-10-11T06:12:22.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.005+0000 7f5d4dcf7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480b2260 con 0x7f5d480c9190 2023-10-11T06:12:22.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d48154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d48154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50710/0 (socket says 172.21.15.84:50710) 2023-10-11T06:12:22.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 -- 172.21.15.84:0/64282771 learned_addr learned my addr 172.21.15.84:0/64282771 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d477fe640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d480c9190 0x7f5d481f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 -- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d480c9190 msgr2=0x7f5d481f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d480c9190 0x7f5d481f2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d47fff640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4815d4f0 0x7f5d480b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 -- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4815d4f0 msgr2=0x7f5d480b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4815d4f0 0x7f5d480b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d46ffd640 1 -- 172.21.15.84:0/64282771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30009b30 con 0x7f5d480ca300 2023-10-11T06:12:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.006+0000 7f5d47fff640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4815d4f0 0x7f5d480b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.007+0000 7f5d46ffd640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d48154920 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5d30000f80 tx=0x7f5d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:22.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.007+0000 7f5d3ffff640 1 -- 172.21.15.84:0/64282771 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30009510 con 0x7f5d480ca300 2023-10-11T06:12:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.007+0000 7f5d3ffff640 1 -- 172.21.15.84:0/64282771 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3003e450 con 0x7f5d480ca300 2023-10-11T06:12:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.007+0000 7f5d3ffff640 1 -- 172.21.15.84:0/64282771 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3003e800 con 0x7f5d480ca300 2023-10-11T06:12:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.007+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d480b2540 con 0x7f5d480ca300 2023-10-11T06:12:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.007+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d480b2b00 con 0x7f5d480ca300 2023-10-11T06:12:22.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.008+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d48073ed0 con 0x7f5d480ca300 2023-10-11T06:12:22.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.009+0000 7f5d3ffff640 1 -- 172.21.15.84:0/64282771 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5d3003e960 con 0x7f5d480ca300 2023-10-11T06:12:22.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.010+0000 7f5d3ffff640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2806a830 0x7f5d2806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.010+0000 7f5d3ffff640 1 -- 172.21.15.84:0/64282771 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5d300abe20 con 0x7f5d480ca300 2023-10-11T06:12:22.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.010+0000 7f5d477fe640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2806a830 0x7f5d2806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.011+0000 7f5d477fe640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2806a830 0x7f5d2806cce0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5d480c50e0 tx=0x7f5d38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:22.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.014+0000 7f5d3ffff640 1 -- 172.21.15.84:0/64282771 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5d30077060 con 0x7f5d480ca300 2023-10-11T06:12:22.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.218+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d480b09b0 con 0x7f5d480ca300 2023-10-11T06:12:22.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.219+0000 7f5d3ffff640 1 -- 172.21.15.84:0/64282771 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5d3007a680 con 0x7f5d480ca300 2023-10-11T06:12:22.221 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:22.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2806a830 msgr2=0x7f5d2806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2806a830 0x7f5d2806cce0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5d480c50e0 tx=0x7f5d38008040 comp rx=0 tx=0).stop 2023-10-11T06:12:22.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 msgr2=0x7f5d48154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d48154920 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5d30000f80 tx=0x7f5d30004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:22.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 shutdown_connections 2023-10-11T06:12:22.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2806a830 0x7f5d2806cce0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d4815d4f0 0x7f5d480b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d480ca300 0x7f5d48154920 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 --2- 172.21.15.84:0/64282771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d480c9190 0x7f5d481f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 >> 172.21.15.84:0/64282771 conn(0x7f5d48069c70 msgr2=0x7f5d4815c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 shutdown_connections 2023-10-11T06:12:22.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.222+0000 7f5d4dcf7640 1 -- 172.21.15.84:0/64282771 wait complete. 2023-10-11T06:12:22.241 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2023-10-11T06:12:22.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:22.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-11T06:12:22.349 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:22.350 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:12:22.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:22.351 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T06:12:22.401 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-11T06:12:22.402 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:12:22.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.624+0000 7fdd32e2d640 1 Processor -- start 2023-10-11T06:12:22.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.625+0000 7fdd32e2d640 1 -- start start 2023-10-11T06:12:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.626+0000 7fdd32e2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd2c15c290 0x7fdd2c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.626+0000 7fdd32e2d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c0d2a60 con 0x7fdd2c0d25e0 2023-10-11T06:12:22.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.626+0000 7fdd32e2d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c06cd10 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.626+0000 7fdd32e2d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c0afb60 con 0x7fdd2c15c290 2023-10-11T06:12:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.626+0000 7fdd30bc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd2c15c290 0x7fdd2c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.627+0000 7fdd30bc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd2c15c290 0x7fdd2c0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50712/0 (socket says 172.21.15.84:50712) 2023-10-11T06:12:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.627+0000 7fdd30bc9640 1 -- 172.21.15.84:0/600766543 learned_addr learned my addr 172.21.15.84:0/600766543 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.627+0000 7fdd30bc9640 1 -- 172.21.15.84:0/600766543 >> v1:172.21.15.99:6789/0 conn(0x7fdd2c0d0c20 legacy=0x7fdd2c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:22.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.627+0000 7fdd30bc9640 1 -- 172.21.15.84:0/600766543 >> v1:172.21.15.84:6789/0 conn(0x7fdd2c0d25e0 legacy=0x7fdd2c0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.627+0000 7fdd30bc9640 1 -- 172.21.15.84:0/600766543 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c0afe40 con 0x7fdd2c15c290 2023-10-11T06:12:22.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.627+0000 7fdd2b7fe640 1 -- 172.21.15.84:0/600766543 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057237020 0 0) 0x7fdd2c0d2a60 con 0x7fdd2c0d25e0 2023-10-11T06:12:22.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.628+0000 7fdd30bc9640 1 --2- 172.21.15.84:0/600766543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c15c290 0x7fdd2c0af4d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e50 tx=0x7fdd1c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f7c9cdf9e682d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:22.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.628+0000 7fdd2b7fe640 1 -- 172.21.15.84:0/600766543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c02ca80 con 0x7fdd2c15c290 2023-10-11T06:12:22.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd2b7fe640 1 -- 172.21.15.84:0/600766543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c02cbe0 con 0x7fdd2c15c290 2023-10-11T06:12:22.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd2b7fe640 1 -- 172.21.15.84:0/600766543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0343f0 con 0x7fdd2c15c290 2023-10-11T06:12:22.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd32e2d640 1 -- 172.21.15.84:0/600766543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c15c290 msgr2=0x7fdd2c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/600766543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c15c290 0x7fdd2c0af4d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e50 tx=0x7fdd1c02bee0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd32e2d640 1 -- 172.21.15.84:0/600766543 shutdown_connections 2023-10-11T06:12:22.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/600766543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c15c290 0x7fdd2c0af4d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd32e2d640 1 -- 172.21.15.84:0/600766543 >> 172.21.15.84:0/600766543 conn(0x7fdd2c071cc0 msgr2=0x7fdd2c09bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:22.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd32e2d640 1 -- 172.21.15.84:0/600766543 shutdown_connections 2023-10-11T06:12:22.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.629+0000 7fdd32e2d640 1 -- 172.21.15.84:0/600766543 wait complete. 2023-10-11T06:12:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.631+0000 7fdd32e2d640 1 Processor -- start 2023-10-11T06:12:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.631+0000 7fdd32e2d640 1 -- start start 2023-10-11T06:12:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.632+0000 7fdd32e2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c0d0c20 0x7fdd2c14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.632+0000 7fdd32e2d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd2c0d25e0 0x7fdd2c09fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.632+0000 7fdd32e2d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2c15c290 0x7fdd2c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.632+0000 7fdd32e2d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0a0550 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.632+0000 7fdd32e2d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1f59f0 con 0x7fdd2c0d25e0 2023-10-11T06:12:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.632+0000 7fdd32e2d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1f4c30 con 0x7fdd2c15c290 2023-10-11T06:12:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c0d0c20 0x7fdd2c14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c0d0c20 0x7fdd2c14ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50722/0 (socket says 172.21.15.84:50722) 2023-10-11T06:12:22.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 -- 172.21.15.84:0/3145294959 learned_addr learned my addr 172.21.15.84:0/3145294959 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:22.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd2bfff640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd2c0d25e0 0x7fdd2c09fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd313ca640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2c15c290 0x7fdd2c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 -- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2c15c290 msgr2=0x7fdd2c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2c15c290 0x7fdd2c0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 -- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd2c0d25e0 msgr2=0x7fdd2c09fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd2c0d25e0 0x7fdd2c09fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd30bc9640 1 -- 172.21.15.84:0/3145294959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c009b30 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.633+0000 7fdd313ca640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2c15c290 0x7fdd2c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.634+0000 7fdd30bc9640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c0d0c20 0x7fdd2c14ef30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036040 tx=0x7fdd1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.634+0000 7fdd297fa640 1 -- 172.21.15.84:0/3145294959 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c03e450 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.634+0000 7fdd297fa640 1 -- 172.21.15.84:0/3145294959 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c03e5f0 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.634+0000 7fdd297fa640 1 -- 172.21.15.84:0/3145294959 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c03e9a0 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.634+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c1f4f10 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.634+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c1f54d0 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.635+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c1f4d70 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.636+0000 7fdd297fa640 1 -- 172.21.15.84:0/3145294959 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdd1c02cd50 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.637+0000 7fdd297fa640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd1806a830 0x7fdd1806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.637+0000 7fdd297fa640 1 -- 172.21.15.84:0/3145294959 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdd1c0abc60 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.637+0000 7fdd2bfff640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd1806a830 0x7fdd1806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.638+0000 7fdd2bfff640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd1806a830 0x7fdd1806cce0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0b8900 tx=0x7fdd20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:22.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.642+0000 7fdd297fa640 1 -- 172.21.15.84:0/3145294959 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdd2c1f4d70 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.867+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd2c1f4d70 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.868+0000 7fdd297fa640 1 -- 172.21.15.84:0/3145294959 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7fdd2c1f4d70 con 0x7fdd2c0d0c20 2023-10-11T06:12:22.870 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:22.870 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:12:22.870 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:12:22.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd1806a830 msgr2=0x7fdd1806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd1806a830 0x7fdd1806cce0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0b8900 tx=0x7fdd20009040 comp rx=0 tx=0).stop 2023-10-11T06:12:22.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c0d0c20 msgr2=0x7fdd2c14ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:22.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c0d0c20 0x7fdd2c14ef30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036040 tx=0x7fdd1c004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:22.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 shutdown_connections 2023-10-11T06:12:22.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd1806a830 0x7fdd1806cce0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2c15c290 0x7fdd2c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd2c0d25e0 0x7fdd2c09fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 --2- 172.21.15.84:0/3145294959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2c0d0c20 0x7fdd2c14ef30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:22.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 >> 172.21.15.84:0/3145294959 conn(0x7fdd2c071cc0 msgr2=0x7fdd2c0d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:22.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 shutdown_connections 2023-10-11T06:12:22.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:22.871+0000 7fdd32e2d640 1 -- 172.21.15.84:0/3145294959 wait complete. 2023-10-11T06:12:22.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:23.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.104+0000 7fefb2e3b640 1 Processor -- start 2023-10-11T06:12:23.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.104+0000 7fefb2e3b640 1 -- start start 2023-10-11T06:12:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.105+0000 7fefb2e3b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 0x7fefac158730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.105+0000 7fefb2e3b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0adc40 con 0x7fefac06c3d0 2023-10-11T06:12:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.106+0000 7fefb2e3b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0c97e0 con 0x7fefac06c000 2023-10-11T06:12:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.106+0000 7fefb2e3b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0c9ac0 con 0x7fefac0c9330 2023-10-11T06:12:23.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.106+0000 7fefb263a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fefac06c3d0 0x7fefac158c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37160/0 (socket says 172.21.15.84:37160) 2023-10-11T06:12:23.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.106+0000 7fefb263a640 1 -- 172.21.15.84:0/153806598 learned_addr learned my addr 172.21.15.84:0/153806598 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.106+0000 7fefb1638640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 0x7fefac158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.106+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4044077154 0 0) 0x7fefac0adc40 con 0x7fefac06c3d0 2023-10-11T06:12:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.106+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef9c003520 con 0x7fefac06c3d0 2023-10-11T06:12:23.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497551773 0 0) 0x7fef9c003520 con 0x7fefac06c3d0 2023-10-11T06:12:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 msgr2=0x7fefac158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 0x7fefac158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 >> v1:172.21.15.99:6789/0 conn(0x7fefac06c000 legacy=0x7fefac0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0c9da0 con 0x7fefac06c3d0 2023-10-11T06:12:23.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fefa4001670 con 0x7fefac06c3d0 2023-10-11T06:12:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 >> v1:172.21.15.84:6789/0 conn(0x7fefac06c3d0 legacy=0x7fefac158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 0x7fef9c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef9c003f70 0x7fef9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef9c004d00 0x7fef9c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0c9ac0 con 0x7fef9c004d00 2023-10-11T06:12:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c00dca0 con 0x7fef9c003f70 2023-10-11T06:12:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c00de60 con 0x7fefac0c9330 2023-10-11T06:12:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb263a640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef9c004d00 0x7fef9c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.107+0000 7fefb1e39640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 0x7fef9c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.108+0000 7fefb263a640 1 -- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 msgr2=0x7fef9c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.108+0000 7fefb1638640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef9c003f70 0x7fef9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.108+0000 7fefb263a640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 0x7fef9c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.108+0000 7fefb263a640 1 -- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef9c003f70 msgr2=0x7fef9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.108+0000 7fefb263a640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef9c003f70 0x7fef9c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.108+0000 7fefb263a640 1 -- 172.21.15.84:0/153806598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c003520 con 0x7fef9c004d00 2023-10-11T06:12:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.108+0000 7fefb1e39640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefac0c9330 0x7fef9c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.109+0000 7fefb263a640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef9c004d00 0x7fef9c005170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fefa400c0e0 tx=0x7fef9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a852a469f2087e7d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.109+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4005ad0 con 0x7fef9c004d00 2023-10-11T06:12:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.109+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4005c50 con 0x7fef9c004d00 2023-10-11T06:12:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.109+0000 7fefb0e37640 1 -- 172.21.15.84:0/153806598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa402c810 con 0x7fef9c004d00 2023-10-11T06:12:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.110+0000 7fefb2e3b640 1 -- 172.21.15.84:0/153806598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef9c004d00 msgr2=0x7fef9c005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.110+0000 7fefb2e3b640 1 --2- 172.21.15.84:0/153806598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef9c004d00 0x7fef9c005170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fefa400c0e0 tx=0x7fef9c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.110+0000 7fefb263a640 1 -- 172.21.15.84:0/153806598 reap_dead start 2023-10-11T06:12:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.110+0000 7fefb2e3b640 1 -- 172.21.15.84:0/153806598 shutdown_connections 2023-10-11T06:12:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.110+0000 7fefb2e3b640 1 -- 172.21.15.84:0/153806598 >> 172.21.15.84:0/153806598 conn(0x7fefac069c70 msgr2=0x7fefac06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.110+0000 7fefb2e3b640 1 -- 172.21.15.84:0/153806598 shutdown_connections 2023-10-11T06:12:23.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.110+0000 7fefb2e3b640 1 -- 172.21.15.84:0/153806598 wait complete. 2023-10-11T06:12:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.111+0000 7fefb2e3b640 1 Processor -- start 2023-10-11T06:12:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.112+0000 7fefb2e3b640 1 -- start start 2023-10-11T06:12:23.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb2e3b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef9c004d00 0x7fefac0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb2e3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefac0c6720 0x7fefac0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb2e3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac0c6b90 0x7fefac0b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb2e3b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0b0de0 con 0x7fefac0c6b90 2023-10-11T06:12:23.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb2e3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0b08b0 con 0x7fefac0c6720 2023-10-11T06:12:23.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb2e3b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac1541e0 con 0x7fef9c004d00 2023-10-11T06:12:23.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac0c6b90 0x7fefac0b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac0c6b90 0x7fefac0b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50744/0 (socket says 172.21.15.84:50744) 2023-10-11T06:12:23.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 -- 172.21.15.84:0/2704724191 learned_addr learned my addr 172.21.15.84:0/2704724191 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:23.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb1638640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefac0c6720 0x7fefac0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb1e39640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef9c004d00 0x7fefac0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 -- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef9c004d00 msgr2=0x7fefac0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef9c004d00 0x7fefac0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 -- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefac0c6720 msgr2=0x7fefac0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefac0c6720 0x7fefac0afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.113+0000 7fefb263a640 1 -- 172.21.15.84:0/2704724191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0af3e0 con 0x7fefac0c6b90 2023-10-11T06:12:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.114+0000 7fefb263a640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac0c6b90 0x7fefac0b0370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fefa400c0e0 tx=0x7fefa402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.114+0000 7fefaa7fc640 1 -- 172.21.15.84:0/2704724191 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa403c040 con 0x7fefac0c6b90 2023-10-11T06:12:23.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.114+0000 7fefaa7fc640 1 -- 172.21.15.84:0/2704724191 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa402cc20 con 0x7fefac0c6b90 2023-10-11T06:12:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.114+0000 7fefaa7fc640 1 -- 172.21.15.84:0/2704724191 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4000ed0 con 0x7fefac0c6b90 2023-10-11T06:12:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.114+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac157890 con 0x7fefac0c6b90 2023-10-11T06:12:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.114+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac0af630 con 0x7fefac0c6b90 2023-10-11T06:12:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.115+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac04e890 con 0x7fefac0c6b90 2023-10-11T06:12:23.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.116+0000 7fefaa7fc640 1 -- 172.21.15.84:0/2704724191 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fefa4032070 con 0x7fefac0c6b90 2023-10-11T06:12:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.117+0000 7fefaa7fc640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fef9006a860 0x7fef9006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.117+0000 7fefaa7fc640 1 -- 172.21.15.84:0/2704724191 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fefa40a9730 con 0x7fefac0c6b90 2023-10-11T06:12:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.117+0000 7fefb1e39640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fef9006a860 0x7fef9006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.118+0000 7fefb1e39640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fef9006a860 0x7fef9006cd10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefac0c4fd0 tx=0x7fefa0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:23.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.122+0000 7fefaa7fc640 1 -- 172.21.15.84:0/2704724191 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fefa4074970 con 0x7fefac0c6b90 2023-10-11T06:12:23.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.312+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefac154d00 con 0x7fefac0c6b90 2023-10-11T06:12:23.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.314+0000 7fefaa7fc640 1 -- 172.21.15.84:0/2704724191 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fefa4077f90 con 0x7fefac0c6b90 2023-10-11T06:12:23.316 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:23.316 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:23.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.316+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fef9006a860 msgr2=0x7fef9006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.316+0000 7fefb2e3b640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fef9006a860 0x7fef9006cd10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefac0c4fd0 tx=0x7fefa0009040 comp rx=0 tx=0).stop 2023-10-11T06:12:23.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac0c6b90 msgr2=0x7fefac0b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac0c6b90 0x7fefac0b0370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fefa400c0e0 tx=0x7fefa402e070 comp rx=0 tx=0).stop 2023-10-11T06:12:23.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 shutdown_connections 2023-10-11T06:12:23.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fef9006a860 0x7fef9006cd10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac0c6b90 0x7fefac0b0370 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefac0c6720 0x7fefac0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 --2- 172.21.15.84:0/2704724191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef9c004d00 0x7fefac0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 >> 172.21.15.84:0/2704724191 conn(0x7fefac069c70 msgr2=0x7fefac0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:23.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 shutdown_connections 2023-10-11T06:12:23.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.317+0000 7fefb2e3b640 1 -- 172.21.15.84:0/2704724191 wait complete. 2023-10-11T06:12:23.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:11:12.075151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:11:12.661632+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:23.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:23.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.558+0000 7f0c51173640 1 Processor -- start 2023-10-11T06:12:23.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.558+0000 7f0c51173640 1 -- start start 2023-10-11T06:12:23.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.559+0000 7f0c51173640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c4c0d2ac0 0x7f0c4c0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.559+0000 7f0c51173640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0cf220 con 0x7f0c4c15f7e0 2023-10-11T06:12:23.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.559+0000 7f0c51173640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0cf500 con 0x7f0c4c0d0940 2023-10-11T06:12:23.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.559+0000 7f0c51173640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c0d3480 con 0x7f0c4c0d2ac0 2023-10-11T06:12:23.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.560+0000 7f0c43fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c4c0d2ac0 0x7f0c4c0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.560+0000 7f0c4ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0c4c15f7e0 0x7f0c4c0cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37170/0 (socket says 172.21.15.84:37170) 2023-10-11T06:12:23.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.560+0000 7f0c4ad9d640 1 -- 172.21.15.84:0/518378128 learned_addr learned my addr 172.21.15.84:0/518378128 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:23.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.560+0000 7f0c43fff640 1 -- 172.21.15.84:0/518378128 >> v1:172.21.15.99:6789/0 conn(0x7f0c4c0d0940 legacy=0x7f0c4c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:23.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.560+0000 7f0c43fff640 1 -- 172.21.15.84:0/518378128 >> v1:172.21.15.84:6789/0 conn(0x7f0c4c15f7e0 legacy=0x7f0c4c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.560+0000 7f0c4a59c640 1 -- 172.21.15.84:0/518378128 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198956566 0 0) 0x7f0c4c0cf220 con 0x7f0c4c15f7e0 2023-10-11T06:12:23.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.560+0000 7f0c43fff640 1 -- 172.21.15.84:0/518378128 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0cf7e0 con 0x7f0c4c0d2ac0 2023-10-11T06:12:23.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.561+0000 7f0c43fff640 1 --2- 172.21.15.84:0/518378128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d2ac0 0x7f0c4c0d2f40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097a0 tx=0x7f0c3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9438841fc42ea7c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:23.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c4a59c640 1 -- 172.21.15.84:0/518378128 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34031830 con 0x7f0c4c0d2ac0 2023-10-11T06:12:23.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c4a59c640 1 -- 172.21.15.84:0/518378128 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34031990 con 0x7f0c4c0d2ac0 2023-10-11T06:12:23.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c4a59c640 1 -- 172.21.15.84:0/518378128 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34031d20 con 0x7f0c4c0d2ac0 2023-10-11T06:12:23.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c51173640 1 -- 172.21.15.84:0/518378128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d2ac0 msgr2=0x7f0c4c0d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c51173640 1 --2- 172.21.15.84:0/518378128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d2ac0 0x7f0c4c0d2f40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097a0 tx=0x7f0c3402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c51173640 1 -- 172.21.15.84:0/518378128 shutdown_connections 2023-10-11T06:12:23.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c51173640 1 --2- 172.21.15.84:0/518378128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d2ac0 0x7f0c4c0d2f40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.562+0000 7f0c51173640 1 -- 172.21.15.84:0/518378128 >> 172.21.15.84:0/518378128 conn(0x7f0c4c0713a0 msgr2=0x7f0c4c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:23.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.563+0000 7f0c51173640 1 -- 172.21.15.84:0/518378128 shutdown_connections 2023-10-11T06:12:23.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.563+0000 7f0c51173640 1 -- 172.21.15.84:0/518378128 wait complete. 2023-10-11T06:12:23.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.564+0000 7f0c51173640 1 Processor -- start 2023-10-11T06:12:23.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.564+0000 7f0c51173640 1 -- start start 2023-10-11T06:12:23.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.565+0000 7f0c51173640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d0940 0x7f0c4c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.565+0000 7f0c51173640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c4c0d2ac0 0x7f0c4c14d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.565+0000 7f0c51173640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c4c15f7e0 0x7f0c4c14dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.565+0000 7f0c51173640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0ccab0 con 0x7f0c4c0d0940 2023-10-11T06:12:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.565+0000 7f0c51173640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0cba10 con 0x7f0c4c0d2ac0 2023-10-11T06:12:23.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.565+0000 7f0c51173640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0cbcf0 con 0x7f0c4c15f7e0 2023-10-11T06:12:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d0940 0x7f0c4c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d0940 0x7f0c4c0ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50760/0 (socket says 172.21.15.84:50760) 2023-10-11T06:12:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 -- 172.21.15.84:0/342485436 learned_addr learned my addr 172.21.15.84:0/342485436 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:23.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 -- 172.21.15.84:0/342485436 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c4c15f7e0 msgr2=0x7f0c4c14dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c4c15f7e0 0x7f0c4c14dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 -- 172.21.15.84:0/342485436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c4c0d2ac0 msgr2=0x7f0c4c14d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c4c0d2ac0 0x7f0c4c14d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 -- 172.21.15.84:0/342485436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c34009400 con 0x7f0c4c0d0940 2023-10-11T06:12:23.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.566+0000 7f0c4ad9d640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d0940 0x7f0c4c0ca610 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c430 tx=0x7f0c3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:23.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.567+0000 7f0c437fe640 1 -- 172.21.15.84:0/342485436 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0074e0 con 0x7f0c4c0d0940 2023-10-11T06:12:23.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.567+0000 7f0c437fe640 1 -- 172.21.15.84:0/342485436 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c007660 con 0x7f0c4c0d0940 2023-10-11T06:12:23.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.567+0000 7f0c437fe640 1 -- 172.21.15.84:0/342485436 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c004a20 con 0x7f0c4c0d0940 2023-10-11T06:12:23.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.567+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c0cbfd0 con 0x7f0c4c0d0940 2023-10-11T06:12:23.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.567+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c0cc590 con 0x7f0c4c0d0940 2023-10-11T06:12:23.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.568+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c0cc780 con 0x7f0c4c0d0940 2023-10-11T06:12:23.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.569+0000 7f0c437fe640 1 -- 172.21.15.84:0/342485436 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0c3c004b80 con 0x7f0c4c0d0940 2023-10-11T06:12:23.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.570+0000 7f0c437fe640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c2c06a860 0x7f0c2c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.570+0000 7f0c437fe640 1 -- 172.21.15.84:0/342485436 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0c3c088e90 con 0x7f0c4c0d0940 2023-10-11T06:12:23.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.570+0000 7f0c43fff640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c2c06a860 0x7f0c2c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.571+0000 7f0c43fff640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c2c06a860 0x7f0c2c06cd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c34036000 tx=0x7f0c34030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:23.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.574+0000 7f0c437fe640 1 -- 172.21.15.84:0/342485436 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0c3c054030 con 0x7f0c4c0d0940 2023-10-11T06:12:23.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.762+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c4c073ed0 con 0x7f0c4c0d0940 2023-10-11T06:12:23.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.763+0000 7f0c437fe640 1 -- 172.21.15.84:0/342485436 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c3c057650 con 0x7f0c4c0d0940 2023-10-11T06:12:23.764 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:23.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c2c06a860 msgr2=0x7f0c2c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c2c06a860 0x7f0c2c06cd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c34036000 tx=0x7f0c34030000 comp rx=0 tx=0).stop 2023-10-11T06:12:23.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d0940 msgr2=0x7f0c4c0ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d0940 0x7f0c4c0ca610 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c430 tx=0x7f0c3c00c8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 shutdown_connections 2023-10-11T06:12:23.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c2c06a860 0x7f0c2c06cd10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c4c15f7e0 0x7f0c4c14dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c4c0d2ac0 0x7f0c4c14d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 --2- 172.21.15.84:0/342485436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c4c0d0940 0x7f0c4c0ca610 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 >> 172.21.15.84:0/342485436 conn(0x7f0c4c0713a0 msgr2=0x7f0c4c15ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:23.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 shutdown_connections 2023-10-11T06:12:23.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.765+0000 7f0c51173640 1 -- 172.21.15.84:0/342485436 wait complete. 2023-10-11T06:12:23.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:23.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.992+0000 7f5f8d2d4640 1 Processor -- start 2023-10-11T06:12:23.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.992+0000 7f5f8d2d4640 1 -- start start 2023-10-11T06:12:23.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.993+0000 7f5f8d2d4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5f8806c640 0x7f5f88159830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.993+0000 7f5f8d2d4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:23.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.993+0000 7f5f8d2d4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8806dd40 con 0x7f5f8806ca10 2023-10-11T06:12:23.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.993+0000 7f5f8d2d4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8815c480 con 0x7f5f8806c640 2023-10-11T06:12:23.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.993+0000 7f5f8d2d4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8806e020 con 0x7f5f880ce6b0 2023-10-11T06:12:23.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.994+0000 7f5f86ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5f8806c640 0x7f5f88159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.994+0000 7f5f86ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5f8806c640 0x7f5f88159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:46558/0 (socket says 172.21.15.84:46558) 2023-10-11T06:12:23.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.994+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2806948423 learned_addr learned my addr 172.21.15.84:0/2806948423 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:23.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.994+0000 7f5f877fe640 1 --2- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:23.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.994+0000 7f5f7ffff640 1 --1- 172.21.15.84:0/2806948423 >> v1:172.21.15.99:6789/0 conn(0x7f5f8806ca10 0x7f5f88152d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:56792/0 (socket says 172.21.15.84:56792) 2023-10-11T06:12:23.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.995+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 msgr2=0x7f5f880caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:23.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.995+0000 7f5f86ffd640 1 --2- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:23.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.995+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2806948423 >> v1:172.21.15.99:6789/0 conn(0x7f5f8806ca10 legacy=0x7f5f88152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:23.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.995+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2806948423 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880cc420 con 0x7f5f8806c640 2023-10-11T06:12:24.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.996+0000 7f5f86ffd640 1 --2- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806c640 0x7f5f88159830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009870 tx=0x7f5f7802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ecacf634ef9c03 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.997+0000 7f5f867fc640 1 -- 172.21.15.84:0/2806948423 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78030840 con 0x7f5f8806c640 2023-10-11T06:12:24.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.997+0000 7f5f867fc640 1 -- 172.21.15.84:0/2806948423 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780309a0 con 0x7f5f8806c640 2023-10-11T06:12:24.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.997+0000 7f5f867fc640 1 -- 172.21.15.84:0/2806948423 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78030d30 con 0x7f5f8806c640 2023-10-11T06:12:24.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.997+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806c640 msgr2=0x7f5f88159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.997+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806c640 0x7f5f88159830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009870 tx=0x7f5f7802b980 comp rx=0 tx=0).stop 2023-10-11T06:12:24.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.997+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2806948423 shutdown_connections 2023-10-11T06:12:24.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.997+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.998+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2806948423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806c640 0x7f5f88159830 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.998+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2806948423 >> 172.21.15.84:0/2806948423 conn(0x7f5f88069c70 msgr2=0x7f5f8806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:24.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.998+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2806948423 shutdown_connections 2023-10-11T06:12:24.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:23.998+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2806948423 wait complete. 2023-10-11T06:12:24.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.000+0000 7f5f8d2d4640 1 Processor -- start 2023-10-11T06:12:24.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.000+0000 7f5f8d2d4640 1 -- start start 2023-10-11T06:12:24.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f8d2d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8806c640 0x7f5f88131f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f8d2d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806ca10 0x7f5f880b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f8d2d4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f8d2d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881f61b0 con 0x7f5f8806c640 2023-10-11T06:12:24.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f8d2d4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881f52e0 con 0x7f5f8806ca10 2023-10-11T06:12:24.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f8d2d4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881f5530 con 0x7f5f880ce6b0 2023-10-11T06:12:24.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8806c640 0x7f5f88131f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8806c640 0x7f5f88131f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50766/0 (socket says 172.21.15.84:50766) 2023-10-11T06:12:24.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.001+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2125369290 learned_addr learned my addr 172.21.15.84:0/2125369290 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:24.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806ca10 0x7f5f880b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f877fe640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 msgr2=0x7f5f880b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f86ffd640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806ca10 msgr2=0x7f5f880b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f86ffd640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806ca10 0x7f5f880b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f86ffd640 1 -- 172.21.15.84:0/2125369290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f780094d0 con 0x7f5f8806c640 2023-10-11T06:12:24.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806ca10 0x7f5f880b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:24.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.002+0000 7f5f86ffd640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8806c640 0x7f5f88131f30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.003+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/2125369290 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78039c90 con 0x7f5f8806c640 2023-10-11T06:12:24.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.003+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/2125369290 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f78039df0 con 0x7f5f8806c640 2023-10-11T06:12:24.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.003+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f881f5810 con 0x7f5f8806c640 2023-10-11T06:12:24.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.003+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/2125369290 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78043640 con 0x7f5f8806c640 2023-10-11T06:12:24.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.003+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f881f5dd0 con 0x7f5f8806c640 2023-10-11T06:12:24.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.004+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8806ab90 con 0x7f5f8806c640 2023-10-11T06:12:24.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.005+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/2125369290 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5f780437a0 con 0x7f5f8806c640 2023-10-11T06:12:24.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.005+0000 7f5f7f7fe640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5f6806a830 0x7f5f6806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.006+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/2125369290 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5f7803e070 con 0x7f5f8806c640 2023-10-11T06:12:24.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.006+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5f6806a830 0x7f5f6806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.006+0000 7f5f7ffff640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5f6806a830 0x7f5f6806cce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f88145b20 tx=0x7f5f70007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.010+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/2125369290 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5f78077300 con 0x7f5f8806c640 2023-10-11T06:12:24.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.198+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5f88073ed0 con 0x7f5f8806c640 2023-10-11T06:12:24.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.199+0000 7f5f7f7fe640 1 -- 172.21.15.84:0/2125369290 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5f7807a920 con 0x7f5f8806c640 2023-10-11T06:12:24.201 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:24.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5f6806a830 msgr2=0x7f5f6806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5f6806a830 0x7f5f6806cce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f88145b20 tx=0x7f5f70007ca0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8806c640 msgr2=0x7f5f88131f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8806c640 0x7f5f88131f30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f78004730 comp rx=0 tx=0).stop 2023-10-11T06:12:24.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 shutdown_connections 2023-10-11T06:12:24.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5f6806a830 0x7f5f6806cce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f880ce6b0 0x7f5f880b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8806ca10 0x7f5f880b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 --2- 172.21.15.84:0/2125369290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f8806c640 0x7f5f88131f30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 >> 172.21.15.84:0/2125369290 conn(0x7f5f88069c70 msgr2=0x7f5f880c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:24.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 shutdown_connections 2023-10-11T06:12:24.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.202+0000 7f5f8d2d4640 1 -- 172.21.15.84:0/2125369290 wait complete. 2023-10-11T06:12:24.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:24.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.428+0000 7f3673fdd640 1 Processor -- start 2023-10-11T06:12:24.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.429+0000 7f3673fdd640 1 -- start start 2023-10-11T06:12:24.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.430+0000 7f3673fdd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f366c156810 0x7f366c06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.430+0000 7f3673fdd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366c0cde80 con 0x7f366c06c880 2023-10-11T06:12:24.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.430+0000 7f3673fdd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366c06df80 con 0x7f366c156be0 2023-10-11T06:12:24.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.430+0000 7f3673fdd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366c0ce140 con 0x7f366c156810 2023-10-11T06:12:24.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.431+0000 7f367257a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f366c06c880 0x7f366c0ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37176/0 (socket says 172.21.15.84:37176) 2023-10-11T06:12:24.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.431+0000 7f367257a640 1 -- 172.21.15.84:0/612821337 learned_addr learned my addr 172.21.15.84:0/612821337 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:24.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.431+0000 7f3671d79640 1 --2- 172.21.15.84:0/612821337 >> v2:172.21.15.99:3300/0 conn(0x7f366c156810 0x7f366c06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.431+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320990768 0 0) 0x7f366c0cde80 con 0x7f366c06c880 2023-10-11T06:12:24.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.431+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3658003520 con 0x7f366c06c880 2023-10-11T06:12:24.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.431+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 161312819 0 0) 0x7f3658003520 con 0x7f366c06c880 2023-10-11T06:12:24.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 >> v2:172.21.15.99:3300/0 conn(0x7f366c156810 msgr2=0x7f366c06d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:24.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 --2- 172.21.15.84:0/612821337 >> v2:172.21.15.99:3300/0 conn(0x7f366c156810 0x7f366c06d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 >> v1:172.21.15.99:6789/0 conn(0x7f366c156be0 legacy=0x7f366c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c0cc3b0 con 0x7f366c06c880 2023-10-11T06:12:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f36640019f0 con 0x7f366c06c880 2023-10-11T06:12:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 >> v1:172.21.15.84:6789/0 conn(0x7f366c06c880 legacy=0x7f366c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3658003970 0x7f3658003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36580047c0 0x7f3658004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3658005170 0x7f365800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c0ce140 con 0x7f3658003970 2023-10-11T06:12:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365800c080 con 0x7f36580047c0 2023-10-11T06:12:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365800c240 con 0x7f3658005170 2023-10-11T06:12:24.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3671d79640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3658003970 0x7f3658003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3671d79640 1 -- 172.21.15.84:0/612821337 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3658005170 msgr2=0x7f365800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:24.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3671d79640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3658005170 0x7f365800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3671d79640 1 -- 172.21.15.84:0/612821337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36580047c0 msgr2=0x7f3658004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:24.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3671d79640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36580047c0 0x7f3658004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f3671d79640 1 -- 172.21.15.84:0/612821337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658003520 con 0x7f3658003970 2023-10-11T06:12:24.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.432+0000 7f367257a640 1 -- 172.21.15.84:0/612821337 reap_dead start 2023-10-11T06:12:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.433+0000 7f3671d79640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3658003970 0x7f3658003d80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3668009ac0 tx=0x7f36580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4df306e5e5983b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366802c6b0 con 0x7f3658003970 2023-10-11T06:12:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366802c810 con 0x7f3658003970 2023-10-11T06:12:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3670d77640 1 -- 172.21.15.84:0/612821337 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366802cba0 con 0x7f3658003970 2023-10-11T06:12:24.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3673fdd640 1 -- 172.21.15.84:0/612821337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3658003970 msgr2=0x7f3658003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3673fdd640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3658003970 0x7f3658003d80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3668009ac0 tx=0x7f36580057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3673fdd640 1 -- 172.21.15.84:0/612821337 shutdown_connections 2023-10-11T06:12:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3673fdd640 1 --2- 172.21.15.84:0/612821337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3658003970 0x7f3658003d80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3673fdd640 1 -- 172.21.15.84:0/612821337 >> 172.21.15.84:0/612821337 conn(0x7f366c069c70 msgr2=0x7f366c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:24.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3673fdd640 1 -- 172.21.15.84:0/612821337 shutdown_connections 2023-10-11T06:12:24.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.434+0000 7f3673fdd640 1 -- 172.21.15.84:0/612821337 wait complete. 2023-10-11T06:12:24.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.436+0000 7f3673fdd640 1 Processor -- start 2023-10-11T06:12:24.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.436+0000 7f3673fdd640 1 -- start start 2023-10-11T06:12:24.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3673fdd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3658003970 0x7f366c0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3673fdd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f366c0ac920 0x7f366c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3673fdd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c0c9c40 0x7f366c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3673fdd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c0ace20 con 0x7f366c0ac920 2023-10-11T06:12:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3673fdd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c0ad100 con 0x7f366c0c9c40 2023-10-11T06:12:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3673fdd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c0ad3e0 con 0x7f3658003970 2023-10-11T06:12:24.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3671578640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f366c0ac920 0x7f366c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3671578640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f366c0ac920 0x7f366c0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50784/0 (socket says 172.21.15.84:50784) 2023-10-11T06:12:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.437+0000 7f3671578640 1 -- 172.21.15.84:0/2225374424 learned_addr learned my addr 172.21.15.84:0/2225374424 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3671578640 1 -- 172.21.15.84:0/2225374424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3658003970 msgr2=0x7f366c0adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3671578640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3658003970 0x7f366c0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3671578640 1 -- 172.21.15.84:0/2225374424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c0c9c40 msgr2=0x7f366c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3671578640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c0c9c40 0x7f366c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3671578640 1 -- 172.21.15.84:0/2225374424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c0ad6c0 con 0x7f366c0ac920 2023-10-11T06:12:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3671578640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f366c0ac920 0x7f366c0c6710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f365c00c500 tx=0x7f365c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f36627fc640 1 -- 172.21.15.84:0/2225374424 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0052d0 con 0x7f366c0ac920 2023-10-11T06:12:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f36627fc640 1 -- 172.21.15.84:0/2225374424 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c005450 con 0x7f366c0ac920 2023-10-11T06:12:24.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c0ad9a0 con 0x7f366c0ac920 2023-10-11T06:12:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f36627fc640 1 -- 172.21.15.84:0/2225374424 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c003830 con 0x7f366c0ac920 2023-10-11T06:12:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.438+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c0c6c50 con 0x7f366c0ac920 2023-10-11T06:12:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.439+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c0cbd30 con 0x7f366c0ac920 2023-10-11T06:12:24.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.441+0000 7f36627fc640 1 -- 172.21.15.84:0/2225374424 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f365c003990 con 0x7f366c0ac920 2023-10-11T06:12:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.442+0000 7f36627fc640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f365006a860 0x7f365006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.442+0000 7f36627fc640 1 -- 172.21.15.84:0/2225374424 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f365c088660 con 0x7f366c0ac920 2023-10-11T06:12:24.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.442+0000 7f3671d79640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f365006a860 0x7f365006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.442+0000 7f3671d79640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f365006a860 0x7f365006cd10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3668036000 tx=0x7f366802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.446+0000 7f36627fc640 1 -- 172.21.15.84:0/2225374424 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f365c057320 con 0x7f366c0ac920 2023-10-11T06:12:24.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.633+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f366c073ed0 con 0x7f366c0ac920 2023-10-11T06:12:24.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.633+0000 7f36627fc640 1 -- 172.21.15.84:0/2225374424 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f365c0011b0 con 0x7f366c0ac920 2023-10-11T06:12:24.635 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:24.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f365006a860 msgr2=0x7f365006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f365006a860 0x7f365006cd10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3668036000 tx=0x7f366802e000 comp rx=0 tx=0).stop 2023-10-11T06:12:24.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f366c0ac920 msgr2=0x7f366c0c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f366c0ac920 0x7f366c0c6710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f365c00c500 tx=0x7f365c00c9c0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 shutdown_connections 2023-10-11T06:12:24.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f365006a860 0x7f365006cd10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f366c0c9c40 0x7f366c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f366c0ac920 0x7f366c0c6710 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 --2- 172.21.15.84:0/2225374424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3658003970 0x7f366c0adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.636+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 >> 172.21.15.84:0/2225374424 conn(0x7f366c069c70 msgr2=0x7f366c0b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:24.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.637+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 shutdown_connections 2023-10-11T06:12:24.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.637+0000 7f3673fdd640 1 -- 172.21.15.84:0/2225374424 wait complete. 2023-10-11T06:12:24.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:24.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.874+0000 7fe234e54640 1 Processor -- start 2023-10-11T06:12:24.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.874+0000 7fe234e54640 1 -- start start 2023-10-11T06:12:24.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe234e54640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 0x7fe2300cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe234e54640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2300b1ee0 0x7fe2300cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe234e54640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23015cba0 con 0x7fe2300cecb0 2023-10-11T06:12:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe234e54640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23009c220 con 0x7fe2300b1ee0 2023-10-11T06:12:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe234e54640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300d1360 con 0x7fe2300b1b10 2023-10-11T06:12:24.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe22effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2300b1ee0 0x7fe2300cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe22effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2300b1ee0 0x7fe2300cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50798/0 (socket says 172.21.15.84:50798) 2023-10-11T06:12:24.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.875+0000 7fe22effd640 1 -- 172.21.15.84:0/4019342272 learned_addr learned my addr 172.21.15.84:0/4019342272 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:24.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.876+0000 7fe22f7fe640 1 --2- 172.21.15.84:0/4019342272 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 0x7fe2300cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.876+0000 7fe22effd640 1 -- 172.21.15.84:0/4019342272 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 msgr2=0x7fe2300cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:24.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.876+0000 7fe22effd640 1 --2- 172.21.15.84:0/4019342272 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 0x7fe2300cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.876+0000 7fe22effd640 1 -- 172.21.15.84:0/4019342272 >> v1:172.21.15.99:6789/0 conn(0x7fe2300cecb0 legacy=0x7fe2300ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:24.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.876+0000 7fe22effd640 1 -- 172.21.15.84:0/4019342272 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2300d0ac0 con 0x7fe2300b1ee0 2023-10-11T06:12:24.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.877+0000 7fe22effd640 1 --2- 172.21.15.84:0/4019342272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2300cf740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe218009e50 tx=0x7fe21802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771e78bd8699a1b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.877+0000 7fe22e7fc640 1 -- 172.21.15.84:0/4019342272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21802cb00 con 0x7fe2300b1ee0 2023-10-11T06:12:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.877+0000 7fe22e7fc640 1 -- 172.21.15.84:0/4019342272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21802cc60 con 0x7fe2300b1ee0 2023-10-11T06:12:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.877+0000 7fe22e7fc640 1 -- 172.21.15.84:0/4019342272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2180343f0 con 0x7fe2300b1ee0 2023-10-11T06:12:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 -- 172.21.15.84:0/4019342272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 msgr2=0x7fe2300cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 --2- 172.21.15.84:0/4019342272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2300cf740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe218009e50 tx=0x7fe21802bf60 comp rx=0 tx=0).stop 2023-10-11T06:12:24.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 -- 172.21.15.84:0/4019342272 shutdown_connections 2023-10-11T06:12:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 --2- 172.21.15.84:0/4019342272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2300cf740 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 --2- 172.21.15.84:0/4019342272 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 0x7fe2300cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 -- 172.21.15.84:0/4019342272 >> 172.21.15.84:0/4019342272 conn(0x7fe23009bc20 msgr2=0x7fe2300d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:24.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 -- 172.21.15.84:0/4019342272 shutdown_connections 2023-10-11T06:12:24.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.878+0000 7fe234e54640 1 -- 172.21.15.84:0/4019342272 wait complete. 2023-10-11T06:12:24.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.880+0000 7fe234e54640 1 Processor -- start 2023-10-11T06:12:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.880+0000 7fe234e54640 1 -- start start 2023-10-11T06:12:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe234e54640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 0x7fe2300b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe234e54640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2301f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe234e54640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2300cecb0 0x7fe23014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe234e54640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300b76a0 con 0x7fe2300b1ee0 2023-10-11T06:12:24.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe234e54640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300ca290 con 0x7fe2300cecb0 2023-10-11T06:12:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe234e54640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300c99f0 con 0x7fe2300b1b10 2023-10-11T06:12:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2301f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2301f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50802/0 (socket says 172.21.15.84:50802) 2023-10-11T06:12:24.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 -- 172.21.15.84:0/862932600 learned_addr learned my addr 172.21.15.84:0/862932600 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:24.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 -- 172.21.15.84:0/862932600 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 msgr2=0x7fe2300b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 0x7fe2300b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 -- 172.21.15.84:0/862932600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2300cecb0 msgr2=0x7fe23014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:24.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2300cecb0 0x7fe23014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:24.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.881+0000 7fe22effd640 1 -- 172.21.15.84:0/862932600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218009b30 con 0x7fe2300b1ee0 2023-10-11T06:12:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.882+0000 7fe22effd640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2301f4210 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe218009e50 tx=0x7fe218004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.882+0000 7fe227fff640 1 -- 172.21.15.84:0/862932600 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218009510 con 0x7fe2300b1ee0 2023-10-11T06:12:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.882+0000 7fe227fff640 1 -- 172.21.15.84:0/862932600 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21803e450 con 0x7fe2300b1ee0 2023-10-11T06:12:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.882+0000 7fe227fff640 1 -- 172.21.15.84:0/862932600 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21803e800 con 0x7fe2300b1ee0 2023-10-11T06:12:24.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.882+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2300c9cd0 con 0x7fe2300b1ee0 2023-10-11T06:12:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.882+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2300b7330 con 0x7fe2300b1ee0 2023-10-11T06:12:24.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.883+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2300b74c0 con 0x7fe2300b1ee0 2023-10-11T06:12:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.884+0000 7fe227fff640 1 -- 172.21.15.84:0/862932600 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe21803e960 con 0x7fe2300b1ee0 2023-10-11T06:12:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.885+0000 7fe227fff640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe21006a830 0x7fe21006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:24.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.885+0000 7fe227fff640 1 -- 172.21.15.84:0/862932600 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe2180abea0 con 0x7fe2300b1ee0 2023-10-11T06:12:24.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.885+0000 7fe22f7fe640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe21006a830 0x7fe21006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:24.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.886+0000 7fe22f7fe640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe21006a830 0x7fe21006cce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe220006e20 tx=0x7fe220008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:24.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:24.889+0000 7fe227fff640 1 -- 172.21.15.84:0/862932600 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe218077040 con 0x7fe2300b1ee0 2023-10-11T06:12:25.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.075+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe23004e890 con 0x7fe2300b1ee0 2023-10-11T06:12:25.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.076+0000 7fe227fff640 1 -- 172.21.15.84:0/862932600 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe21807a660 con 0x7fe2300b1ee0 2023-10-11T06:12:25.077 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:25.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe21006a830 msgr2=0x7fe21006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe21006a830 0x7fe21006cce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe220006e20 tx=0x7fe220008040 comp rx=0 tx=0).stop 2023-10-11T06:12:25.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 msgr2=0x7fe2301f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2301f4210 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe218009e50 tx=0x7fe218004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:25.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 shutdown_connections 2023-10-11T06:12:25.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe21006a830 0x7fe21006cce0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2300cecb0 0x7fe23014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2300b1ee0 0x7fe2301f4210 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 --2- 172.21.15.84:0/862932600 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2300b1b10 0x7fe2300b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 >> 172.21.15.84:0/862932600 conn(0x7fe23009bc20 msgr2=0x7fe2300b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:25.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 shutdown_connections 2023-10-11T06:12:25.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.078+0000 7fe234e54640 1 -- 172.21.15.84:0/862932600 wait complete. 2023-10-11T06:12:25.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:12:25.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.303+0000 7f6450947640 1 Processor -- start 2023-10-11T06:12:25.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.303+0000 7f6450947640 1 -- start start 2023-10-11T06:12:25.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f6450947640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f644c06fd80 0x7f644c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f6450947640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c15da50 con 0x7f644c0b3e70 2023-10-11T06:12:25.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f6450947640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c0cad10 con 0x7f644c0ceae0 2023-10-11T06:12:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f6450947640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c15dca0 con 0x7f644c06fd80 2023-10-11T06:12:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f644b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f644c0b3e70 0x7f644c0ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37178/0 (socket says 172.21.15.84:37178) 2023-10-11T06:12:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f644b7fe640 1 -- 172.21.15.84:0/1623975110 learned_addr learned my addr 172.21.15.84:0/1623975110 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f644bfff640 1 --2- 172.21.15.84:0/1623975110 >> v2:172.21.15.84:3300/0 conn(0x7f644c06fd80 0x7f644c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f644bfff640 1 -- 172.21.15.84:0/1623975110 >> v1:172.21.15.99:6789/0 conn(0x7f644c0ceae0 legacy=0x7f644c15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:25.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f644bfff640 1 -- 172.21.15.84:0/1623975110 >> v1:172.21.15.84:6789/0 conn(0x7f644c0b3e70 legacy=0x7f644c0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f644bfff640 1 -- 172.21.15.84:0/1623975110 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c15df80 con 0x7f644c06fd80 2023-10-11T06:12:25.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.305+0000 7f644a7fc640 1 -- 172.21.15.84:0/1623975110 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44224828 0 0) 0x7f644c15da50 con 0x7f644c0b3e70 2023-10-11T06:12:25.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.306+0000 7f644bfff640 1 --2- 172.21.15.84:0/1623975110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c06fd80 0x7f644c15ab80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64440097a0 tx=0x7f644402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7fe3b82c80b5cd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:25.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f644a7fc640 1 -- 172.21.15.84:0/1623975110 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444031830 con 0x7f644c06fd80 2023-10-11T06:12:25.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f644a7fc640 1 -- 172.21.15.84:0/1623975110 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444031990 con 0x7f644c06fd80 2023-10-11T06:12:25.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f644a7fc640 1 -- 172.21.15.84:0/1623975110 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444031d20 con 0x7f644c06fd80 2023-10-11T06:12:25.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f6450947640 1 -- 172.21.15.84:0/1623975110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c06fd80 msgr2=0x7f644c15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f6450947640 1 --2- 172.21.15.84:0/1623975110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c06fd80 0x7f644c15ab80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64440097a0 tx=0x7f644402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f6450947640 1 -- 172.21.15.84:0/1623975110 shutdown_connections 2023-10-11T06:12:25.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f6450947640 1 --2- 172.21.15.84:0/1623975110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c06fd80 0x7f644c15ab80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.307+0000 7f6450947640 1 -- 172.21.15.84:0/1623975110 >> 172.21.15.84:0/1623975110 conn(0x7f644c069c70 msgr2=0x7f644c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:25.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.308+0000 7f6450947640 1 -- 172.21.15.84:0/1623975110 shutdown_connections 2023-10-11T06:12:25.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.308+0000 7f6450947640 1 -- 172.21.15.84:0/1623975110 wait complete. 2023-10-11T06:12:25.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.309+0000 7f6450947640 1 Processor -- start 2023-10-11T06:12:25.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.309+0000 7f6450947640 1 -- start start 2023-10-11T06:12:25.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.310+0000 7f6450947640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06fd80 0x7f644c14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.310+0000 7f6450947640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f644c0b3e70 0x7f644c0c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.310+0000 7f6450947640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c0ceae0 0x7f644c14dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.310+0000 7f6450947640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0c5a50 con 0x7f644c0ceae0 2023-10-11T06:12:25.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.310+0000 7f6450947640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0c5c40 con 0x7f644c06fd80 2023-10-11T06:12:25.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.310+0000 7f6450947640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0c5f20 con 0x7f644c0b3e70 2023-10-11T06:12:25.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c0ceae0 0x7f644c14dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c0ceae0 0x7f644c14dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50822/0 (socket says 172.21.15.84:50822) 2023-10-11T06:12:25.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 -- 172.21.15.84:0/1491712145 learned_addr learned my addr 172.21.15.84:0/1491712145 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:25.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644b7fe640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06fd80 0x7f644c14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644affd640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f644c0b3e70 0x7f644c0c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 -- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f644c0b3e70 msgr2=0x7f644c0c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f644c0b3e70 0x7f644c0c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 -- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06fd80 msgr2=0x7f644c14e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06fd80 0x7f644c14e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644bfff640 1 -- 172.21.15.84:0/1491712145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444009400 con 0x7f644c0ceae0 2023-10-11T06:12:25.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644affd640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f644c0b3e70 0x7f644c0c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:25.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.311+0000 7f644b7fe640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06fd80 0x7f644c14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:25.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.312+0000 7f644bfff640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c0ceae0 0x7f644c14dcd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f64440098c0 tx=0x7f64440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:25.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.312+0000 7f6433fff640 1 -- 172.21.15.84:0/1491712145 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444004230 con 0x7f644c0ceae0 2023-10-11T06:12:25.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.312+0000 7f6433fff640 1 -- 172.21.15.84:0/1491712145 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444004390 con 0x7f644c0ceae0 2023-10-11T06:12:25.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.312+0000 7f6433fff640 1 -- 172.21.15.84:0/1491712145 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644403a920 con 0x7f644c0ceae0 2023-10-11T06:12:25.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.312+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c0c6200 con 0x7f644c0ceae0 2023-10-11T06:12:25.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.312+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c0c67c0 con 0x7f644c0ceae0 2023-10-11T06:12:25.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.313+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c04e890 con 0x7f644c0ceae0 2023-10-11T06:12:25.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.314+0000 7f6433fff640 1 -- 172.21.15.84:0/1491712145 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f644403aa80 con 0x7f644c0ceae0 2023-10-11T06:12:25.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.315+0000 7f6433fff640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f643806a830 0x7f643806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.315+0000 7f6433fff640 1 -- 172.21.15.84:0/1491712145 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f64440ab840 con 0x7f644c0ceae0 2023-10-11T06:12:25.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.315+0000 7f644b7fe640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f643806a830 0x7f643806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.316+0000 7f644b7fe640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f643806a830 0x7f643806cce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f643c0012e0 tx=0x7f643c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:25.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.319+0000 7f6433fff640 1 -- 172.21.15.84:0/1491712145 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f64440769e0 con 0x7f644c0ceae0 2023-10-11T06:12:25.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.548+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f644c0cb510 con 0x7f644c0ceae0 2023-10-11T06:12:25.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.549+0000 7f6433fff640 1 -- 172.21.15.84:0/1491712145 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7f644407a000 con 0x7f644c0ceae0 2023-10-11T06:12:25.551 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:25.551 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:12:25.551 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:12:25.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f643806a830 msgr2=0x7f643806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f643806a830 0x7f643806cce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f643c0012e0 tx=0x7f643c009040 comp rx=0 tx=0).stop 2023-10-11T06:12:25.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c0ceae0 msgr2=0x7f644c14dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c0ceae0 0x7f644c14dcd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f64440098c0 tx=0x7f64440046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 shutdown_connections 2023-10-11T06:12:25.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f643806a830 0x7f643806cce0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f644c0ceae0 0x7f644c14dcd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f644c0b3e70 0x7f644c0c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 --2- 172.21.15.84:0/1491712145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06fd80 0x7f644c14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 >> 172.21.15.84:0/1491712145 conn(0x7f644c069c70 msgr2=0x7f644c0c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:25.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 shutdown_connections 2023-10-11T06:12:25.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.552+0000 7f6450947640 1 -- 172.21.15.84:0/1491712145 wait complete. 2023-10-11T06:12:25.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:25.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.788+0000 7f0878bbb640 1 Processor -- start 2023-10-11T06:12:25.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.789+0000 7f0878bbb640 1 -- start start 2023-10-11T06:12:25.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.790+0000 7f0878bbb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08740ca300 0x7f08740cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.790+0000 7f0878bbb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f08740c9360 0x7f08740ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.790+0000 7f0878bbb640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0874071a90 con 0x7f087415d4f0 2023-10-11T06:12:25.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.790+0000 7f0878bbb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0874072310 con 0x7f08740ca300 2023-10-11T06:12:25.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.790+0000 7f0878bbb640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0874070280 con 0x7f08740c9360 2023-10-11T06:12:25.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f086bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08740ca300 0x7f08740cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f086bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08740ca300 0x7f08740cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50826/0 (socket says 172.21.15.84:50826) 2023-10-11T06:12:25.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f086bfff640 1 -- 172.21.15.84:0/963423155 learned_addr learned my addr 172.21.15.84:0/963423155 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:25.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f0872d9d640 1 --2- 172.21.15.84:0/963423155 >> v2:172.21.15.99:3300/0 conn(0x7f08740c9360 0x7f08740ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f086bfff640 1 -- 172.21.15.84:0/963423155 >> v2:172.21.15.99:3300/0 conn(0x7f08740c9360 msgr2=0x7f08740ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:25.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f086bfff640 1 --2- 172.21.15.84:0/963423155 >> v2:172.21.15.99:3300/0 conn(0x7f08740c9360 0x7f08740ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f086bfff640 1 -- 172.21.15.84:0/963423155 >> v1:172.21.15.99:6789/0 conn(0x7f087415d4f0 legacy=0x7f08740c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.791+0000 7f086bfff640 1 -- 172.21.15.84:0/963423155 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0874071cc0 con 0x7f08740ca300 2023-10-11T06:12:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.792+0000 7f086bfff640 1 --2- 172.21.15.84:0/963423155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08740ca300 0x7f08740cb540 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08740b4740 tx=0x7f085c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242242abfa05f215 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.792+0000 7f0871d9b640 1 -- 172.21.15.84:0/963423155 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c031870 con 0x7f08740ca300 2023-10-11T06:12:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0871d9b640 1 -- 172.21.15.84:0/963423155 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c0319d0 con 0x7f08740ca300 2023-10-11T06:12:25.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0871d9b640 1 -- 172.21.15.84:0/963423155 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c031d60 con 0x7f08740ca300 2023-10-11T06:12:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 -- 172.21.15.84:0/963423155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08740ca300 msgr2=0x7f08740cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 --2- 172.21.15.84:0/963423155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08740ca300 0x7f08740cb540 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08740b4740 tx=0x7f085c02b910 comp rx=0 tx=0).stop 2023-10-11T06:12:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 -- 172.21.15.84:0/963423155 shutdown_connections 2023-10-11T06:12:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 --2- 172.21.15.84:0/963423155 >> v2:172.21.15.99:3300/0 conn(0x7f08740c9360 0x7f08740ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 --2- 172.21.15.84:0/963423155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08740ca300 0x7f08740cb540 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 -- 172.21.15.84:0/963423155 >> 172.21.15.84:0/963423155 conn(0x7f0874069c70 msgr2=0x7f087406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 -- 172.21.15.84:0/963423155 shutdown_connections 2023-10-11T06:12:25.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.793+0000 7f0878bbb640 1 -- 172.21.15.84:0/963423155 wait complete. 2023-10-11T06:12:25.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.795+0000 7f0878bbb640 1 Processor -- start 2023-10-11T06:12:25.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.795+0000 7f0878bbb640 1 -- start start 2023-10-11T06:12:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0878bbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08740c9360 0x7f0874151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0878bbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08740ca300 0x7f08740a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0878bbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f087415d4f0 0x7f08740a8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0878bbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740c60e0 con 0x7f087415d4f0 2023-10-11T06:12:25.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0878bbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740c63c0 con 0x7f08740ca300 2023-10-11T06:12:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0878bbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08740c66a0 con 0x7f08740c9360 2023-10-11T06:12:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0872d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f087415d4f0 0x7f08740a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0872d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f087415d4f0 0x7f08740a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50828/0 (socket says 172.21.15.84:50828) 2023-10-11T06:12:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.796+0000 7f0872d9d640 1 -- 172.21.15.84:0/3625650579 learned_addr learned my addr 172.21.15.84:0/3625650579 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:25.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f087259c640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08740c9360 0x7f0874151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f0872d9d640 1 -- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08740c9360 msgr2=0x7f0874151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f0872d9d640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08740c9360 0x7f0874151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f0872d9d640 1 -- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08740ca300 msgr2=0x7f08740a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f086bfff640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08740ca300 0x7f08740a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f0872d9d640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08740ca300 0x7f08740a83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:25.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f0872d9d640 1 -- 172.21.15.84:0/3625650579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c0094d0 con 0x7f087415d4f0 2023-10-11T06:12:25.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f0872d9d640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f087415d4f0 0x7f08740a8910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f086c007e60 tx=0x7f086c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:25.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f086affd640 1 -- 172.21.15.84:0/3625650579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c014420 con 0x7f087415d4f0 2023-10-11T06:12:25.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.797+0000 7f086affd640 1 -- 172.21.15.84:0/3625650579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c0145a0 con 0x7f087415d4f0 2023-10-11T06:12:25.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.798+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08740c6980 con 0x7f087415d4f0 2023-10-11T06:12:25.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.798+0000 7f086affd640 1 -- 172.21.15.84:0/3625650579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c014950 con 0x7f087415d4f0 2023-10-11T06:12:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.798+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740c6f40 con 0x7f087415d4f0 2023-10-11T06:12:25.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.799+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0874073ed0 con 0x7f087415d4f0 2023-10-11T06:12:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.800+0000 7f086affd640 1 -- 172.21.15.84:0/3625650579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f086c014ab0 con 0x7f087415d4f0 2023-10-11T06:12:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.801+0000 7f086affd640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085406a860 0x7f085406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:25.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.801+0000 7f086affd640 1 -- 172.21.15.84:0/3625650579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f086c089980 con 0x7f087415d4f0 2023-10-11T06:12:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.801+0000 7f087259c640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085406a860 0x7f085406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.802+0000 7f087259c640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085406a860 0x7f085406cd10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f08740ac870 tx=0x7f0864009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:25.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:25.805+0000 7f086affd640 1 -- 172.21.15.84:0/3625650579 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f086c054b20 con 0x7f087415d4f0 2023-10-11T06:12:26.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.000+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08740b09b0 con 0x7f087415d4f0 2023-10-11T06:12:26.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.003+0000 7f086affd640 1 -- 172.21.15.84:0/3625650579 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f086c058140 con 0x7f087415d4f0 2023-10-11T06:12:26.005 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:26.006 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:26.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085406a860 msgr2=0x7f085406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085406a860 0x7f085406cd10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f08740ac870 tx=0x7f0864009040 comp rx=0 tx=0).stop 2023-10-11T06:12:26.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f087415d4f0 msgr2=0x7f08740a8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f087415d4f0 0x7f08740a8910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f086c007e60 tx=0x7f086c00e4b0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 shutdown_connections 2023-10-11T06:12:26.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f085406a860 0x7f085406cd10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f087415d4f0 0x7f08740a8910 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08740ca300 0x7f08740a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 --2- 172.21.15.84:0/3625650579 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08740c9360 0x7f0874151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 >> 172.21.15.84:0/3625650579 conn(0x7f0874069c70 msgr2=0x7f08740b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:26.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 shutdown_connections 2023-10-11T06:12:26.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.006+0000 7f0878bbb640 1 -- 172.21.15.84:0/3625650579 wait complete. 2023-10-11T06:12:26.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T06:12:26.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:12:26.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.246+0000 7f0de261a640 1 Processor -- start 2023-10-11T06:12:26.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.246+0000 7f0de261a640 1 -- start start 2023-10-11T06:12:26.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.247+0000 7f0de261a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0ddc0cb5b0 0x7f0ddc0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.247+0000 7f0de261a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc0b6830 con 0x7f0ddc0ced80 2023-10-11T06:12:26.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.247+0000 7f0de261a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc0cf500 con 0x7f0ddc15a710 2023-10-11T06:12:26.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.247+0000 7f0de261a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc0cd2b0 con 0x7f0ddc0cb5b0 2023-10-11T06:12:26.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.247+0000 7f0de0bb7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0ddc0ced80 0x7f0ddc0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37192/0 (socket says 172.21.15.84:37192) 2023-10-11T06:12:26.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.247+0000 7f0de0bb7640 1 -- 172.21.15.84:0/620514538 learned_addr learned my addr 172.21.15.84:0/620514538 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:26.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392480192 0 0) 0x7f0ddc0b6830 con 0x7f0ddc0ced80 2023-10-11T06:12:26.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc8003520 con 0x7f0ddc0ced80 2023-10-11T06:12:26.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddb7fe640 1 --2- 172.21.15.84:0/620514538 >> v2:172.21.15.99:3300/0 conn(0x7f0ddc0cb5b0 0x7f0ddc0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985191549 0 0) 0x7f0dc8003520 con 0x7f0ddc0ced80 2023-10-11T06:12:26.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 >> v2:172.21.15.99:3300/0 conn(0x7f0ddc0cb5b0 msgr2=0x7f0ddc0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:26.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 --2- 172.21.15.84:0/620514538 >> v2:172.21.15.99:3300/0 conn(0x7f0ddc0cb5b0 0x7f0ddc0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 >> v1:172.21.15.99:6789/0 conn(0x7f0ddc15a710 legacy=0x7f0ddc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc0cfcd0 con 0x7f0ddc0ced80 2023-10-11T06:12:26.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f0dd4001670 con 0x7f0ddc0ced80 2023-10-11T06:12:26.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 >> v1:172.21.15.84:6789/0 conn(0x7f0ddc0ced80 legacy=0x7f0ddc0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8003970 0x7f0dc8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dc80047c0 0x7f0dc8004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dc8005170 0x7f0dc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0cd2b0 con 0x7f0dc8005170 2023-10-11T06:12:26.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc800c100 con 0x7f0dc8003970 2023-10-11T06:12:26.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.248+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc800c2c0 con 0x7f0dc80047c0 2023-10-11T06:12:26.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0de0bb7640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dc8005170 0x7f0dc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0ddb7fe640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dc80047c0 0x7f0dc8004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0ddbfff640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8003970 0x7f0dc8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0de0bb7640 1 -- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dc80047c0 msgr2=0x7f0dc8004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:26.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0de0bb7640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dc80047c0 0x7f0dc8004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0de0bb7640 1 -- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8003970 msgr2=0x7f0dc8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:26.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0de0bb7640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8003970 0x7f0dc8003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0de0bb7640 1 -- 172.21.15.84:0/620514538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc8003520 con 0x7f0dc8005170 2023-10-11T06:12:26.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.249+0000 7f0ddb7fe640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dc80047c0 0x7f0dc8004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:26.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.250+0000 7f0de0bb7640 1 -- 172.21.15.84:0/620514538 reap_dead start 2023-10-11T06:12:26.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.250+0000 7f0de0bb7640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dc8005170 0x7f0dc800b980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400c210 tx=0x7f0dc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd3e73ec496746f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:26.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.250+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4005ad0 con 0x7f0dc8005170 2023-10-11T06:12:26.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.250+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd4005c30 con 0x7f0dc8005170 2023-10-11T06:12:26.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.250+0000 7f0ddaffd640 1 -- 172.21.15.84:0/620514538 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd402c890 con 0x7f0dc8005170 2023-10-11T06:12:26.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.251+0000 7f0de261a640 1 -- 172.21.15.84:0/620514538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dc8005170 msgr2=0x7f0dc800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.251+0000 7f0de261a640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dc8005170 0x7f0dc800b980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400c210 tx=0x7f0dc80057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.251+0000 7f0de261a640 1 -- 172.21.15.84:0/620514538 shutdown_connections 2023-10-11T06:12:26.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.251+0000 7f0de261a640 1 --2- 172.21.15.84:0/620514538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0dc8005170 0x7f0dc800b980 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.251+0000 7f0de261a640 1 -- 172.21.15.84:0/620514538 >> 172.21.15.84:0/620514538 conn(0x7f0ddc06e2a0 msgr2=0x7f0ddc06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.251+0000 7f0de261a640 1 -- 172.21.15.84:0/620514538 shutdown_connections 2023-10-11T06:12:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.251+0000 7f0de261a640 1 -- 172.21.15.84:0/620514538 wait complete. 2023-10-11T06:12:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.253+0000 7f0de261a640 1 Processor -- start 2023-10-11T06:12:26.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.253+0000 7f0de261a640 1 -- start start 2023-10-11T06:12:26.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0de261a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8005170 0x7f0ddc149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0de261a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0ca4e0 0x7f0ddc0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0de261a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ddc0c9b30 0x7f0ddc0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0de261a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0c9710 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0de261a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0cac50 con 0x7f0dc8005170 2023-10-11T06:12:26.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0de261a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0ca970 con 0x7f0ddc0c9b30 2023-10-11T06:12:26.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0ca4e0 0x7f0ddc0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0ddb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0ca4e0 0x7f0ddc0c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50844/0 (socket says 172.21.15.84:50844) 2023-10-11T06:12:26.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.254+0000 7f0ddb7fe640 1 -- 172.21.15.84:0/1265561507 learned_addr learned my addr 172.21.15.84:0/1265561507 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:26.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0ddb7fe640 1 -- 172.21.15.84:0/1265561507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ddc0c9b30 msgr2=0x7f0ddc0d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0ddb7fe640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ddc0c9b30 0x7f0ddc0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0ddb7fe640 1 -- 172.21.15.84:0/1265561507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8005170 msgr2=0x7f0ddc149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0ddb7fe640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8005170 0x7f0ddc149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0ddb7fe640 1 -- 172.21.15.84:0/1265561507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc0c9050 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0ddb7fe640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0ca4e0 0x7f0ddc0c7b80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e810 tx=0x7f0dc400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:26.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0dd8ff9640 1 -- 172.21.15.84:0/1265561507 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400c6d0 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0dd8ff9640 1 -- 172.21.15.84:0/1265561507 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc400c830 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc0c9330 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0dd8ff9640 1 -- 172.21.15.84:0/1265561507 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400cbe0 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.255+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc0d0bf0 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.257+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc073ed0 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.258+0000 7f0dd8ff9640 1 -- 172.21.15.84:0/1265561507 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0dc400cd40 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.260+0000 7f0dd8ff9640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0dbc06a860 0x7f0dbc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.260+0000 7f0dd8ff9640 1 -- 172.21.15.84:0/1265561507 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0dc4088dc0 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.260+0000 7f0ddbfff640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0dbc06a860 0x7f0dbc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.261+0000 7f0ddbfff640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0dbc06a860 0x7f0dbc06cd10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0012e0 tx=0x7f0dcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:26.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.263+0000 7f0dd8ff9640 1 -- 172.21.15.84:0/1265561507 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0dc4053ff0 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.490+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ddc06e710 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.491+0000 7f0dd8ff9640 1 -- 172.21.15.84:0/1265561507 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7f0dc4057610 con 0x7f0ddc0ca4e0 2023-10-11T06:12:26.493 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:26.493 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:12:26.494 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:12:26.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0dbc06a860 msgr2=0x7f0dbc06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0dbc06a860 0x7f0dbc06cd10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0012e0 tx=0x7f0dcc009040 comp rx=0 tx=0).stop 2023-10-11T06:12:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0ca4e0 msgr2=0x7f0ddc0c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0ca4e0 0x7f0ddc0c7b80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e810 tx=0x7f0dc400ecd0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 shutdown_connections 2023-10-11T06:12:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0dbc06a860 0x7f0dbc06cd10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ddc0c9b30 0x7f0ddc0d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0ca4e0 0x7f0ddc0c7b80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 --2- 172.21.15.84:0/1265561507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc8005170 0x7f0ddc149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 >> 172.21.15.84:0/1265561507 conn(0x7f0ddc06e2a0 msgr2=0x7f0ddc0b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:26.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 shutdown_connections 2023-10-11T06:12:26.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.495+0000 7f0de261a640 1 -- 172.21.15.84:0/1265561507 wait complete. 2023-10-11T06:12:26.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:12:26.570 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:12:26.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:12:26.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.723+0000 7f74fb1f0640 1 Processor -- start 2023-10-11T06:12:26.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.724+0000 7f74fb1f0640 1 -- start start 2023-10-11T06:12:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74fb1f0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74f40adb90 0x7f74f40caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74fb1f0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f74f40cd300 0x7f74f40cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74fb1f0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f40cb960 con 0x7f74f40ce0d0 2023-10-11T06:12:26.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74fb1f0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f415a340 con 0x7f74f40adb90 2023-10-11T06:12:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74fb1f0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f40ce4a0 con 0x7f74f40cd300 2023-10-11T06:12:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74ebfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74f40adb90 0x7f74f40caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74ebfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74f40adb90 0x7f74f40caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50860/0 (socket says 172.21.15.84:50860) 2023-10-11T06:12:26.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.725+0000 7f74ebfff640 1 -- 172.21.15.84:0/2876256820 learned_addr learned my addr 172.21.15.84:0/2876256820 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:26.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.726+0000 7f74ebfff640 1 -- 172.21.15.84:0/2876256820 >> v2:172.21.15.99:3300/0 conn(0x7f74f40cd300 msgr2=0x7f74f40cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:26.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.726+0000 7f74ebfff640 1 --2- 172.21.15.84:0/2876256820 >> v2:172.21.15.99:3300/0 conn(0x7f74f40cd300 0x7f74f40cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.726+0000 7f74ebfff640 1 -- 172.21.15.84:0/2876256820 >> v1:172.21.15.99:6789/0 conn(0x7f74f40ce0d0 legacy=0x7f74f40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:26.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.726+0000 7f74ebfff640 1 -- 172.21.15.84:0/2876256820 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f41593f0 con 0x7f74f40adb90 2023-10-11T06:12:26.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.727+0000 7f74ebfff640 1 --2- 172.21.15.84:0/2876256820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40adb90 0x7f74f40caee0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f74f406fd30 tx=0x7f74e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f114eac6e4337bf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.727+0000 7f74eb7fe640 1 -- 172.21.15.84:0/2876256820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4031870 con 0x7f74f40adb90 2023-10-11T06:12:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.727+0000 7f74eb7fe640 1 -- 172.21.15.84:0/2876256820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e40319d0 con 0x7f74f40adb90 2023-10-11T06:12:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.727+0000 7f74eb7fe640 1 -- 172.21.15.84:0/2876256820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4031d60 con 0x7f74f40adb90 2023-10-11T06:12:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 -- 172.21.15.84:0/2876256820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40adb90 msgr2=0x7f74f40caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/2876256820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40adb90 0x7f74f40caee0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f74f406fd30 tx=0x7f74e402b910 comp rx=0 tx=0).stop 2023-10-11T06:12:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 -- 172.21.15.84:0/2876256820 shutdown_connections 2023-10-11T06:12:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/2876256820 >> v2:172.21.15.99:3300/0 conn(0x7f74f40cd300 0x7f74f40cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/2876256820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40adb90 0x7f74f40caee0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 -- 172.21.15.84:0/2876256820 >> 172.21.15.84:0/2876256820 conn(0x7f74f4069c70 msgr2=0x7f74f406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:26.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 -- 172.21.15.84:0/2876256820 shutdown_connections 2023-10-11T06:12:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.728+0000 7f74fb1f0640 1 -- 172.21.15.84:0/2876256820 wait complete. 2023-10-11T06:12:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.730+0000 7f74fb1f0640 1 Processor -- start 2023-10-11T06:12:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.730+0000 7f74fb1f0640 1 -- start start 2023-10-11T06:12:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74fb1f0640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74f40adb90 0x7f74f40b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74fb1f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40cd300 0x7f74f40b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74fb1f0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f40ce0d0 0x7f74f41f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74fb1f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40c2960 con 0x7f74f40cd300 2023-10-11T06:12:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74fb1f0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40b3df0 con 0x7f74f40ce0d0 2023-10-11T06:12:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74fb1f0640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40b31e0 con 0x7f74f40adb90 2023-10-11T06:12:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40cd300 0x7f74f40b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40cd300 0x7f74f40b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50864/0 (socket says 172.21.15.84:50864) 2023-10-11T06:12:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.731+0000 7f74ebfff640 1 -- 172.21.15.84:0/1938156466 learned_addr learned my addr 172.21.15.84:0/1938156466 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74ebfff640 1 -- 172.21.15.84:0/1938156466 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74f40adb90 msgr2=0x7f74f40b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74ebfff640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74f40adb90 0x7f74f40b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74ebfff640 1 -- 172.21.15.84:0/1938156466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f40ce0d0 msgr2=0x7f74f41f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74ebfff640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f40ce0d0 0x7f74f41f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74ebfff640 1 -- 172.21.15.84:0/1938156466 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e40094d0 con 0x7f74f40cd300 2023-10-11T06:12:26.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74ebfff640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40cd300 0x7f74f40b2af0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74e4000c00 tx=0x7f74e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:26.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74e97fa640 1 -- 172.21.15.84:0/1938156466 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4031840 con 0x7f74f40cd300 2023-10-11T06:12:26.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74e97fa640 1 -- 172.21.15.84:0/1938156466 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e40443f0 con 0x7f74f40cd300 2023-10-11T06:12:26.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.732+0000 7f74e97fa640 1 -- 172.21.15.84:0/1938156466 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40447a0 con 0x7f74f40cd300 2023-10-11T06:12:26.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.733+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f40b34c0 con 0x7f74f40cd300 2023-10-11T06:12:26.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.733+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f40b3a80 con 0x7f74f40cd300 2023-10-11T06:12:26.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.734+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4073ed0 con 0x7f74f40cd300 2023-10-11T06:12:26.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.735+0000 7f74e97fa640 1 -- 172.21.15.84:0/1938156466 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f74e4044900 con 0x7f74f40cd300 2023-10-11T06:12:26.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.736+0000 7f74e97fa640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74e006a830 0x7f74e006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:26.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.736+0000 7f74e97fa640 1 -- 172.21.15.84:0/1938156466 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f74e402d040 con 0x7f74f40cd300 2023-10-11T06:12:26.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.736+0000 7f74f8f8c640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74e006a830 0x7f74e006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:26.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.737+0000 7f74f8f8c640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74e006a830 0x7f74e006cce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74f0007fd0 tx=0x7f74f0007d80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:26.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.740+0000 7f74e97fa640 1 -- 172.21.15.84:0/1938156466 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f74e4076b80 con 0x7f74f40cd300 2023-10-11T06:12:26.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.969+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f4159c50 con 0x7f74f40cd300 2023-10-11T06:12:26.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.970+0000 7f74e97fa640 1 -- 172.21.15.84:0/1938156466 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4633 (secure 0 0 0) 0x7f74e4035090 con 0x7f74f40cd300 2023-10-11T06:12:26.971 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:26.972 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/2130526167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2130526167},{"type":"v1","addr":"172.21.15.99:6834","nonce":2130526167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/349810845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":349810845},{"type":"v1","addr":"172.21.15.84:6836","nonce":349810845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/65906626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":65906626},{"type":"v1","addr":"172.21.15.84:6837","nonce":65906626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:11:13.667179+0000","modified":"2023-10-11T06:11:16.824257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4314},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4314":{"gid":4314,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/2350376542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2350376542},{"type":"v1","addr":"172.21.15.99:6835","nonce":2350376542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-11T06:12:26.972 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-10-11T06:12:26.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.972+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74e006a830 msgr2=0x7f74e006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.972+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74e006a830 0x7f74e006cce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74f0007fd0 tx=0x7f74f0007d80 comp rx=0 tx=0).stop 2023-10-11T06:12:26.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.972+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40cd300 msgr2=0x7f74f40b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:26.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.972+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40cd300 0x7f74f40b2af0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74e4000c00 tx=0x7f74e40046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 shutdown_connections 2023-10-11T06:12:26.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74e006a830 0x7f74e006cce0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74f40ce0d0 0x7f74f41f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40cd300 0x7f74f40b2af0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 --2- 172.21.15.84:0/1938156466 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74f40adb90 0x7f74f40b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:26.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 >> 172.21.15.84:0/1938156466 conn(0x7f74f4069c70 msgr2=0x7f74f40ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:26.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 shutdown_connections 2023-10-11T06:12:26.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:26.973+0000 7f74fb1f0640 1 -- 172.21.15.84:0/1938156466 wait complete. 2023-10-11T06:12:26.993 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:27.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.254+0000 7f130144d640 1 Processor -- start 2023-10-11T06:12:27.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.254+0000 7f130144d640 1 -- start start 2023-10-11T06:12:27.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f130144d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 0x7f12fc15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f130144d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc15da50 con 0x7f12fc0b3e70 2023-10-11T06:12:27.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f130144d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0cad10 con 0x7f12fc0ceae0 2023-10-11T06:12:27.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f130144d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc15dca0 con 0x7f12fc06fd80 2023-10-11T06:12:27.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f12fbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f12fc0b3e70 0x7f12fc0ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37208/0 (socket says 172.21.15.84:37208) 2023-10-11T06:12:27.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f12fbfff640 1 -- 172.21.15.84:0/3901546331 learned_addr learned my addr 172.21.15.84:0/3901546331 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:27.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460914952 0 0) 0x7f12fc15da50 con 0x7f12fc0b3e70 2023-10-11T06:12:27.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.256+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e8003520 con 0x7f12fc0b3e70 2023-10-11T06:12:27.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f1300c4c640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 0x7f12fc15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1387604342 0 0) 0x7f12e8003520 con 0x7f12fc0b3e70 2023-10-11T06:12:27.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 msgr2=0x7f12fc15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:27.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 0x7f12fc15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 >> v1:172.21.15.99:6789/0 conn(0x7f12fc0ceae0 legacy=0x7f12fc15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc15df80 con 0x7f12fc0b3e70 2023-10-11T06:12:27.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f12ec001670 con 0x7f12fc0b3e70 2023-10-11T06:12:27.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 >> v1:172.21.15.84:6789/0 conn(0x7f12fc0b3e70 legacy=0x7f12fc0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80039a0 0x7f12e8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 0x7f12e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12e8004d00 0x7f12e8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc15dca0 con 0x7f12e8004d00 2023-10-11T06:12:27.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e800bac0 con 0x7f12e80039a0 2023-10-11T06:12:27.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e800bc80 con 0x7f12fc06fd80 2023-10-11T06:12:27.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.257+0000 7f1300c4c640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12e8004d00 0x7f12e8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.258+0000 7f12fbfff640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80039a0 0x7f12e8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.258+0000 7f12f37fe640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 0x7f12e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.258+0000 7f1300c4c640 1 -- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 msgr2=0x7f12e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:27.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.258+0000 7f1300c4c640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc06fd80 0x7f12e80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.258+0000 7f1300c4c640 1 -- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80039a0 msgr2=0x7f12e8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:27.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.258+0000 7f1300c4c640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12e80039a0 0x7f12e8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.258+0000 7f1300c4c640 1 -- 172.21.15.84:0/3901546331 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8003520 con 0x7f12e8004d00 2023-10-11T06:12:27.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.259+0000 7f1300c4c640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12e8004d00 0x7f12e8005170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ba30 tx=0x7f12e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39109a067d9e9736 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:27.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.259+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4032630 con 0x7f12e8004d00 2023-10-11T06:12:27.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.259+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f4032790 con 0x7f12e8004d00 2023-10-11T06:12:27.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.259+0000 7f12fb7fe640 1 -- 172.21.15.84:0/3901546331 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4032b20 con 0x7f12e8004d00 2023-10-11T06:12:27.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.259+0000 7f130144d640 1 -- 172.21.15.84:0/3901546331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12e8004d00 msgr2=0x7f12e8005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.259+0000 7f130144d640 1 --2- 172.21.15.84:0/3901546331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12e8004d00 0x7f12e8005170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ba30 tx=0x7f12e80057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.260+0000 7f1300c4c640 1 -- 172.21.15.84:0/3901546331 reap_dead start 2023-10-11T06:12:27.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.260+0000 7f130144d640 1 -- 172.21.15.84:0/3901546331 shutdown_connections 2023-10-11T06:12:27.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.260+0000 7f130144d640 1 -- 172.21.15.84:0/3901546331 >> 172.21.15.84:0/3901546331 conn(0x7f12fc069c70 msgr2=0x7f12fc06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:27.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.260+0000 7f130144d640 1 -- 172.21.15.84:0/3901546331 shutdown_connections 2023-10-11T06:12:27.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.260+0000 7f130144d640 1 -- 172.21.15.84:0/3901546331 wait complete. 2023-10-11T06:12:27.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.261+0000 7f130144d640 1 Processor -- start 2023-10-11T06:12:27.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.262+0000 7f130144d640 1 -- start start 2023-10-11T06:12:27.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.262+0000 7f130144d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc0cd290 0x7f12fc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.262+0000 7f130144d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc0cd660 0x7f12fc0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.262+0000 7f130144d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0c8090 0x7f12fc154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.262+0000 7f130144d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc140840 con 0x7f12fc0c8090 2023-10-11T06:12:27.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f130144d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc140b20 con 0x7f12fc0cd290 2023-10-11T06:12:27.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f130144d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc140e00 con 0x7f12fc0cd660 2023-10-11T06:12:27.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0c8090 0x7f12fc154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f12fbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc0cd290 0x7f12fc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0c8090 0x7f12fc154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50876/0 (socket says 172.21.15.84:50876) 2023-10-11T06:12:27.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 -- 172.21.15.84:0/916633828 learned_addr learned my addr 172.21.15.84:0/916633828 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:27.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f12f37fe640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc0cd660 0x7f12fc0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 -- 172.21.15.84:0/916633828 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc0cd660 msgr2=0x7f12fc0c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc0cd660 0x7f12fc0c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 -- 172.21.15.84:0/916633828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc0cd290 msgr2=0x7f12fc0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc0cd290 0x7f12fc0b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.263+0000 7f1300c4c640 1 -- 172.21.15.84:0/916633828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc1410e0 con 0x7f12fc0c8090 2023-10-11T06:12:27.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.264+0000 7f1300c4c640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0c8090 0x7f12fc154d70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ba30 tx=0x7f12f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:27.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.264+0000 7f12f97fa640 1 -- 172.21.15.84:0/916633828 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4004020 con 0x7f12fc0c8090 2023-10-11T06:12:27.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.264+0000 7f12f97fa640 1 -- 172.21.15.84:0/916633828 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f4004180 con 0x7f12fc0c8090 2023-10-11T06:12:27.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.264+0000 7f12f97fa640 1 -- 172.21.15.84:0/916633828 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40044f0 con 0x7f12fc0c8090 2023-10-11T06:12:27.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.264+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc1413c0 con 0x7f12fc0c8090 2023-10-11T06:12:27.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.264+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc0b1810 con 0x7f12fc0c8090 2023-10-11T06:12:27.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.265+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc04e890 con 0x7f12fc0c8090 2023-10-11T06:12:27.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.266+0000 7f12f97fa640 1 -- 172.21.15.84:0/916633828 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f12f403b630 con 0x7f12fc0c8090 2023-10-11T06:12:27.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.267+0000 7f12f97fa640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f12dc06a860 0x7f12dc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.267+0000 7f12f97fa640 1 -- 172.21.15.84:0/916633828 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f12f4000e40 con 0x7f12fc0c8090 2023-10-11T06:12:27.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.267+0000 7f12fbfff640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f12dc06a860 0x7f12dc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.268+0000 7f12fbfff640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f12dc06a860 0x7f12dc06cd10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12ec005b30 tx=0x7f12ec007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:27.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.271+0000 7f12f97fa640 1 -- 172.21.15.84:0/916633828 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f12f4076170 con 0x7f12fc0c8090 2023-10-11T06:12:27.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.464+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12fc0cb280 con 0x7f12fc0c8090 2023-10-11T06:12:27.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.466+0000 7f12f97fa640 1 -- 172.21.15.84:0/916633828 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f12f4079790 con 0x7f12fc0c8090 2023-10-11T06:12:27.467 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:27.467 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:11:51.869623+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:11:12.075151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-11T06:11:12.661632+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.21","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:27.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.468+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f12dc06a860 msgr2=0x7f12dc06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.468+0000 7f130144d640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f12dc06a860 0x7f12dc06cd10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12ec005b30 tx=0x7f12ec007040 comp rx=0 tx=0).stop 2023-10-11T06:12:27.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.468+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0c8090 msgr2=0x7f12fc154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.468+0000 7f130144d640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0c8090 0x7f12fc154d70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ba30 tx=0x7f12f4004730 comp rx=0 tx=0).stop 2023-10-11T06:12:27.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.468+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 shutdown_connections 2023-10-11T06:12:27.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.468+0000 7f130144d640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f12dc06a860 0x7f12dc06cd10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.468+0000 7f130144d640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12fc0c8090 0x7f12fc154d70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.469+0000 7f130144d640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12fc0cd660 0x7f12fc0c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.469+0000 7f130144d640 1 --2- 172.21.15.84:0/916633828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc0cd290 0x7f12fc0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.469+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 >> 172.21.15.84:0/916633828 conn(0x7f12fc069c70 msgr2=0x7f12fc15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:27.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.469+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 shutdown_connections 2023-10-11T06:12:27.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.469+0000 7f130144d640 1 -- 172.21.15.84:0/916633828 wait complete. 2023-10-11T06:12:27.488 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T06:12:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.697+0000 7f3ddb234640 1 Processor -- start 2023-10-11T06:12:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.698+0000 7f3ddb234640 1 -- start start 2023-10-11T06:12:27.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3ddb234640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3dd40ca300 0x7f3dd40cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3ddb234640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3dd40c9360 0x7f3dd40ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3ddb234640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd4071a90 con 0x7f3dd415d4f0 2023-10-11T06:12:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3ddb234640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd4072310 con 0x7f3dd40c9360 2023-10-11T06:12:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3ddb234640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd4070280 con 0x7f3dd40ca300 2023-10-11T06:12:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3dd97d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3dd40c9360 0x7f3dd40ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3dd8fd0640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3dd415d4f0 0x7f3dd40c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:37222/0 (socket says 172.21.15.84:37222) 2023-10-11T06:12:27.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3dd97d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3dd40c9360 0x7f3dd40ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50892/0 (socket says 172.21.15.84:50892) 2023-10-11T06:12:27.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.699+0000 7f3dd8fd0640 1 -- 172.21.15.84:0/4044744390 learned_addr learned my addr 172.21.15.84:0/4044744390 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:27.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.700+0000 7f3dd3fff640 1 --2- 172.21.15.84:0/4044744390 >> v2:172.21.15.99:3300/0 conn(0x7f3dd40ca300 0x7f3dd40cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.700+0000 7f3dd97d1640 1 -- 172.21.15.84:0/4044744390 >> v2:172.21.15.99:3300/0 conn(0x7f3dd40ca300 msgr2=0x7f3dd40cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.700+0000 7f3dd97d1640 1 --2- 172.21.15.84:0/4044744390 >> v2:172.21.15.99:3300/0 conn(0x7f3dd40ca300 0x7f3dd40cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.700+0000 7f3dd97d1640 1 -- 172.21.15.84:0/4044744390 >> v1:172.21.15.84:6789/0 conn(0x7f3dd415d4f0 legacy=0x7f3dd40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.700+0000 7f3dd97d1640 1 -- 172.21.15.84:0/4044744390 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4071cc0 con 0x7f3dd40c9360 2023-10-11T06:12:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.701+0000 7f3dd97d1640 1 --2- 172.21.15.84:0/4044744390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40ce550 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40b0980 tx=0x7f3dcc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4608f7e2d7db1e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:27.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.701+0000 7f3dd37fe640 1 -- 172.21.15.84:0/4044744390 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc031830 con 0x7f3dd40c9360 2023-10-11T06:12:27.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.701+0000 7f3dd37fe640 1 -- 172.21.15.84:0/4044744390 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc031990 con 0x7f3dd40c9360 2023-10-11T06:12:27.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.701+0000 7f3dd37fe640 1 -- 172.21.15.84:0/4044744390 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc031d20 con 0x7f3dd40c9360 2023-10-11T06:12:27.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 -- 172.21.15.84:0/4044744390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 msgr2=0x7f3dd40ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 --2- 172.21.15.84:0/4044744390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40ce550 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40b0980 tx=0x7f3dcc02b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 -- 172.21.15.84:0/4044744390 shutdown_connections 2023-10-11T06:12:27.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 --2- 172.21.15.84:0/4044744390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40ce550 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 --2- 172.21.15.84:0/4044744390 >> v2:172.21.15.99:3300/0 conn(0x7f3dd40ca300 0x7f3dd40cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 -- 172.21.15.84:0/4044744390 >> 172.21.15.84:0/4044744390 conn(0x7f3dd4069c70 msgr2=0x7f3dd406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:27.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 -- 172.21.15.84:0/4044744390 shutdown_connections 2023-10-11T06:12:27.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.702+0000 7f3ddb234640 1 -- 172.21.15.84:0/4044744390 wait complete. 2023-10-11T06:12:27.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.704+0000 7f3ddb234640 1 Processor -- start 2023-10-11T06:12:27.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.704+0000 7f3ddb234640 1 -- start start 2023-10-11T06:12:27.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.705+0000 7f3ddb234640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.705+0000 7f3ddb234640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dd40ca300 0x7f3dd40aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.705+0000 7f3ddb234640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dd415d4f0 0x7f3dd40af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.705+0000 7f3ddb234640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4159950 con 0x7f3dd40c9360 2023-10-11T06:12:27.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.705+0000 7f3ddb234640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4159c30 con 0x7f3dd415d4f0 2023-10-11T06:12:27.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.705+0000 7f3ddb234640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4159f10 con 0x7f3dd40ca300 2023-10-11T06:12:27.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50900/0 (socket says 172.21.15.84:50900) 2023-10-11T06:12:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 -- 172.21.15.84:0/2056560382 learned_addr learned my addr 172.21.15.84:0/2056560382 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd3fff640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dd40ca300 0x7f3dd40aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd97d1640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dd415d4f0 0x7f3dd40af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 -- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dd40ca300 msgr2=0x7f3dd40aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dd40ca300 0x7f3dd40aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 -- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dd415d4f0 msgr2=0x7f3dd40af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dd415d4f0 0x7f3dd40af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd8fd0640 1 -- 172.21.15.84:0/2056560382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc009400 con 0x7f3dd40c9360 2023-10-11T06:12:27.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.706+0000 7f3dd3fff640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dd40ca300 0x7f3dd40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.707+0000 7f3dd8fd0640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40c4620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c800 tx=0x7f3dc400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.707+0000 7f3dd17fa640 1 -- 172.21.15.84:0/2056560382 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400a640 con 0x7f3dd40c9360 2023-10-11T06:12:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.707+0000 7f3dd17fa640 1 -- 172.21.15.84:0/2056560382 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400a7a0 con 0x7f3dd40c9360 2023-10-11T06:12:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.707+0000 7f3dd17fa640 1 -- 172.21.15.84:0/2056560382 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400ab70 con 0x7f3dd40c9360 2023-10-11T06:12:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.707+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd415a1f0 con 0x7f3dd40c9360 2023-10-11T06:12:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.707+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd415aab0 con 0x7f3dd40c9360 2023-10-11T06:12:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.708+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd4073ed0 con 0x7f3dd40c9360 2023-10-11T06:12:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.709+0000 7f3dd17fa640 1 -- 172.21.15.84:0/2056560382 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3dc400acd0 con 0x7f3dd40c9360 2023-10-11T06:12:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.710+0000 7f3dd17fa640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3dc006a830 0x7f3dc006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.710+0000 7f3dd17fa640 1 -- 172.21.15.84:0/2056560382 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3dc4088990 con 0x7f3dd40c9360 2023-10-11T06:12:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.710+0000 7f3dd3fff640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3dc006a830 0x7f3dc006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:27.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.711+0000 7f3dd3fff640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3dc006a830 0x7f3dc006cce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006fd0 tx=0x7f3dc8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:27.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.715+0000 7f3dd17fa640 1 -- 172.21.15.84:0/2056560382 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3dc4053b30 con 0x7f3dd40c9360 2023-10-11T06:12:27.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.944+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3dd40b09b0 con 0x7f3dd40c9360 2023-10-11T06:12:27.947 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-11T06:12:27.946+0000 7feb66d4a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T06:12:27.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.960+0000 7f3dd17fa640 1 -- 172.21.15.84:0/2056560382 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3dc4017020 con 0x7f3dd40c9360 2023-10-11T06:12:27.961 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T06:12:27.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3dc006a830 msgr2=0x7f3dc006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3dc006a830 0x7f3dc006cce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006fd0 tx=0x7f3dc8008040 comp rx=0 tx=0).stop 2023-10-11T06:12:27.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 msgr2=0x7f3dd40c4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:27.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40c4620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c800 tx=0x7f3dc400ccc0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 shutdown_connections 2023-10-11T06:12:27.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3dc006a830 0x7f3dc006cce0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dd415d4f0 0x7f3dd40af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dd40ca300 0x7f3dd40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 --2- 172.21.15.84:0/2056560382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c9360 0x7f3dd40c4620 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:27.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 >> 172.21.15.84:0/2056560382 conn(0x7f3dd4069c70 msgr2=0x7f3dd40b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:27.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 shutdown_connections 2023-10-11T06:12:27.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:27.963+0000 7f3ddb234640 1 -- 172.21.15.84:0/2056560382 wait complete. 2023-10-11T06:12:27.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T06:12:27.986 INFO:tasks.ceph.mds.d.smithi099.stderr: -3888> 2023-10-11T06:11:19.336+0000 7f9b2306b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T06:12:28.065 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2023-10-11T06:12:28.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.192+0000 7f8da681a640 1 Processor -- start 2023-10-11T06:12:28.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.192+0000 7f8da681a640 1 -- start start 2023-10-11T06:12:28.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.194+0000 7f8da681a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:28.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.194+0000 7f8da681a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8da00adb90 0x7f8da00caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:28.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.194+0000 7f8da681a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da01593f0 con 0x7f8da00cd300 2023-10-11T06:12:28.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.194+0000 7f8da681a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da015a340 con 0x7f8da00adb90 2023-10-11T06:12:28.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.194+0000 7f8da681a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00ce4a0 con 0x7f8da00ce0d0 2023-10-11T06:12:28.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.194+0000 7f8d977fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8da00adb90 0x7f8da00caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:28.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.194+0000 7f8d9ffff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:28.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.195+0000 7f8d977fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8da00adb90 0x7f8da00caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:46738/0 (socket says 172.21.15.84:46738) 2023-10-11T06:12:28.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.195+0000 7f8d977fe640 1 -- 172.21.15.84:0/284938928 learned_addr learned my addr 172.21.15.84:0/284938928 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:28.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.195+0000 7f8d9ffff640 1 -- 172.21.15.84:0/284938928 >> v2:172.21.15.99:3300/0 conn(0x7f8da00adb90 msgr2=0x7f8da00caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:28.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.195+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/284938928 >> v2:172.21.15.99:3300/0 conn(0x7f8da00adb90 0x7f8da00caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.196+0000 7f8d9ffff640 1 -- 172.21.15.84:0/284938928 >> v1:172.21.15.99:6789/0 conn(0x7f8da00cd300 legacy=0x7f8da00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:28.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.196+0000 7f8d9f7fe640 1 -- 172.21.15.84:0/284938928 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584655094 0 0) 0x7f8da01593f0 con 0x7f8da00cd300 2023-10-11T06:12:28.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.196+0000 7f8d9ffff640 1 -- 172.21.15.84:0/284938928 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da01596c0 con 0x7f8da00ce0d0 2023-10-11T06:12:28.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.196+0000 7f8d977fe640 1 --2- 172.21.15.84:0/284938928 >> v2:172.21.15.99:3300/0 conn(0x7f8da00adb90 0x7f8da00caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:28.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.197+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/284938928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c8cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000d050 tx=0x7f8d9002e100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d0ddbb1bd71f08 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:28.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.197+0000 7f8d9f7fe640 1 -- 172.21.15.84:0/284938928 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9002eaa0 con 0x7f8da00ce0d0 2023-10-11T06:12:28.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8d9f7fe640 1 -- 172.21.15.84:0/284938928 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9002ec00 con 0x7f8da00ce0d0 2023-10-11T06:12:28.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8d9f7fe640 1 -- 172.21.15.84:0/284938928 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90009540 con 0x7f8da00ce0d0 2023-10-11T06:12:28.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 -- 172.21.15.84:0/284938928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 msgr2=0x7f8da00c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:28.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 --2- 172.21.15.84:0/284938928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c8cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000d050 tx=0x7f8d9002e100 comp rx=0 tx=0).stop 2023-10-11T06:12:28.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 -- 172.21.15.84:0/284938928 shutdown_connections 2023-10-11T06:12:28.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 --2- 172.21.15.84:0/284938928 >> v2:172.21.15.99:3300/0 conn(0x7f8da00adb90 0x7f8da00caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 --2- 172.21.15.84:0/284938928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c8cc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 -- 172.21.15.84:0/284938928 >> 172.21.15.84:0/284938928 conn(0x7f8da0069c70 msgr2=0x7f8da006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:28.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 -- 172.21.15.84:0/284938928 shutdown_connections 2023-10-11T06:12:28.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.198+0000 7f8da681a640 1 -- 172.21.15.84:0/284938928 wait complete. 2023-10-11T06:12:28.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.201+0000 7f8da681a640 1 Processor -- start 2023-10-11T06:12:28.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.201+0000 7f8da681a640 1 -- start start 2023-10-11T06:12:28.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8da681a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00adb90 0x7f8da00c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:28.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8da681a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da00cd300 0x7f8da0147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:28.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8da681a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:28.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8da681a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da0147fe0 con 0x7f8da00adb90 2023-10-11T06:12:28.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8da681a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00c7250 con 0x7f8da00cd300 2023-10-11T06:12:28.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8da681a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00c7530 con 0x7f8da00ce0d0 2023-10-11T06:12:28.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8d9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00adb90 0x7f8da00c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:28.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8d9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00adb90 0x7f8da00c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50908/0 (socket says 172.21.15.84:50908) 2023-10-11T06:12:28.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.202+0000 7f8d9ffff640 1 -- 172.21.15.84:0/2021413814 learned_addr learned my addr 172.21.15.84:0/2021413814 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:28.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d977fe640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da00cd300 0x7f8da0147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:28.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d9ffff640 1 -- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 msgr2=0x7f8da00c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:28.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8da4db7640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:28.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d9ffff640 1 -- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da00cd300 msgr2=0x7f8da0147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:28.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da00cd300 0x7f8da0147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d9ffff640 1 -- 172.21.15.84:0/2021413814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880094d0 con 0x7f8da00adb90 2023-10-11T06:12:28.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d977fe640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da00cd300 0x7f8da0147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:28.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d9ffff640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00adb90 0x7f8da00c8010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d90038040 tx=0x7f8d90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:28.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.203+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/2021413814 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90040630 con 0x7f8da00adb90 2023-10-11T06:12:28.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.204+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/2021413814 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d900407b0 con 0x7f8da00adb90 2023-10-11T06:12:28.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.204+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9000bc80 con 0x7f8da00adb90 2023-10-11T06:12:28.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.204+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/2021413814 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90040b60 con 0x7f8da00adb90 2023-10-11T06:12:28.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.204+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da00c7af0 con 0x7f8da00adb90 2023-10-11T06:12:28.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.205+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da00c8550 con 0x7f8da00adb90 2023-10-11T06:12:28.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.206+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/2021413814 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8d90040cc0 con 0x7f8da00adb90 2023-10-11T06:12:28.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.207+0000 7f8d9d7fa640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8d8006a830 0x7f8d8006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:28.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.207+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/2021413814 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f8d900ac290 con 0x7f8da00adb90 2023-10-11T06:12:28.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.207+0000 7f8d977fe640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8d8006a830 0x7f8d8006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:28.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.208+0000 7f8d977fe640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8d8006a830 0x7f8d8006cce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8da00cddd0 tx=0x7f8d88008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:28.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.211+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/2021413814 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8d90077630 con 0x7f8da00adb90 2023-10-11T06:12:28.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.444+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8da0073ed0 con 0x7f8da00adb90 2023-10-11T06:12:28.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.965+0000 7f8d9d7fa640 1 -- 172.21.15.84:0/2021413814 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8d9007fe60 con 0x7f8da00adb90 2023-10-11T06:12:28.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8d8006a830 msgr2=0x7f8d8006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:28.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8d8006a830 0x7f8d8006cce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8da00cddd0 tx=0x7f8d88008040 comp rx=0 tx=0).stop 2023-10-11T06:12:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00adb90 msgr2=0x7f8da00c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00adb90 0x7f8da00c8010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d90038040 tx=0x7f8d90004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 shutdown_connections 2023-10-11T06:12:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8d8006a830 0x7f8d8006cce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da00ce0d0 0x7f8da00c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da00cd300 0x7f8da0147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 --2- 172.21.15.84:0/2021413814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00adb90 0x7f8da00c8010 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 >> 172.21.15.84:0/2021413814 conn(0x7f8da0069c70 msgr2=0x7f8da01589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 shutdown_connections 2023-10-11T06:12:28.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:28.969+0000 7f8da681a640 1 -- 172.21.15.84:0/2021413814 wait complete. 2023-10-11T06:12:28.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T06:12:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.217+0000 7f9dcf8d5640 1 Processor -- start 2023-10-11T06:12:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.218+0000 7f9dcf8d5640 1 -- start start 2023-10-11T06:12:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.219+0000 7f9dcf8d5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 0x7f9dc80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.219+0000 7f9dcf8d5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc8071380 con 0x7f9dc815a0b0 2023-10-11T06:12:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.219+0000 7f9dcf8d5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc806de10 con 0x7f9dc80c8d00 2023-10-11T06:12:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.219+0000 7f9dcf8d5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc806e0f0 con 0x7f9dc80ced30 2023-10-11T06:12:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.220+0000 7f9dcd671640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9dc815a0b0 0x7f9dc80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34574/0 (socket says 172.21.15.84:34574) 2023-10-11T06:12:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.220+0000 7f9dcd671640 1 -- 172.21.15.84:0/3319193483 learned_addr learned my addr 172.21.15.84:0/3319193483 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.220+0000 7f9dcce70640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 0x7f9dc80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.220+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154447138 0 0) 0x7f9dc8071380 con 0x7f9dc815a0b0 2023-10-11T06:12:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.220+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db4003520 con 0x7f9dc815a0b0 2023-10-11T06:12:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217529850 0 0) 0x7f9db4003520 con 0x7f9dc815a0b0 2023-10-11T06:12:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 msgr2=0x7f9dc80cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 0x7f9dc80cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 >> v1:172.21.15.99:6789/0 conn(0x7f9dc80c8d00 legacy=0x7f9dc806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8070dc0 con 0x7f9dc815a0b0 2023-10-11T06:12:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f9dc4001670 con 0x7f9dc815a0b0 2023-10-11T06:12:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 >> v1:172.21.15.84:6789/0 conn(0x7f9dc815a0b0 legacy=0x7f9dc80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9db4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 0x7f9db40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9db4004d00 0x7f9db4005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc806e0f0 con 0x7f9db40039a0 2023-10-11T06:12:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db400bb40 con 0x7f9db4004d00 2023-10-11T06:12:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db400bd00 con 0x7f9dc80ced30 2023-10-11T06:12:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcd671640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9db4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcce70640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 0x7f9db40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcde72640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9db4004d00 0x7f9db4005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcd671640 1 -- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 msgr2=0x7f9db40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcd671640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80ced30 0x7f9db40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcd671640 1 -- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9db4004d00 msgr2=0x7f9db4005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcd671640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9db4004d00 0x7f9db4005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.221+0000 7f9dcd671640 1 -- 172.21.15.84:0/3319193483 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4003520 con 0x7f9db40039a0 2023-10-11T06:12:29.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dcd671640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9db4003db0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400c210 tx=0x7f9db40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7293aaddaf207a4c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:29.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4005ad0 con 0x7f9db40039a0 2023-10-11T06:12:29.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4005c30 con 0x7f9db40039a0 2023-10-11T06:12:29.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dbffff640 1 -- 172.21.15.84:0/3319193483 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc402c890 con 0x7f9db40039a0 2023-10-11T06:12:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/3319193483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 msgr2=0x7f9db4003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dcf8d5640 1 --2- 172.21.15.84:0/3319193483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9db4003db0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400c210 tx=0x7f9db40057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dcde72640 1 -- 172.21.15.84:0/3319193483 reap_dead start 2023-10-11T06:12:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.223+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/3319193483 shutdown_connections 2023-10-11T06:12:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.224+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/3319193483 >> 172.21.15.84:0/3319193483 conn(0x7f9dc8069c70 msgr2=0x7f9dc806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.224+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/3319193483 shutdown_connections 2023-10-11T06:12:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.224+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/3319193483 wait complete. 2023-10-11T06:12:29.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.225+0000 7f9dcf8d5640 1 Processor -- start 2023-10-11T06:12:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.226+0000 7f9dcf8d5640 1 -- start start 2023-10-11T06:12:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcf8d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9dc81f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcf8d5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80c8150 0x7f9dc814e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcf8d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dc80b20b0 0x7f9dc80b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcf8d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80b2a90 con 0x7f9db40039a0 2023-10-11T06:12:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcf8d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc81f2950 con 0x7f9dc80b20b0 2023-10-11T06:12:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcf8d5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc806c810 con 0x7f9dc80c8150 2023-10-11T06:12:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9dc81f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9dc81f2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54396/0 (socket says 172.21.15.84:54396) 2023-10-11T06:12:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 -- 172.21.15.84:0/1896343128 learned_addr learned my addr 172.21.15.84:0/1896343128 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:29.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 -- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80c8150 msgr2=0x7f9dc814e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcce70640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80c8150 0x7f9dc814e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80c8150 0x7f9dc814e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcde72640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dc80b20b0 0x7f9dc80b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 -- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dc80b20b0 msgr2=0x7f9dc80b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dc80b20b0 0x7f9dc80b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.227+0000 7f9dcd671640 1 -- 172.21.15.84:0/1896343128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80b15c0 con 0x7f9db40039a0 2023-10-11T06:12:29.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.228+0000 7f9dcd671640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9dc81f2410 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9dc402c430 tx=0x7f9dc4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:29.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.228+0000 7f9dbdffb640 1 -- 172.21.15.84:0/1896343128 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4039070 con 0x7f9db40039a0 2023-10-11T06:12:29.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.228+0000 7f9dbdffb640 1 -- 172.21.15.84:0/1896343128 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc402cca0 con 0x7f9db40039a0 2023-10-11T06:12:29.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.228+0000 7f9dbdffb640 1 -- 172.21.15.84:0/1896343128 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc403c040 con 0x7f9db40039a0 2023-10-11T06:12:29.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.228+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc806ca60 con 0x7f9db40039a0 2023-10-11T06:12:29.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.228+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc80b1880 con 0x7f9db40039a0 2023-10-11T06:12:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.229+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc8073ed0 con 0x7f9db40039a0 2023-10-11T06:12:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.230+0000 7f9dbdffb640 1 -- 172.21.15.84:0/1896343128 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9dc4032070 con 0x7f9db40039a0 2023-10-11T06:12:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.231+0000 7f9dbdffb640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9dac06a830 0x7f9dac06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.231+0000 7f9dbdffb640 1 -- 172.21.15.84:0/1896343128 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f9dc40aa310 con 0x7f9db40039a0 2023-10-11T06:12:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.231+0000 7f9dcce70640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9dac06a830 0x7f9dac06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:29.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.232+0000 7f9dcce70640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9dac06a830 0x7f9dac06cce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8159af0 tx=0x7f9db8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:29.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.235+0000 7f9dbdffb640 1 -- 172.21.15.84:0/1896343128 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9dc40754f0 con 0x7f9db40039a0 2023-10-11T06:12:29.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:29.421+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9dc8070760 con 0x7f9db40039a0 2023-10-11T06:12:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.131+0000 7f9dbdffb640 1 -- 172.21.15.84:0/1896343128 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9dc4078b10 con 0x7f9db40039a0 2023-10-11T06:12:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-11T06:12:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.137+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9dac06a830 msgr2=0x7f9dac06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.137+0000 7f9dcf8d5640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9dac06a830 0x7f9dac06cce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8159af0 tx=0x7f9db8008040 comp rx=0 tx=0).stop 2023-10-11T06:12:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.137+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 msgr2=0x7f9dc81f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.137+0000 7f9dcf8d5640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9dc81f2410 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9dc402c430 tx=0x7f9dc4001a00 comp rx=0 tx=0).stop 2023-10-11T06:12:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.138+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 shutdown_connections 2023-10-11T06:12:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.138+0000 7f9dcf8d5640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9dac06a830 0x7f9dac06cce0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.138+0000 7f9dcf8d5640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dc80b20b0 0x7f9dc80b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.138+0000 7f9dcf8d5640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9dc80c8150 0x7f9dc814e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.138+0000 7f9dcf8d5640 1 --2- 172.21.15.84:0/1896343128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9db40039a0 0x7f9dc81f2410 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.138+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 >> 172.21.15.84:0/1896343128 conn(0x7f9dc8069c70 msgr2=0x7f9dc80c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.139+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 shutdown_connections 2023-10-11T06:12:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.139+0000 7f9dcf8d5640 1 -- 172.21.15.84:0/1896343128 wait complete. 2023-10-11T06:12:30.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T06:12:30.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.359+0000 7f512e9ae640 1 Processor -- start 2023-10-11T06:12:30.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.360+0000 7f512e9ae640 1 -- start start 2023-10-11T06:12:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.361+0000 7f512e9ae640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512815d510 0x7f512806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.361+0000 7f512e9ae640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51280ca750 0x7f512806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.361+0000 7f512e9ae640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280b3f20 con 0x7f512815e5d0 2023-10-11T06:12:30.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.361+0000 7f512e9ae640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51280caf10 con 0x7f51280ca750 2023-10-11T06:12:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.361+0000 7f512e9ae640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51280cac30 con 0x7f512815d510 2023-10-11T06:12:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512e1ad640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f512815e5d0 0x7f51280c8ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34588/0 (socket says 172.21.15.84:34588) 2023-10-11T06:12:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512e1ad640 1 -- 172.21.15.84:0/966873670 learned_addr learned my addr 172.21.15.84:0/966873670 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512d1ab640 1 --2- 172.21.15.84:0/966873670 >> v2:172.21.15.84:3300/0 conn(0x7f51280ca750 0x7f512806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:30.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512d9ac640 1 --2- 172.21.15.84:0/966873670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512815d510 0x7f512806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:30.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512d1ab640 1 -- 172.21.15.84:0/966873670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512815d510 msgr2=0x7f512806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:30.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512d1ab640 1 --2- 172.21.15.84:0/966873670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512815d510 0x7f512806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512d1ab640 1 -- 172.21.15.84:0/966873670 >> v1:172.21.15.84:6789/0 conn(0x7f512815e5d0 legacy=0x7f51280c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:30.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.362+0000 7f512d1ab640 1 -- 172.21.15.84:0/966873670 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512815daf0 con 0x7f51280ca750 2023-10-11T06:12:30.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.363+0000 7f512d1ab640 1 --2- 172.21.15.84:0/966873670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51280ca750 0x7f512806bdb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f51180097a0 tx=0x7f511802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f053b2edccc54b73 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:30.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512c9aa640 1 -- 172.21.15.84:0/966873670 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511802d040 con 0x7f51280ca750 2023-10-11T06:12:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512c9aa640 1 -- 172.21.15.84:0/966873670 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118031810 con 0x7f51280ca750 2023-10-11T06:12:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512c9aa640 1 -- 172.21.15.84:0/966873670 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118031ba0 con 0x7f51280ca750 2023-10-11T06:12:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512e9ae640 1 -- 172.21.15.84:0/966873670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51280ca750 msgr2=0x7f512806bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512e9ae640 1 --2- 172.21.15.84:0/966873670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51280ca750 0x7f512806bdb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f51180097a0 tx=0x7f511802b8b0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512e9ae640 1 -- 172.21.15.84:0/966873670 shutdown_connections 2023-10-11T06:12:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512e9ae640 1 --2- 172.21.15.84:0/966873670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51280ca750 0x7f512806bdb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512e9ae640 1 --2- 172.21.15.84:0/966873670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512815d510 0x7f512806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.364+0000 7f512e9ae640 1 -- 172.21.15.84:0/966873670 >> 172.21.15.84:0/966873670 conn(0x7f5128069c70 msgr2=0x7f512806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.365+0000 7f512e9ae640 1 -- 172.21.15.84:0/966873670 shutdown_connections 2023-10-11T06:12:30.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.365+0000 7f512e9ae640 1 -- 172.21.15.84:0/966873670 wait complete. 2023-10-11T06:12:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.366+0000 7f512e9ae640 1 Processor -- start 2023-10-11T06:12:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.366+0000 7f512e9ae640 1 -- start start 2023-10-11T06:12:30.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.367+0000 7f512e9ae640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51280ca750 0x7f51281f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.367+0000 7f512e9ae640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f512815d510 0x7f51280c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.367+0000 7f512e9ae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512815e5d0 0x7f512814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.367+0000 7f512e9ae640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51281f5810 con 0x7f512815d510 2023-10-11T06:12:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.367+0000 7f512e9ae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51281f4f70 con 0x7f512815e5d0 2023-10-11T06:12:30.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.367+0000 7f512e9ae640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51281f5250 con 0x7f51280ca750 2023-10-11T06:12:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f512815d510 0x7f51280c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f512815d510 0x7f51280c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54422/0 (socket says 172.21.15.84:54422) 2023-10-11T06:12:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 -- 172.21.15.84:0/829338894 learned_addr learned my addr 172.21.15.84:0/829338894 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:30.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d9ac640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51280ca750 0x7f51281f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:30.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 -- 172.21.15.84:0/829338894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51280ca750 msgr2=0x7f51281f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:30.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51280ca750 0x7f51281f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 -- 172.21.15.84:0/829338894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512815e5d0 msgr2=0x7f512814e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:30.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512815e5d0 0x7f512814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:30.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 -- 172.21.15.84:0/829338894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009400 con 0x7f512815d510 2023-10-11T06:12:30.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.368+0000 7f512d1ab640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f512815d510 0x7f51280c66b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51180098c0 tx=0x7f51180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:30.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.369+0000 7f511e7fc640 1 -- 172.21.15.84:0/829338894 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511803ad60 con 0x7f512815d510 2023-10-11T06:12:30.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.369+0000 7f511e7fc640 1 -- 172.21.15.84:0/829338894 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118044400 con 0x7f512815d510 2023-10-11T06:12:30.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.369+0000 7f511e7fc640 1 -- 172.21.15.84:0/829338894 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180447b0 con 0x7f512815d510 2023-10-11T06:12:30.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.369+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51281f5530 con 0x7f512815d510 2023-10-11T06:12:30.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.369+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512814ed10 con 0x7f512815d510 2023-10-11T06:12:30.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.370+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51280c7090 con 0x7f512815d510 2023-10-11T06:12:30.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.371+0000 7f511e7fc640 1 -- 172.21.15.84:0/829338894 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f511804d440 con 0x7f512815d510 2023-10-11T06:12:30.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.372+0000 7f511e7fc640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f510c06a860 0x7f510c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:30.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.372+0000 7f511e7fc640 1 -- 172.21.15.84:0/829338894 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f51180ab290 con 0x7f512815d510 2023-10-11T06:12:30.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.373+0000 7f512d9ac640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f510c06a860 0x7f510c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:30.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.373+0000 7f512d9ac640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f510c06a860 0x7f510c06cd10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5124006fd0 tx=0x7f5124008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:30.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.377+0000 7f511e7fc640 1 -- 172.21.15.84:0/829338894 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f51280c7090 con 0x7f512815d510 2023-10-11T06:12:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:30.564+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f51280c7090 con 0x7f512815d510 2023-10-11T06:12:31.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.145+0000 7f511e7fc640 1 -- 172.21.15.84:0/829338894 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f51280c7090 con 0x7f512815d510 2023-10-11T06:12:31.150 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-11T06:12:31.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.150+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f510c06a860 msgr2=0x7f510c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:31.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.150+0000 7f512e9ae640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f510c06a860 0x7f510c06cd10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5124006fd0 tx=0x7f5124008040 comp rx=0 tx=0).stop 2023-10-11T06:12:31.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.150+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f512815d510 msgr2=0x7f51280c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:31.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.150+0000 7f512e9ae640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f512815d510 0x7f51280c66b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51180098c0 tx=0x7f51180046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 shutdown_connections 2023-10-11T06:12:31.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f510c06a860 0x7f510c06cd10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512815e5d0 0x7f512814e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f512815d510 0x7f51280c66b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 --2- 172.21.15.84:0/829338894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51280ca750 0x7f51281f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 >> 172.21.15.84:0/829338894 conn(0x7f5128069c70 msgr2=0x7f512806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:31.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 shutdown_connections 2023-10-11T06:12:31.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.151+0000 7f512e9ae640 1 -- 172.21.15.84:0/829338894 wait complete. 2023-10-11T06:12:31.176 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T06:12:31.176 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T06:12:31.176 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T06:12:31.177 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:31.177 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T06:12:31.177 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T06:12:31.178 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T06:12:31.178 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:31.178 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T06:12:31.178 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T06:12:31.179 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T06:12:31.179 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:31.179 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T06:12:31.179 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T06:12:31.180 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T06:12:31.180 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:12:31.197 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T06:12:31.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T06:12:31.198 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T06:12:31.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T06:12:31.201 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T06:12:31.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T06:12:31.204 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T06:12:31.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T06:12:31.247 INFO:tasks.ceph.mds.a:Started 2023-10-11T06:12:31.247 DEBUG:teuthology.parallel:result is None 2023-10-11T06:12:31.251 INFO:tasks.ceph.mds.d:Started 2023-10-11T06:12:31.251 DEBUG:teuthology.parallel:result is None 2023-10-11T06:12:31.254 INFO:tasks.ceph.mds.c:Started 2023-10-11T06:12:31.254 DEBUG:teuthology.parallel:result is None 2023-10-11T06:12:31.258 INFO:tasks.ceph.mds.b:Started 2023-10-11T06:12:31.259 DEBUG:teuthology.parallel:result is None 2023-10-11T06:12:31.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-11T06:12:31.404 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-11T06:12:31.407 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-11T06:12:31.413 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2023-10-11T06:12:31.435 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2023-10-11T06:12:31.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.448+0000 7f5d442b4640 1 Processor -- start 2023-10-11T06:12:31.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.449+0000 7f5d442b4640 1 -- start start 2023-10-11T06:12:31.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.450+0000 7f5d442b4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d3c15c290 0x7f5d3c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.450+0000 7f5d442b4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d3c0d0c20 0x7f5d3c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d442b4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0d2a60 con 0x7f5d3c0d25e0 2023-10-11T06:12:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d442b4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c06cd10 con 0x7f5d3c15c290 2023-10-11T06:12:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d442b4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c0afb60 con 0x7f5d3c0d0c20 2023-10-11T06:12:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42050640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d3c15c290 0x7f5d3c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42050640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d3c15c290 0x7f5d3c0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54452/0 (socket says 172.21.15.84:54452) 2023-10-11T06:12:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42050640 1 -- 172.21.15.84:0/1444278795 learned_addr learned my addr 172.21.15.84:0/1444278795 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42851640 1 --2- 172.21.15.84:0/1444278795 >> v2:172.21.15.99:3300/0 conn(0x7f5d3c0d0c20 0x7f5d3c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42050640 1 -- 172.21.15.84:0/1444278795 >> v2:172.21.15.99:3300/0 conn(0x7f5d3c0d0c20 msgr2=0x7f5d3c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:31.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42050640 1 --2- 172.21.15.84:0/1444278795 >> v2:172.21.15.99:3300/0 conn(0x7f5d3c0d0c20 0x7f5d3c0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42050640 1 -- 172.21.15.84:0/1444278795 >> v1:172.21.15.99:6789/0 conn(0x7f5d3c0d25e0 legacy=0x7f5d3c0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:31.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42050640 1 -- 172.21.15.84:0/1444278795 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0afe40 con 0x7f5d3c15c290 2023-10-11T06:12:31.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.451+0000 7f5d42851640 1 --2- 172.21.15.84:0/1444278795 >> v2:172.21.15.99:3300/0 conn(0x7f5d3c0d0c20 0x7f5d3c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:31.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.453+0000 7f5d42050640 1 --2- 172.21.15.84:0/1444278795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c0af4d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0d2150 tx=0x7f5d3802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d728166bb72a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:31.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.456+0000 7f5d4104e640 1 -- 172.21.15.84:0/1444278795 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38031800 con 0x7f5d3c15c290 2023-10-11T06:12:31.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.457+0000 7f5d4104e640 1 -- 172.21.15.84:0/1444278795 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38031960 con 0x7f5d3c15c290 2023-10-11T06:12:31.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.457+0000 7f5d4104e640 1 -- 172.21.15.84:0/1444278795 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38031cf0 con 0x7f5d3c15c290 2023-10-11T06:12:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.458+0000 7f5d442b4640 1 -- 172.21.15.84:0/1444278795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 msgr2=0x7f5d3c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.458+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1444278795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c0af4d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0d2150 tx=0x7f5d3802b8a0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.459+0000 7f5d442b4640 1 -- 172.21.15.84:0/1444278795 shutdown_connections 2023-10-11T06:12:31.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.459+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1444278795 >> v2:172.21.15.99:3300/0 conn(0x7f5d3c0d0c20 0x7f5d3c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.459+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1444278795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c0af4d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.459+0000 7f5d442b4640 1 -- 172.21.15.84:0/1444278795 >> 172.21.15.84:0/1444278795 conn(0x7f5d3c071cc0 msgr2=0x7f5d3c09bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.459+0000 7f5d442b4640 1 -- 172.21.15.84:0/1444278795 shutdown_connections 2023-10-11T06:12:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.459+0000 7f5d442b4640 1 -- 172.21.15.84:0/1444278795 wait complete. 2023-10-11T06:12:31.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.463+0000 7f5d442b4640 1 Processor -- start 2023-10-11T06:12:31.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.464+0000 7f5d442b4640 1 -- start start 2023-10-11T06:12:31.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d442b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3c0d0c20 0x7f5d3c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:31.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d442b4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d3c0d25e0 0x7f5d3c1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d442b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c09cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d442b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1f3960 con 0x7f5d3c15c290 2023-10-11T06:12:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d442b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1f3c40 con 0x7f5d3c0d0c20 2023-10-11T06:12:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d442b4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1f3f20 con 0x7f5d3c0d25e0 2023-10-11T06:12:31.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d42851640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c09cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:31.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d42050640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3c0d0c20 0x7f5d3c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:31.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d42851640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c09cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54456/0 (socket says 172.21.15.84:54456) 2023-10-11T06:12:31.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.465+0000 7f5d42851640 1 -- 172.21.15.84:0/1458949119 learned_addr learned my addr 172.21.15.84:0/1458949119 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:31.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d4184f640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d3c0d25e0 0x7f5d3c1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:31.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d42851640 1 -- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d3c0d25e0 msgr2=0x7f5d3c1f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:31.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d42851640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d3c0d25e0 0x7f5d3c1f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d42851640 1 -- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3c0d0c20 msgr2=0x7f5d3c155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:31.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d42851640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3c0d0c20 0x7f5d3c155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:31.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d42851640 1 -- 172.21.15.84:0/1458949119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380094d0 con 0x7f5d3c15c290 2023-10-11T06:12:31.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d42050640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3c0d0c20 0x7f5d3c155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:31.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.466+0000 7f5d42851640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c09cc00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5d34007ec0 tx=0x7f5d3400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:31.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.467+0000 7f5d32ffd640 1 -- 172.21.15.84:0/1458949119 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34003b90 con 0x7f5d3c15c290 2023-10-11T06:12:31.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.467+0000 7f5d32ffd640 1 -- 172.21.15.84:0/1458949119 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400f040 con 0x7f5d3c15c290 2023-10-11T06:12:31.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.467+0000 7f5d32ffd640 1 -- 172.21.15.84:0/1458949119 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400de80 con 0x7f5d3c15c290 2023-10-11T06:12:31.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.467+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c1f4200 con 0x7f5d3c15c290 2023-10-11T06:12:31.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.467+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c155cd0 con 0x7f5d3c15c290 2023-10-11T06:12:31.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.468+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c1f4720 con 0x7f5d3c15c290 2023-10-11T06:12:31.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.469+0000 7f5d32ffd640 1 -- 172.21.15.84:0/1458949119 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5d34004030 con 0x7f5d3c15c290 2023-10-11T06:12:31.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.470+0000 7f5d32ffd640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2006a830 0x7f5d2006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:31.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.471+0000 7f5d32ffd640 1 -- 172.21.15.84:0/1458949119 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5d34089180 con 0x7f5d3c15c290 2023-10-11T06:12:31.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.471+0000 7f5d42050640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2006a830 0x7f5d2006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:31.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.472+0000 7f5d42050640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2006a830 0x7f5d2006cce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d3803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:31.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.476+0000 7f5d32ffd640 1 -- 172.21.15.84:0/1458949119 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5d3c1f4720 con 0x7f5d3c15c290 2023-10-11T06:12:31.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:31.673+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5d3c1f4720 con 0x7f5d3c15c290 2023-10-11T06:12:32.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.469+0000 7f5d32ffd640 1 -- 172.21.15.84:0/1458949119 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5d3c1f4720 con 0x7f5d3c15c290 2023-10-11T06:12:32.471 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-11T06:12:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2006a830 msgr2=0x7f5d2006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2006a830 0x7f5d2006cce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d3803c040 comp rx=0 tx=0).stop 2023-10-11T06:12:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 msgr2=0x7f5d3c09cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c09cc00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5d34007ec0 tx=0x7f5d3400d4a0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 shutdown_connections 2023-10-11T06:12:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d2006a830 0x7f5d2006cce0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d3c15c290 0x7f5d3c09cc00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d3c0d25e0 0x7f5d3c1f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 --2- 172.21.15.84:0/1458949119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3c0d0c20 0x7f5d3c155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.473+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 >> 172.21.15.84:0/1458949119 conn(0x7f5d3c071cc0 msgr2=0x7f5d3c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.474+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 shutdown_connections 2023-10-11T06:12:32.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.474+0000 7f5d442b4640 1 -- 172.21.15.84:0/1458949119 wait complete. 2023-10-11T06:12:32.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-11T06:12:32.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.701+0000 7f2a98a6a640 1 Processor -- start 2023-10-11T06:12:32.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.701+0000 7f2a98a6a640 1 -- start start 2023-10-11T06:12:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.702+0000 7f2a98a6a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a98a6a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a940c9360 0x7f2a940ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a98a6a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a94071a90 con 0x7f2a940ca300 2023-10-11T06:12:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a98a6a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a94072310 con 0x7f2a940c9360 2023-10-11T06:12:32.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a98a6a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94070280 con 0x7f2a9415d4f0 2023-10-11T06:12:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a9259c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a92d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a940c9360 0x7f2a940ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a9259c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:45600/0 (socket says 172.21.15.84:45600) 2023-10-11T06:12:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.703+0000 7f2a9259c640 1 -- 172.21.15.84:0/1876823619 learned_addr learned my addr 172.21.15.84:0/1876823619 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.704+0000 7f2a9259c640 1 -- 172.21.15.84:0/1876823619 >> v2:172.21.15.99:3300/0 conn(0x7f2a940c9360 msgr2=0x7f2a940ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:32.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.704+0000 7f2a9259c640 1 --2- 172.21.15.84:0/1876823619 >> v2:172.21.15.99:3300/0 conn(0x7f2a940c9360 0x7f2a940ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.704+0000 7f2a9259c640 1 -- 172.21.15.84:0/1876823619 >> v1:172.21.15.99:6789/0 conn(0x7f2a940ca300 legacy=0x7f2a940c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.704+0000 7f2a9259c640 1 -- 172.21.15.84:0/1876823619 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94071cc0 con 0x7f2a9415d4f0 2023-10-11T06:12:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.704+0000 7f2a9159a640 1 -- 172.21.15.84:0/1876823619 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400909951 0 0) 0x7f2a94071a90 con 0x7f2a940ca300 2023-10-11T06:12:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.706+0000 7f2a9259c640 1 --2- 172.21.15.84:0/1876823619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940cb540 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009ea0 tx=0x7f2a8402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f8f44c16fce9d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.706+0000 7f2a9159a640 1 -- 172.21.15.84:0/1876823619 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8402cb40 con 0x7f2a9415d4f0 2023-10-11T06:12:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.706+0000 7f2a9159a640 1 -- 172.21.15.84:0/1876823619 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8402cca0 con 0x7f2a9415d4f0 2023-10-11T06:12:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.706+0000 7f2a9159a640 1 -- 172.21.15.84:0/1876823619 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84034610 con 0x7f2a9415d4f0 2023-10-11T06:12:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1876823619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 msgr2=0x7f2a940cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1876823619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940cb540 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009ea0 tx=0x7f2a8402bfa0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1876823619 shutdown_connections 2023-10-11T06:12:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1876823619 >> v2:172.21.15.99:3300/0 conn(0x7f2a940c9360 0x7f2a940ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1876823619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940cb540 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1876823619 >> 172.21.15.84:0/1876823619 conn(0x7f2a94069c70 msgr2=0x7f2a9406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1876823619 shutdown_connections 2023-10-11T06:12:32.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.707+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1876823619 wait complete. 2023-10-11T06:12:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.709+0000 7f2a98a6a640 1 Processor -- start 2023-10-11T06:12:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.709+0000 7f2a98a6a640 1 -- start start 2023-10-11T06:12:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a98a6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a940c9360 0x7f2a941558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a98a6a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a940ca300 0x7f2a941ef840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a98a6a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a98a6a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940c6170 con 0x7f2a940ca300 2023-10-11T06:12:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a98a6a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941efd80 con 0x7f2a940c9360 2023-10-11T06:12:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a98a6a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9415b610 con 0x7f2a9415d4f0 2023-10-11T06:12:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a940ca300 0x7f2a941ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a91d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a940ca300 0x7f2a941ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54470/0 (socket says 172.21.15.84:54470) 2023-10-11T06:12:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a91d9b640 1 -- 172.21.15.84:0/1630517288 learned_addr learned my addr 172.21.15.84:0/1630517288 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a9259c640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a940c9360 0x7f2a941558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.710+0000 7f2a92d9d640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a91d9b640 1 -- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 msgr2=0x7f2a940c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a91d9b640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a91d9b640 1 -- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a940c9360 msgr2=0x7f2a941558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a91d9b640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a940c9360 0x7f2a941558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a91d9b640 1 -- 172.21.15.84:0/1630517288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a84009b30 con 0x7f2a940ca300 2023-10-11T06:12:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a91d9b640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a940ca300 0x7f2a941ef840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c410 tx=0x7f2a7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a8affd640 1 -- 172.21.15.84:0/1630517288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0035f0 con 0x7f2a940ca300 2023-10-11T06:12:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a8affd640 1 -- 172.21.15.84:0/1630517288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c003770 con 0x7f2a940ca300 2023-10-11T06:12:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a8affd640 1 -- 172.21.15.84:0/1630517288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c003b20 con 0x7f2a940ca300 2023-10-11T06:12:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9415ace0 con 0x7f2a940ca300 2023-10-11T06:12:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.711+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9415b2a0 con 0x7f2a940ca300 2023-10-11T06:12:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.712+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a94073ed0 con 0x7f2a940ca300 2023-10-11T06:12:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.713+0000 7f2a8affd640 1 -- 172.21.15.84:0/1630517288 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2a7c003c80 con 0x7f2a940ca300 2023-10-11T06:12:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.714+0000 7f2a8affd640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2a7406a860 0x7f2a7406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.714+0000 7f2a8affd640 1 -- 172.21.15.84:0/1630517288 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2a7c088240 con 0x7f2a940ca300 2023-10-11T06:12:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.714+0000 7f2a9259c640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2a7406a860 0x7f2a7406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.715+0000 7f2a9259c640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2a7406a860 0x7f2a7406cd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a84035000 tx=0x7f2a84030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.718+0000 7f2a8affd640 1 -- 172.21.15.84:0/1630517288 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2a7c053880 con 0x7f2a940ca300 2023-10-11T06:12:32.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.977+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2a940b09b0 con 0x7f2a940ca300 2023-10-11T06:12:32.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.978+0000 7f2a8affd640 1 -- 172.21.15.84:0/1630517288 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2a7c056ea0 con 0x7f2a940ca300 2023-10-11T06:12:32.979 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:32.979 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:12:32.979 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-11T06:12:32.980 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.980 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-11T06:12:32.980 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZl2TIZBBAAHTDKquuoZy+2HB2lk815Fg==", 2023-10-11T06:12:32.980 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.980 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:32.981 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:12:32.981 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:12:32.981 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.981 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.981 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.982 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.982 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-11T06:12:32.982 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZlaZYAJhAAzmJNe7HYcmHJu+7Fbfv6dw==", 2023-10-11T06:12:32.982 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.982 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:32.983 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:12:32.983 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:12:32.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.983 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.983 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-11T06:12:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZlj+QWFBAAw6XDEihH6iXYbQsLht/VsQ==", 2023-10-11T06:12:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:12:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:12:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-11T06:12:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZlgvsCNxAAJL7HKUMYQA6zG6useZDoQw==", 2023-10-11T06:12:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:12:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:12:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-11T06:12:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA9PCZljp29IBAAFJgwhWwcw3OWMIDeeXR2sw==", 2023-10-11T06:12:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-11T06:12:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBAPCZlBCAQLBAAeWQVnN/FYimDHxX1nYzVKA==", 2023-10-11T06:12:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-11T06:12:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBDPCZlu1o+NRAAjAauNPRb88x0p37gT88vBw==", 2023-10-11T06:12:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-11T06:12:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBHPCZlcy98BBAALniVjO3ojVM/+q9fw6tNsw==", 2023-10-11T06:12:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-11T06:12:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBOPCZlmds+CBAAb0nHqugOzMsqLZCUabaHGQ==", 2023-10-11T06:12:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-11T06:12:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBRPCZlPObDEhAA1yMeENf3iV3lQf6f6WM8UQ==", 2023-10-11T06:12:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-11T06:12:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBUPCZlZQvrHBAAc/hx1KwJuGfHv5M+Tobkjg==", 2023-10-11T06:12:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-11T06:12:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBXPCZlKKt5NRAAkM8aLRyZPYaG0n1AH2tDIw==", 2023-10-11T06:12:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:12:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:12:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-11T06:12:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZlP/K+CRAAYT5I5tAyH8nHrsBReUAL9A==", 2023-10-11T06:12:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-11T06:12:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-11T06:12:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-11T06:12:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-11T06:12:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZlzhkWHxAAMV2BVukY7a/l7AXFNPCMhg==", 2023-10-11T06:12:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-11T06:12:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-11T06:12:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-11T06:12:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-11T06:12:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZli1j2ExAA77o/7VayFJb+C/TfdZrxRg==", 2023-10-11T06:12:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-11T06:12:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-11T06:12:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-11T06:12:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-11T06:12:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZljnzfKRAA2s7Yg5NkKNp/LP1eWIBEVA==", 2023-10-11T06:12:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:12:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-11T06:12:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-11T06:12:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-11T06:12:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-11T06:12:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA2PCZlczZbMRAAMctxZazuzrHiDNfSBIb+sA==", 2023-10-11T06:12:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-11T06:12:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:12:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:12:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-mds", 2023-10-11T06:12:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBlPCZlTM4VNRAAxx7jpGND2bot9Phfdun2GQ==", 2023-10-11T06:12:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-mds" 2023-10-11T06:12:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-mgr", 2023-10-11T06:12:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBlPCZlYekVNRAAjp+R3sABtn470xLxczd9nQ==", 2023-10-11T06:12:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-11T06:12:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-osd", 2023-10-11T06:12:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBlPCZl2QEWNRAAKLrckDSujrTAk6/IblVWug==", 2023-10-11T06:12:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-osd" 2023-10-11T06:12:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rbd", 2023-10-11T06:12:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBlPCZl2BkWNRAAmqYVkfvQtCd0Z5S5IKigPQ==", 2023-10-11T06:12:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-11T06:12:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-11T06:12:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBlPCZlWTIWNRAAMPENucgAxtjoiq82VkA5Yw==", 2023-10-11T06:12:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-11T06:12:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rgw", 2023-10-11T06:12:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBlPCZliFAWNRAAvTWygESsRjeYJlLtE8pZwg==", 2023-10-11T06:12:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-11T06:12:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-11T06:12:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA3PCZlsakeMxAA52n80muv9FKTIvML6txp5w==", 2023-10-11T06:12:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-11T06:12:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-11T06:12:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-11T06:12:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA3PCZlcQtlJhAAyb6WWJKJgbNLkVkNL36ttg==", 2023-10-11T06:12:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:12:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-11T06:12:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-11T06:12:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:12:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-11T06:12:33.020 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:12:33.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.980+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2a7406a860 msgr2=0x7f2a7406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.980+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2a7406a860 0x7f2a7406cd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a84035000 tx=0x7f2a84030000 comp rx=0 tx=0).stop 2023-10-11T06:12:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a940ca300 msgr2=0x7f2a941ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a940ca300 0x7f2a941ef840 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c410 tx=0x7f2a7c00c8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 shutdown_connections 2023-10-11T06:12:33.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2a7406a860 0x7f2a7406cd10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a9415d4f0 0x7f2a940c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a940ca300 0x7f2a941ef840 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 --2- 172.21.15.84:0/1630517288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a940c9360 0x7f2a941558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 >> 172.21.15.84:0/1630517288 conn(0x7f2a94069c70 msgr2=0x7f2a940b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:33.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 shutdown_connections 2023-10-11T06:12:33.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:32.981+0000 7f2a98a6a640 1 -- 172.21.15.84:0/1630517288 wait complete. 2023-10-11T06:12:33.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-11T06:12:33.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.242+0000 7f82a5204640 1 Processor -- start 2023-10-11T06:12:33.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.242+0000 7f82a5204640 1 -- start start 2023-10-11T06:12:33.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.243+0000 7f82a5204640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f82a00ce0d0 0x7f82a00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.243+0000 7f82a5204640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00adb90 0x7f82a00caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f82a5204640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a01593f0 con 0x7f82a00cd300 2023-10-11T06:12:33.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f82a5204640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a015a340 con 0x7f82a00ce0d0 2023-10-11T06:12:33.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f82a5204640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00ce4a0 con 0x7f82a00adb90 2023-10-11T06:12:33.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f829f59e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f82a00cd300 0x7f82a00b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34616/0 (socket says 172.21.15.84:34616) 2023-10-11T06:12:33.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f829f59e640 1 -- 172.21.15.84:0/3811937665 learned_addr learned my addr 172.21.15.84:0/3811937665 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:33.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f829e59c640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00adb90 0x7f82a00caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f829ed9d640 1 --2- 172.21.15.84:0/3811937665 >> v2:172.21.15.99:3300/0 conn(0x7f82a00ce0d0 0x7f82a00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17671367 0 0) 0x7f82a01593f0 con 0x7f82a00cd300 2023-10-11T06:12:33.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.244+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8288003520 con 0x7f82a00cd300 2023-10-11T06:12:33.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473752409 0 0) 0x7f8288003520 con 0x7f82a00cd300 2023-10-11T06:12:33.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00adb90 msgr2=0x7f82a00caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:33.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00adb90 0x7f82a00caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 >> v2:172.21.15.99:3300/0 conn(0x7f82a00ce0d0 msgr2=0x7f82a00c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:33.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 --2- 172.21.15.84:0/3811937665 >> v2:172.21.15.99:3300/0 conn(0x7f82a00ce0d0 0x7f82a00c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a01596c0 con 0x7f82a00cd300 2023-10-11T06:12:33.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f82980019f0 con 0x7f82a00cd300 2023-10-11T06:12:33.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 >> v1:172.21.15.84:6789/0 conn(0x7f82a00cd300 legacy=0x7f82a00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f8288003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82880047c0 0x7f8288004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00adb90 0x7f8288005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a015a340 con 0x7f8288003970 2023-10-11T06:12:33.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00ce4a0 con 0x7f82880047c0 2023-10-11T06:12:33.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828800dc40 con 0x7f82a00adb90 2023-10-11T06:12:33.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829ed9d640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f8288003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829ed9d640 1 -- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00adb90 msgr2=0x7f8288005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:33.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829ed9d640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00adb90 0x7f8288005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829ed9d640 1 -- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82880047c0 msgr2=0x7f8288004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:33.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829ed9d640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82880047c0 0x7f8288004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.245+0000 7f829ed9d640 1 -- 172.21.15.84:0/3811937665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288003520 con 0x7f8288003970 2023-10-11T06:12:33.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.246+0000 7f829ed9d640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f8288003d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f829000ba10 tx=0x7f82880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb96a39eef5245a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:33.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290005990 con 0x7f8288003970 2023-10-11T06:12:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290005af0 con 0x7f8288003970 2023-10-11T06:12:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f829dd9b640 1 -- 172.21.15.84:0/3811937665 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290005e80 con 0x7f8288003970 2023-10-11T06:12:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f82a5204640 1 -- 172.21.15.84:0/3811937665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 msgr2=0x7f8288003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f82a5204640 1 --2- 172.21.15.84:0/3811937665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f8288003d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f829000ba10 tx=0x7f82880057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f829f59e640 1 -- 172.21.15.84:0/3811937665 reap_dead start 2023-10-11T06:12:33.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f82a5204640 1 -- 172.21.15.84:0/3811937665 shutdown_connections 2023-10-11T06:12:33.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f82a5204640 1 -- 172.21.15.84:0/3811937665 >> 172.21.15.84:0/3811937665 conn(0x7f82a0069c70 msgr2=0x7f82a006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:33.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f82a5204640 1 -- 172.21.15.84:0/3811937665 shutdown_connections 2023-10-11T06:12:33.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.247+0000 7f82a5204640 1 -- 172.21.15.84:0/3811937665 wait complete. 2023-10-11T06:12:33.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.249+0000 7f82a5204640 1 Processor -- start 2023-10-11T06:12:33.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.249+0000 7f82a5204640 1 -- start start 2023-10-11T06:12:33.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f82a5204640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f82a00ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f82a5204640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a0157750 0x7f82a0157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f82a5204640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00b0330 0x7f82a00b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f82a5204640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00c30d0 con 0x7f8288003970 2023-10-11T06:12:33.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f82a5204640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0158100 con 0x7f82a0157750 2023-10-11T06:12:33.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f82a5204640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00b0fd0 con 0x7f82a00b0330 2023-10-11T06:12:33.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f829ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f82a00ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.250+0000 7f829ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f82a00ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54490/0 (socket says 172.21.15.84:54490) 2023-10-11T06:12:33.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f829ed9d640 1 -- 172.21.15.84:0/2610540839 learned_addr learned my addr 172.21.15.84:0/2610540839 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:33.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f829ed9d640 1 -- 172.21.15.84:0/2610540839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00b0330 msgr2=0x7f82a00b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f829ed9d640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00b0330 0x7f82a00b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f829ed9d640 1 -- 172.21.15.84:0/2610540839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a0157750 msgr2=0x7f82a0157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f829ed9d640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a0157750 0x7f82a0157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f829ed9d640 1 -- 172.21.15.84:0/2610540839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00aef00 con 0x7f8288003970 2023-10-11T06:12:33.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f829ed9d640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f82a00ae9c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f829000ba70 tx=0x7f8290004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:33.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f82877fe640 1 -- 172.21.15.84:0/2610540839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290005ea0 con 0x7f8288003970 2023-10-11T06:12:33.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f82877fe640 1 -- 172.21.15.84:0/2610540839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290033830 con 0x7f8288003970 2023-10-11T06:12:33.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.251+0000 7f82877fe640 1 -- 172.21.15.84:0/2610540839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290033ba0 con 0x7f8288003970 2023-10-11T06:12:33.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.252+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00c1ed0 con 0x7f8288003970 2023-10-11T06:12:33.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.252+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a00c2730 con 0x7f8288003970 2023-10-11T06:12:33.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.253+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a0073ed0 con 0x7f8288003970 2023-10-11T06:12:33.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.254+0000 7f82877fe640 1 -- 172.21.15.84:0/2610540839 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8290031070 con 0x7f8288003970 2023-10-11T06:12:33.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.254+0000 7f82877fe640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f828c06a860 0x7f828c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.255+0000 7f82877fe640 1 -- 172.21.15.84:0/2610540839 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f82900ab300 con 0x7f8288003970 2023-10-11T06:12:33.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.255+0000 7f829e59c640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f828c06a860 0x7f828c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.255+0000 7f829e59c640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f828c06a860 0x7f828c06cd10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8294006f00 tx=0x7f8294008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:33.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.259+0000 7f82877fe640 1 -- 172.21.15.84:0/2610540839 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f829007a350 con 0x7f8288003970 2023-10-11T06:12:33.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.498+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f82a0159c50 con 0x7f8288003970 2023-10-11T06:12:33.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.504+0000 7f82877fe640 1 -- 172.21.15.84:0/2610540839 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8290079ee0 con 0x7f8288003970 2023-10-11T06:12:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f828c06a860 msgr2=0x7f828c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f828c06a860 0x7f828c06cd10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8294006f00 tx=0x7f8294008040 comp rx=0 tx=0).stop 2023-10-11T06:12:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 msgr2=0x7f82a00ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f82a00ae9c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f829000ba70 tx=0x7f8290004730 comp rx=0 tx=0).stop 2023-10-11T06:12:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 shutdown_connections 2023-10-11T06:12:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f828c06a860 0x7f828c06cd10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82a00b0330 0x7f82a00b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a0157750 0x7f82a0157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 --2- 172.21.15.84:0/2610540839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8288003970 0x7f82a00ae9c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 >> 172.21.15.84:0/2610540839 conn(0x7f82a0069c70 msgr2=0x7f82a00c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 shutdown_connections 2023-10-11T06:12:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.507+0000 7f82a5204640 1 -- 172.21.15.84:0/2610540839 wait complete. 2023-10-11T06:12:33.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-11T06:12:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.735+0000 7ff3fff77640 1 Processor -- start 2023-10-11T06:12:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.735+0000 7ff3fff77640 1 -- start start 2023-10-11T06:12:33.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.736+0000 7ff3fff77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.736+0000 7ff3fff77640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff3f80c9360 0x7ff3f80ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.736+0000 7ff3fff77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f8071a90 con 0x7ff3f80ca300 2023-10-11T06:12:33.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.736+0000 7ff3fff77640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f8072310 con 0x7ff3f80c9360 2023-10-11T06:12:33.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.736+0000 7ff3fff77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8070280 con 0x7ff3f815d4f0 2023-10-11T06:12:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.737+0000 7ff3fe514640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff3f80c9360 0x7ff3f80ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.737+0000 7ff3fdd13640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.737+0000 7ff3fe514640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff3f80c9360 0x7ff3f80ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:46224/0 (socket says 172.21.15.84:46224) 2023-10-11T06:12:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.737+0000 7ff3fe514640 1 -- 172.21.15.84:0/2520173488 learned_addr learned my addr 172.21.15.84:0/2520173488 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.738+0000 7ff3fe514640 1 -- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 msgr2=0x7ff3f80cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.738+0000 7ff3fe514640 1 --2- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f80cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.738+0000 7ff3fe514640 1 -- 172.21.15.84:0/2520173488 >> v1:172.21.15.99:6789/0 conn(0x7ff3f80ca300 legacy=0x7ff3f80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.738+0000 7ff3fe514640 1 -- 172.21.15.84:0/2520173488 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f8071cc0 con 0x7ff3f80c9360 2023-10-11T06:12:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.738+0000 7ff3fdd13640 1 --2- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:33.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.739+0000 7ff3fe514640 1 --2- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80c9360 0x7ff3f80ce550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009e50 tx=0x7ff3f002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a0518b9a121b6b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.739+0000 7ff3fcd11640 1 -- 172.21.15.84:0/2520173488 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f002cb00 con 0x7ff3f80c9360 2023-10-11T06:12:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fcd11640 1 -- 172.21.15.84:0/2520173488 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f002cc60 con 0x7ff3f80c9360 2023-10-11T06:12:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fcd11640 1 -- 172.21.15.84:0/2520173488 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f00343f0 con 0x7ff3f80c9360 2023-10-11T06:12:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 -- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80c9360 msgr2=0x7ff3f80ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80c9360 0x7ff3f80ce550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009e50 tx=0x7ff3f002bf60 comp rx=0 tx=0).stop 2023-10-11T06:12:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 -- 172.21.15.84:0/2520173488 shutdown_connections 2023-10-11T06:12:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80c9360 0x7ff3f80ce550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2520173488 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 -- 172.21.15.84:0/2520173488 >> 172.21.15.84:0/2520173488 conn(0x7ff3f8069c70 msgr2=0x7ff3f806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:33.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 -- 172.21.15.84:0/2520173488 shutdown_connections 2023-10-11T06:12:33.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.740+0000 7ff3fff77640 1 -- 172.21.15.84:0/2520173488 wait complete. 2023-10-11T06:12:33.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.742+0000 7ff3fff77640 1 Processor -- start 2023-10-11T06:12:33.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.742+0000 7ff3fff77640 1 -- start start 2023-10-11T06:12:33.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.743+0000 7ff3fff77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3f80c9360 0x7ff3f8149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.743+0000 7ff3fff77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80ca300 0x7ff3f80c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.743+0000 7ff3fff77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f8149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.743+0000 7ff3fff77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80c7ab0 con 0x7ff3f80c9360 2023-10-11T06:12:33.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.743+0000 7ff3fff77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80c5c10 con 0x7ff3f80ca300 2023-10-11T06:12:33.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.743+0000 7ff3fff77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f815cc30 con 0x7ff3f815d4f0 2023-10-11T06:12:33.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3f80c9360 0x7ff3f8149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3f80c9360 0x7ff3f8149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54500/0 (socket says 172.21.15.84:54500) 2023-10-11T06:12:33.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 -- 172.21.15.84:0/2358760506 learned_addr learned my addr 172.21.15.84:0/2358760506 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:33.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 -- 172.21.15.84:0/2358760506 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 msgr2=0x7ff3f8149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f8149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 -- 172.21.15.84:0/2358760506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80ca300 msgr2=0x7ff3f80c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:33.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80ca300 0x7ff3f80c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 -- 172.21.15.84:0/2358760506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f40093a0 con 0x7ff3f80c9360 2023-10-11T06:12:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3fdd13640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3f80c9360 0x7ff3f8149d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e9e0 tx=0x7ff3f400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.744+0000 7ff3ee7fc640 1 -- 172.21.15.84:0/2358760506 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4005990 con 0x7ff3f80c9360 2023-10-11T06:12:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.745+0000 7ff3ee7fc640 1 -- 172.21.15.84:0/2358760506 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4005af0 con 0x7ff3f80c9360 2023-10-11T06:12:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.745+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f0009b30 con 0x7ff3f80c9360 2023-10-11T06:12:33.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.745+0000 7ff3ee7fc640 1 -- 172.21.15.84:0/2358760506 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4005e80 con 0x7ff3f80c9360 2023-10-11T06:12:33.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.745+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f80c76d0 con 0x7ff3f80c9360 2023-10-11T06:12:33.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.746+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f8073ed0 con 0x7ff3f80c9360 2023-10-11T06:12:33.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.747+0000 7ff3ee7fc640 1 -- 172.21.15.84:0/2358760506 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff3f400c970 con 0x7ff3f80c9360 2023-10-11T06:12:33.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.748+0000 7ff3ee7fc640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3dc06a830 0x7ff3dc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:33.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.748+0000 7ff3ee7fc640 1 -- 172.21.15.84:0/2358760506 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff3f4088ff0 con 0x7ff3f80c9360 2023-10-11T06:12:33.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.748+0000 7ff3fd512640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3dc06a830 0x7ff3dc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:33.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.749+0000 7ff3fd512640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3dc06a830 0x7ff3dc06cce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8000f60 tx=0x7ff3e80079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:33.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.752+0000 7ff3ee7fc640 1 -- 172.21.15.84:0/2358760506 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff3f40545c0 con 0x7ff3f80c9360 2023-10-11T06:12:33.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:33.992+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff3f80b09b0 con 0x7ff3f80c9360 2023-10-11T06:12:34.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.000+0000 7ff3ee7fc640 1 -- 172.21.15.84:0/2358760506 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff3f4057be0 con 0x7ff3f80c9360 2023-10-11T06:12:34.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3dc06a830 msgr2=0x7ff3dc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3dc06a830 0x7ff3dc06cce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8000f60 tx=0x7ff3e80079b0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3f80c9360 msgr2=0x7ff3f8149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3f80c9360 0x7ff3f8149d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e9e0 tx=0x7ff3f400eea0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 shutdown_connections 2023-10-11T06:12:34.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3dc06a830 0x7ff3dc06cce0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3f815d4f0 0x7ff3f8149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f80ca300 0x7ff3f80c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 --2- 172.21.15.84:0/2358760506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3f80c9360 0x7ff3f8149d00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.002+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 >> 172.21.15.84:0/2358760506 conn(0x7ff3f8069c70 msgr2=0x7ff3f80af700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:34.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.003+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 shutdown_connections 2023-10-11T06:12:34.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.003+0000 7ff3fff77640 1 -- 172.21.15.84:0/2358760506 wait complete. 2023-10-11T06:12:34.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-11T06:12:34.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.220+0000 7fddc9967640 1 Processor -- start 2023-10-11T06:12:34.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.220+0000 7fddc9967640 1 -- start start 2023-10-11T06:12:34.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.222+0000 7fddc9967640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddc4157e60 0x7fddc406d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.222+0000 7fddc9967640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fddc40c9900 0x7fddc415a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.222+0000 7fddc9967640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc40cb4b0 con 0x7fddc40c9480 2023-10-11T06:12:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.222+0000 7fddc9967640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc406c860 con 0x7fddc4157e60 2023-10-11T06:12:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.222+0000 7fddc9967640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc406b1e0 con 0x7fddc40c9900 2023-10-11T06:12:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.222+0000 7fddc2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddc4157e60 0x7fddc406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.223+0000 7fddc2ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fddc4157e60 0x7fddc406d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54516/0 (socket says 172.21.15.84:54516) 2023-10-11T06:12:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.223+0000 7fddc2ffd640 1 -- 172.21.15.84:0/620043929 learned_addr learned my addr 172.21.15.84:0/620043929 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.223+0000 7fddc2ffd640 1 -- 172.21.15.84:0/620043929 >> v2:172.21.15.99:3300/0 conn(0x7fddc40c9900 msgr2=0x7fddc415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.223+0000 7fddc2ffd640 1 --2- 172.21.15.84:0/620043929 >> v2:172.21.15.99:3300/0 conn(0x7fddc40c9900 0x7fddc415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.223+0000 7fddc2ffd640 1 -- 172.21.15.84:0/620043929 >> v1:172.21.15.84:6789/0 conn(0x7fddc40c9480 legacy=0x7fddc4070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.223+0000 7fddc2ffd640 1 -- 172.21.15.84:0/620043929 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc406b4c0 con 0x7fddc4157e60 2023-10-11T06:12:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.224+0000 7fddc2ffd640 1 --2- 172.21.15.84:0/620043929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4157e60 0x7fddc406d270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fddc40cae20 tx=0x7fddb402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86088d121eab523b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.224+0000 7fddc27fc640 1 -- 172.21.15.84:0/620043929 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4030870 con 0x7fddc4157e60 2023-10-11T06:12:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.224+0000 7fddc27fc640 1 -- 172.21.15.84:0/620043929 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb40309d0 con 0x7fddc4157e60 2023-10-11T06:12:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.224+0000 7fddc27fc640 1 -- 172.21.15.84:0/620043929 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4030d60 con 0x7fddc4157e60 2023-10-11T06:12:34.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 -- 172.21.15.84:0/620043929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4157e60 msgr2=0x7fddc406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 --2- 172.21.15.84:0/620043929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4157e60 0x7fddc406d270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fddc40cae20 tx=0x7fddb402b9b0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 -- 172.21.15.84:0/620043929 shutdown_connections 2023-10-11T06:12:34.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 --2- 172.21.15.84:0/620043929 >> v2:172.21.15.99:3300/0 conn(0x7fddc40c9900 0x7fddc415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 --2- 172.21.15.84:0/620043929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc4157e60 0x7fddc406d270 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 -- 172.21.15.84:0/620043929 >> 172.21.15.84:0/620043929 conn(0x7fddc40b1630 msgr2=0x7fddc40cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 -- 172.21.15.84:0/620043929 shutdown_connections 2023-10-11T06:12:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.225+0000 7fddc9967640 1 -- 172.21.15.84:0/620043929 wait complete. 2023-10-11T06:12:34.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.227+0000 7fddc9967640 1 Processor -- start 2023-10-11T06:12:34.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.227+0000 7fddc9967640 1 -- start start 2023-10-11T06:12:34.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc9967640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddc40c9480 0x7fddc414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc9967640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc40c9900 0x7fddc4158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc9967640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc4157e60 0x7fddc41592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc9967640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4159b20 con 0x7fddc40c9900 2023-10-11T06:12:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc9967640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4159e00 con 0x7fddc4157e60 2023-10-11T06:12:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc9967640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc415a0e0 con 0x7fddc40c9480 2023-10-11T06:12:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc40c9900 0x7fddc4158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddbbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc40c9900 0x7fddc4158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54518/0 (socket says 172.21.15.84:54518) 2023-10-11T06:12:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddbbfff640 1 -- 172.21.15.84:0/2129260078 learned_addr learned my addr 172.21.15.84:0/2129260078 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc2ffd640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddc40c9480 0x7fddc414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddbbfff640 1 -- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddc40c9480 msgr2=0x7fddc414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddbbfff640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddc40c9480 0x7fddc414c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddbbfff640 1 -- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc4157e60 msgr2=0x7fddc41592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.228+0000 7fddc37fe640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc4157e60 0x7fddc41592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddbbfff640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc4157e60 0x7fddc41592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddbbfff640 1 -- 172.21.15.84:0/2129260078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40094d0 con 0x7fddc40c9900 2023-10-11T06:12:34.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddbbfff640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc40c9900 0x7fddc4158d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c400 tx=0x7fddac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:34.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddc0ff9640 1 -- 172.21.15.84:0/2129260078 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac0052d0 con 0x7fddc40c9900 2023-10-11T06:12:34.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddc0ff9640 1 -- 172.21.15.84:0/2129260078 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac005430 con 0x7fddc40c9900 2023-10-11T06:12:34.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc4154550 con 0x7fddc40c9900 2023-10-11T06:12:34.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddc0ff9640 1 -- 172.21.15.84:0/2129260078 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac003830 con 0x7fddc40c9900 2023-10-11T06:12:34.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.229+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc41597e0 con 0x7fddc40c9900 2023-10-11T06:12:34.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.230+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc4154740 con 0x7fddc40c9900 2023-10-11T06:12:34.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.231+0000 7fddc0ff9640 1 -- 172.21.15.84:0/2129260078 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fddac003990 con 0x7fddc40c9900 2023-10-11T06:12:34.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.232+0000 7fddc0ff9640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda406a860 0x7fdda406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.232+0000 7fddc0ff9640 1 -- 172.21.15.84:0/2129260078 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fddac088230 con 0x7fddc40c9900 2023-10-11T06:12:34.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.232+0000 7fddc2ffd640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda406a860 0x7fdda406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.233+0000 7fddc2ffd640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda406a860 0x7fdda406cd10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fddb4036000 tx=0x7fddb403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:34.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.236+0000 7fddc0ff9640 1 -- 172.21.15.84:0/2129260078 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fddac0537f0 con 0x7fddc40c9900 2023-10-11T06:12:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.485+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fddc406ca40 con 0x7fddc40c9900 2023-10-11T06:12:34.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.492+0000 7fddc0ff9640 1 -- 172.21.15.84:0/2129260078 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fddac056e10 con 0x7fddc40c9900 2023-10-11T06:12:34.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.494+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda406a860 msgr2=0x7fdda406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.494+0000 7fddc9967640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda406a860 0x7fdda406cd10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fddb4036000 tx=0x7fddb403b040 comp rx=0 tx=0).stop 2023-10-11T06:12:34.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.494+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc40c9900 msgr2=0x7fddc4158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.494+0000 7fddc9967640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc40c9900 0x7fddc4158d60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c400 tx=0x7fddac00c8c0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 shutdown_connections 2023-10-11T06:12:34.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda406a860 0x7fdda406cd10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc4157e60 0x7fddc41592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fddc40c9900 0x7fddc4158d60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 --2- 172.21.15.84:0/2129260078 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddc40c9480 0x7fddc414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 >> 172.21.15.84:0/2129260078 conn(0x7fddc40b1630 msgr2=0x7fddc40cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:34.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 shutdown_connections 2023-10-11T06:12:34.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.495+0000 7fddc9967640 1 -- 172.21.15.84:0/2129260078 wait complete. 2023-10-11T06:12:34.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-11T06:12:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.713+0000 7f9f6bdab640 1 Processor -- start 2023-10-11T06:12:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.714+0000 7f9f6bdab640 1 -- start start 2023-10-11T06:12:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.715+0000 7f9f6bdab640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f6415d4f0 0x7f9f640cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.715+0000 7f9f6bdab640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f640ca300 0x7f9f640c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.715+0000 7f9f6bdab640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f64071a90 con 0x7f9f640c9190 2023-10-11T06:12:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.715+0000 7f9f6bdab640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f64072310 con 0x7f9f6415d4f0 2023-10-11T06:12:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.715+0000 7f9f6bdab640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f64070280 con 0x7f9f640ca300 2023-10-11T06:12:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.716+0000 7f9f69b47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f6415d4f0 0x7f9f640cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.716+0000 7f9f69b47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f6415d4f0 0x7f9f640cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54528/0 (socket says 172.21.15.84:54528) 2023-10-11T06:12:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.716+0000 7f9f69b47640 1 -- 172.21.15.84:0/2554111991 learned_addr learned my addr 172.21.15.84:0/2554111991 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.716+0000 7f9f69b47640 1 -- 172.21.15.84:0/2554111991 >> v2:172.21.15.99:3300/0 conn(0x7f9f640ca300 msgr2=0x7f9f640c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.716+0000 7f9f69b47640 1 --2- 172.21.15.84:0/2554111991 >> v2:172.21.15.99:3300/0 conn(0x7f9f640ca300 0x7f9f640c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.716+0000 7f9f69b47640 1 -- 172.21.15.84:0/2554111991 >> v1:172.21.15.99:6789/0 conn(0x7f9f640c9190 legacy=0x7f9f6415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.716+0000 7f9f69b47640 1 -- 172.21.15.84:0/2554111991 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f64071cc0 con 0x7f9f6415d4f0 2023-10-11T06:12:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.717+0000 7f9f69b47640 1 --2- 172.21.15.84:0/2554111991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640cb540 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9f640b0980 tx=0x7f9f6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa20499cd343c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.717+0000 7f9f68b45640 1 -- 172.21.15.84:0/2554111991 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60004230 con 0x7f9f6415d4f0 2023-10-11T06:12:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.717+0000 7f9f68b45640 1 -- 172.21.15.84:0/2554111991 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f60004390 con 0x7f9f6415d4f0 2023-10-11T06:12:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.717+0000 7f9f68b45640 1 -- 172.21.15.84:0/2554111991 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60003810 con 0x7f9f6415d4f0 2023-10-11T06:12:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 -- 172.21.15.84:0/2554111991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 msgr2=0x7f9f640cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/2554111991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640cb540 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9f640b0980 tx=0x7f9f6002b8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 -- 172.21.15.84:0/2554111991 shutdown_connections 2023-10-11T06:12:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/2554111991 >> v2:172.21.15.99:3300/0 conn(0x7f9f640ca300 0x7f9f640c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/2554111991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640cb540 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 -- 172.21.15.84:0/2554111991 >> 172.21.15.84:0/2554111991 conn(0x7f9f64069c70 msgr2=0x7f9f6406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:34.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 -- 172.21.15.84:0/2554111991 shutdown_connections 2023-10-11T06:12:34.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.718+0000 7f9f6bdab640 1 -- 172.21.15.84:0/2554111991 wait complete. 2023-10-11T06:12:34.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.720+0000 7f9f6bdab640 1 Processor -- start 2023-10-11T06:12:34.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.720+0000 7f9f6bdab640 1 -- start start 2023-10-11T06:12:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6bdab640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f640c9190 0x7f9f640c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6bdab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f640ca300 0x7f9f6415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6bdab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6bdab640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640c74b0 con 0x7f9f6415d4f0 2023-10-11T06:12:34.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6bdab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6415b000 con 0x7f9f640ca300 2023-10-11T06:12:34.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6bdab640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641ef990 con 0x7f9f640c9190 2023-10-11T06:12:34.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6a348640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6a348640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640c60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54538/0 (socket says 172.21.15.84:54538) 2023-10-11T06:12:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.721+0000 7f9f6a348640 1 -- 172.21.15.84:0/3323602416 learned_addr learned my addr 172.21.15.84:0/3323602416 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f69b47640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f640c9190 0x7f9f640c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6a348640 1 -- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f640c9190 msgr2=0x7f9f640c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6a348640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f640c9190 0x7f9f640c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6a348640 1 -- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f640ca300 msgr2=0x7f9f6415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f69346640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f640ca300 0x7f9f6415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6a348640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f640ca300 0x7f9f6415aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6a348640 1 -- 172.21.15.84:0/3323602416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f60009460 con 0x7f9f6415d4f0 2023-10-11T06:12:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6a348640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640c60e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00b990 tx=0x7f9f5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f5a7fc640 1 -- 172.21.15.84:0/3323602416 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00cba0 con 0x7f9f6415d4f0 2023-10-11T06:12:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f5a7fc640 1 -- 172.21.15.84:0/3323602416 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00cd00 con 0x7f9f6415d4f0 2023-10-11T06:12:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f5a7fc640 1 -- 172.21.15.84:0/3323602416 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c014680 con 0x7f9f6415d4f0 2023-10-11T06:12:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f640c6620 con 0x7f9f6415d4f0 2023-10-11T06:12:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.722+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641f04d0 con 0x7f9f6415d4f0 2023-10-11T06:12:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.723+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f641f0660 con 0x7f9f6415d4f0 2023-10-11T06:12:34.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.724+0000 7f9f5a7fc640 1 -- 172.21.15.84:0/3323602416 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9f5c01d3b0 con 0x7f9f6415d4f0 2023-10-11T06:12:34.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.725+0000 7f9f5a7fc640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f4806a830 0x7f9f4806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.725+0000 7f9f5a7fc640 1 -- 172.21.15.84:0/3323602416 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9f5c088b10 con 0x7f9f6415d4f0 2023-10-11T06:12:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.725+0000 7f9f69b47640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f4806a830 0x7f9f4806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.726+0000 7f9f69b47640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f4806a830 0x7f9f4806cce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f60035000 tx=0x7f9f60008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.729+0000 7f9f5a7fc640 1 -- 172.21.15.84:0/3323602416 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9f5c054040 con 0x7f9f6415d4f0 2023-10-11T06:12:34.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.970+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9f640c6620 con 0x7f9f6415d4f0 2023-10-11T06:12:34.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.978+0000 7f9f5a7fc640 1 -- 172.21.15.84:0/3323602416 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9f640c6620 con 0x7f9f6415d4f0 2023-10-11T06:12:34.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f4806a830 msgr2=0x7f9f4806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f4806a830 0x7f9f4806cce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9f60035000 tx=0x7f9f60008030 comp rx=0 tx=0).stop 2023-10-11T06:12:34.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 msgr2=0x7f9f640c60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:34.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640c60e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00b990 tx=0x7f9f5c00be50 comp rx=0 tx=0).stop 2023-10-11T06:12:34.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 shutdown_connections 2023-10-11T06:12:34.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9f4806a830 0x7f9f4806cce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f6415d4f0 0x7f9f640c60e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f640ca300 0x7f9f6415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 --2- 172.21.15.84:0/3323602416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f640c9190 0x7f9f640c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:34.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 >> 172.21.15.84:0/3323602416 conn(0x7f9f64069c70 msgr2=0x7f9f640b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:34.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 shutdown_connections 2023-10-11T06:12:34.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:34.981+0000 7f9f6bdab640 1 -- 172.21.15.84:0/3323602416 wait complete. 2023-10-11T06:12:35.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-11T06:12:35.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.211+0000 7fc38678e640 1 Processor -- start 2023-10-11T06:12:35.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.212+0000 7fc38678e640 1 -- start start 2023-10-11T06:12:35.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.213+0000 7fc38678e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc38015d4f0 0x7fc3800cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.213+0000 7fc38678e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 0x7fc3800c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.213+0000 7fc38678e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc380071a90 con 0x7fc3800c9190 2023-10-11T06:12:35.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.213+0000 7fc38678e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc380072310 con 0x7fc38015d4f0 2023-10-11T06:12:35.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.213+0000 7fc38678e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc380070280 con 0x7fc3800ca300 2023-10-11T06:12:35.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc384d2b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc3800c9190 0x7fc38015dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34632/0 (socket says 172.21.15.84:34632) 2023-10-11T06:12:35.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc384d2b640 1 -- 172.21.15.84:0/2116322941 learned_addr learned my addr 172.21.15.84:0/2116322941 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:35.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37ffff640 1 --2- 172.21.15.84:0/2116322941 >> v2:172.21.15.99:3300/0 conn(0x7fc38015d4f0 0x7fc3800cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc377fff640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 0x7fc3800c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634862901 0 0) 0x7fc380071a90 con 0x7fc3800c9190 2023-10-11T06:12:35.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc36c003520 con 0x7fc3800c9190 2023-10-11T06:12:35.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218654543 0 0) 0x7fc36c003520 con 0x7fc3800c9190 2023-10-11T06:12:35.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 msgr2=0x7fc3800c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37f7fe640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 0x7fc3800c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 >> v2:172.21.15.99:3300/0 conn(0x7fc38015d4f0 msgr2=0x7fc3800cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.214+0000 7fc37f7fe640 1 --2- 172.21.15.84:0/2116322941 >> v2:172.21.15.99:3300/0 conn(0x7fc38015d4f0 0x7fc3800cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc380071cc0 con 0x7fc3800c9190 2023-10-11T06:12:35.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fc3780019f0 con 0x7fc3800c9190 2023-10-11T06:12:35.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 >> v1:172.21.15.84:6789/0 conn(0x7fc3800c9190 legacy=0x7fc38015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 0x7fc36c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c003ef0 0x7fc36c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc36c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380072310 con 0x7fc36c004d00 2023-10-11T06:12:35.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380070280 con 0x7fc36c003ef0 2023-10-11T06:12:35.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c00b980 con 0x7fc3800ca300 2023-10-11T06:12:35.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc384d2b640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc36c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc37ffff640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 0x7fc36c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc384d2b640 1 -- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 msgr2=0x7fc36c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc384d2b640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3800ca300 0x7fc36c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc384d2b640 1 -- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c003ef0 msgr2=0x7fc36c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc384d2b640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc36c003ef0 0x7fc36c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.215+0000 7fc384d2b640 1 -- 172.21.15.84:0/2116322941 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c003520 con 0x7fc36c004d00 2023-10-11T06:12:35.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.216+0000 7fc384d2b640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc36c005170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc37800a8d0 tx=0x7fc36c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f4f67820f0f682 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:35.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378034630 con 0x7fc36c004d00 2023-10-11T06:12:35.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37802f070 con 0x7fc36c004d00 2023-10-11T06:12:35.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc37f7fe640 1 -- 172.21.15.84:0/2116322941 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3780349d0 con 0x7fc36c004d00 2023-10-11T06:12:35.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc38678e640 1 -- 172.21.15.84:0/2116322941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 msgr2=0x7fc36c005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc38678e640 1 --2- 172.21.15.84:0/2116322941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc36c005170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc37800a8d0 tx=0x7fc36c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc384d2b640 1 -- 172.21.15.84:0/2116322941 reap_dead start 2023-10-11T06:12:35.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc38678e640 1 -- 172.21.15.84:0/2116322941 shutdown_connections 2023-10-11T06:12:35.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc38678e640 1 -- 172.21.15.84:0/2116322941 >> 172.21.15.84:0/2116322941 conn(0x7fc380069c70 msgr2=0x7fc38006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:35.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc38678e640 1 -- 172.21.15.84:0/2116322941 shutdown_connections 2023-10-11T06:12:35.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.217+0000 7fc38678e640 1 -- 172.21.15.84:0/2116322941 wait complete. 2023-10-11T06:12:35.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.219+0000 7fc38678e640 1 Processor -- start 2023-10-11T06:12:35.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.219+0000 7fc38678e640 1 -- start start 2023-10-11T06:12:35.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc38678e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc38015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc38678e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3801591b0 0x7fc3800c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc38678e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3800c6c60 0x7fc3800ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc38678e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38015a8d0 con 0x7fc36c004d00 2023-10-11T06:12:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc38678e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380157020 con 0x7fc3800c6c60 2023-10-11T06:12:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc38678e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3800c76c0 con 0x7fc3801591b0 2023-10-11T06:12:35.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc37ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc38015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc37ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc38015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54558/0 (socket says 172.21.15.84:54558) 2023-10-11T06:12:35.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.220+0000 7fc37ffff640 1 -- 172.21.15.84:0/2463146735 learned_addr learned my addr 172.21.15.84:0/2463146735 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:35.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37ffff640 1 -- 172.21.15.84:0/2463146735 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3801591b0 msgr2=0x7fc3800c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37ffff640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3801591b0 0x7fc3800c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37ffff640 1 -- 172.21.15.84:0/2463146735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3800c6c60 msgr2=0x7fc3800ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37ffff640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3800c6c60 0x7fc3800ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37ffff640 1 -- 172.21.15.84:0/2463146735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800c7100 con 0x7fc36c004d00 2023-10-11T06:12:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37ffff640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc38015a390 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3700116d0 tx=0x7fc370011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37d7fa640 1 -- 172.21.15.84:0/2463146735 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37000f940 con 0x7fc36c004d00 2023-10-11T06:12:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37d7fa640 1 -- 172.21.15.84:0/2463146735 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37000faa0 con 0x7fc36c004d00 2023-10-11T06:12:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc37d7fa640 1 -- 172.21.15.84:0/2463146735 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37000fe10 con 0x7fc36c004d00 2023-10-11T06:12:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3800c73e0 con 0x7fc36c004d00 2023-10-11T06:12:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.221+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc380158320 con 0x7fc36c004d00 2023-10-11T06:12:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.223+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc380158540 con 0x7fc36c004d00 2023-10-11T06:12:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.224+0000 7fc37d7fa640 1 -- 172.21.15.84:0/2463146735 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc370015460 con 0x7fc36c004d00 2023-10-11T06:12:35.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.224+0000 7fc37d7fa640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc36006a860 0x7fc36006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.225+0000 7fc37d7fa640 1 -- 172.21.15.84:0/2463146735 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc3700894f0 con 0x7fc36c004d00 2023-10-11T06:12:35.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.225+0000 7fc377fff640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc36006a860 0x7fc36006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.225+0000 7fc377fff640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc36006a860 0x7fc36006cd10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc3800c8220 tx=0x7fc3680094e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:35.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.229+0000 7fc37d7fa640 1 -- 172.21.15.84:0/2463146735 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc370054c00 con 0x7fc36c004d00 2023-10-11T06:12:35.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.471+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc380073ed0 con 0x7fc36c004d00 2023-10-11T06:12:35.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.483+0000 7fc37d7fa640 1 -- 172.21.15.84:0/2463146735 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc3700220a0 con 0x7fc36c004d00 2023-10-11T06:12:35.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc36006a860 msgr2=0x7fc36006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc36006a860 0x7fc36006cd10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc3800c8220 tx=0x7fc3680094e0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 msgr2=0x7fc38015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc38015a390 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3700116d0 tx=0x7fc370011b90 comp rx=0 tx=0).stop 2023-10-11T06:12:35.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 shutdown_connections 2023-10-11T06:12:35.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc36006a860 0x7fc36006cd10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3800c6c60 0x7fc3800ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3801591b0 0x7fc3800c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 --2- 172.21.15.84:0/2463146735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc36c004d00 0x7fc38015a390 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 >> 172.21.15.84:0/2463146735 conn(0x7fc380069c70 msgr2=0x7fc38015e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:35.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 shutdown_connections 2023-10-11T06:12:35.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.486+0000 7fc38678e640 1 -- 172.21.15.84:0/2463146735 wait complete. 2023-10-11T06:12:35.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-11T06:12:35.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.709+0000 7fe14abc9640 1 Processor -- start 2023-10-11T06:12:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.709+0000 7fe14abc9640 1 -- start start 2023-10-11T06:12:35.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.710+0000 7fe14abc9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe14415d4f0 0x7fe1440cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.710+0000 7fe14abc9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440ca300 0x7fe1440c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.710+0000 7fe14abc9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe144071a90 con 0x7fe1440c9190 2023-10-11T06:12:35.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe14abc9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe144072310 con 0x7fe14415d4f0 2023-10-11T06:12:35.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe14abc9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe144070280 con 0x7fe1440ca300 2023-10-11T06:12:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe149166640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe1440c9190 0x7fe14415dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34634/0 (socket says 172.21.15.84:34634) 2023-10-11T06:12:35.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe149166640 1 -- 172.21.15.84:0/3980900518 learned_addr learned my addr 172.21.15.84:0/3980900518 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe148965640 1 --2- 172.21.15.84:0/3980900518 >> v2:172.21.15.99:3300/0 conn(0x7fe14415d4f0 0x7fe1440cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe143fff640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440ca300 0x7fe1440c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591982993 0 0) 0x7fe144071a90 con 0x7fe1440c9190 2023-10-11T06:12:35.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.711+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe134003520 con 0x7fe1440c9190 2023-10-11T06:12:35.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166214251 0 0) 0x7fe134003520 con 0x7fe1440c9190 2023-10-11T06:12:35.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440ca300 msgr2=0x7fe1440c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440ca300 0x7fe1440c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 >> v2:172.21.15.99:3300/0 conn(0x7fe14415d4f0 msgr2=0x7fe1440cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 --2- 172.21.15.84:0/3980900518 >> v2:172.21.15.99:3300/0 conn(0x7fe14415d4f0 0x7fe1440cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144071cc0 con 0x7fe1440c9190 2023-10-11T06:12:35.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fe13c0019f0 con 0x7fe1440c9190 2023-10-11T06:12:35.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 >> v1:172.21.15.84:6789/0 conn(0x7fe1440c9190 legacy=0x7fe14415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440ca300 0x7fe134003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134003ef0 0x7fe1340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe134004d00 0x7fe134005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144072310 con 0x7fe134004d00 2023-10-11T06:12:35.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144070280 con 0x7fe134003ef0 2023-10-11T06:12:35.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13400b980 con 0x7fe1440ca300 2023-10-11T06:12:35.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe149166640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe134004d00 0x7fe134005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.712+0000 7fe143fff640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134003ef0 0x7fe1340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.713+0000 7fe149166640 1 -- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440ca300 msgr2=0x7fe134003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.713+0000 7fe149166640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440ca300 0x7fe134003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.713+0000 7fe149166640 1 -- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134003ef0 msgr2=0x7fe1340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:35.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.713+0000 7fe149166640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134003ef0 0x7fe1340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.713+0000 7fe149166640 1 -- 172.21.15.84:0/3980900518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe134003520 con 0x7fe134004d00 2023-10-11T06:12:35.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.714+0000 7fe149166640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe134004d00 0x7fe134005170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00a860 tx=0x7fe1340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb010c7237af4fac server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.714+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c034630 con 0x7fe134004d00 2023-10-11T06:12:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.714+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c02f070 con 0x7fe134004d00 2023-10-11T06:12:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.714+0000 7fe1437fe640 1 -- 172.21.15.84:0/3980900518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c034a30 con 0x7fe134004d00 2023-10-11T06:12:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.714+0000 7fe14abc9640 1 -- 172.21.15.84:0/3980900518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe134004d00 msgr2=0x7fe134005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.714+0000 7fe14abc9640 1 --2- 172.21.15.84:0/3980900518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe134004d00 0x7fe134005170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00a860 tx=0x7fe1340057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.714+0000 7fe149166640 1 -- 172.21.15.84:0/3980900518 reap_dead start 2023-10-11T06:12:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.715+0000 7fe14abc9640 1 -- 172.21.15.84:0/3980900518 shutdown_connections 2023-10-11T06:12:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.715+0000 7fe14abc9640 1 -- 172.21.15.84:0/3980900518 >> 172.21.15.84:0/3980900518 conn(0x7fe144069c70 msgr2=0x7fe14406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.715+0000 7fe14abc9640 1 -- 172.21.15.84:0/3980900518 shutdown_connections 2023-10-11T06:12:35.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.715+0000 7fe14abc9640 1 -- 172.21.15.84:0/3980900518 wait complete. 2023-10-11T06:12:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.716+0000 7fe14abc9640 1 Processor -- start 2023-10-11T06:12:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.717+0000 7fe14abc9640 1 -- start start 2023-10-11T06:12:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe14abc9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134004d00 0x7fe1440c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe14abc9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440cdbf0 0x7fe1441489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe14abc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1440b2e30 0x7fe1440b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe14abc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440cc1d0 con 0x7fe1440b2e30 2023-10-11T06:12:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe14abc9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440b37e0 con 0x7fe134004d00 2023-10-11T06:12:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe14abc9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c7db0 con 0x7fe1440cdbf0 2023-10-11T06:12:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1440b2e30 0x7fe1440b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1440b2e30 0x7fe1440b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54574/0 (socket says 172.21.15.84:54574) 2023-10-11T06:12:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 -- 172.21.15.84:0/2615863002 learned_addr learned my addr 172.21.15.84:0/2615863002 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe143fff640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440cdbf0 0x7fe1441489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 -- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440cdbf0 msgr2=0x7fe1441489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440cdbf0 0x7fe1441489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe148965640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134004d00 0x7fe1440c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 -- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134004d00 msgr2=0x7fe1440c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134004d00 0x7fe1440c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe149166640 1 -- 172.21.15.84:0/2615863002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440c8090 con 0x7fe1440b2e30 2023-10-11T06:12:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.718+0000 7fe143fff640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440cdbf0 0x7fe1441489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.719+0000 7fe149166640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1440b2e30 0x7fe1440b32a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe13c02cba0 tx=0x7fe13c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.719+0000 7fe1417fa640 1 -- 172.21.15.84:0/2615863002 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c034840 con 0x7fe1440b2e30 2023-10-11T06:12:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.719+0000 7fe1417fa640 1 -- 172.21.15.84:0/2615863002 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c0349a0 con 0x7fe1440b2e30 2023-10-11T06:12:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.719+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440c8370 con 0x7fe1440b2e30 2023-10-11T06:12:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.719+0000 7fe1417fa640 1 -- 172.21.15.84:0/2615863002 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c034d30 con 0x7fe1440b2e30 2023-10-11T06:12:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.719+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1440ce060 con 0x7fe1440b2e30 2023-10-11T06:12:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.720+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1440c8560 con 0x7fe1440b2e30 2023-10-11T06:12:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.721+0000 7fe1417fa640 1 -- 172.21.15.84:0/2615863002 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe13c031030 con 0x7fe1440b2e30 2023-10-11T06:12:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.722+0000 7fe1417fa640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe12806a830 0x7fe12806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.722+0000 7fe1417fa640 1 -- 172.21.15.84:0/2615863002 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe13c0aa410 con 0x7fe1440b2e30 2023-10-11T06:12:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.722+0000 7fe148965640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe12806a830 0x7fe12806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.723+0000 7fe148965640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe12806a830 0x7fe12806cce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1380047b0 tx=0x7fe138008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.726+0000 7fe1417fa640 1 -- 172.21.15.84:0/2615863002 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe1440c8560 con 0x7fe1440b2e30 2023-10-11T06:12:35.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.976+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe1440c8560 con 0x7fe1440b2e30 2023-10-11T06:12:35.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.983+0000 7fe1417fa640 1 -- 172.21.15.84:0/2615863002 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe1440c8560 con 0x7fe1440b2e30 2023-10-11T06:12:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.985+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe12806a830 msgr2=0x7fe12806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.985+0000 7fe14abc9640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe12806a830 0x7fe12806cce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1380047b0 tx=0x7fe138008040 comp rx=0 tx=0).stop 2023-10-11T06:12:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.985+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1440b2e30 msgr2=0x7fe1440b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.985+0000 7fe14abc9640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1440b2e30 0x7fe1440b32a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe13c02cba0 tx=0x7fe13c001d80 comp rx=0 tx=0).stop 2023-10-11T06:12:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 shutdown_connections 2023-10-11T06:12:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe12806a830 0x7fe12806cce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1440b2e30 0x7fe1440b32a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1440cdbf0 0x7fe1441489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 --2- 172.21.15.84:0/2615863002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134004d00 0x7fe1440c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 >> 172.21.15.84:0/2615863002 conn(0x7fe144069c70 msgr2=0x7fe1440ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 shutdown_connections 2023-10-11T06:12:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:35.986+0000 7fe14abc9640 1 -- 172.21.15.84:0/2615863002 wait complete. 2023-10-11T06:12:36.005 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:36.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.216+0000 7fba3c08f640 1 Processor -- start 2023-10-11T06:12:36.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.216+0000 7fba3c08f640 1 -- start start 2023-10-11T06:12:36.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba3c08f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fba340ce0d0 0x7fba340c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba3c08f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340adb90 0x7fba340caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba3c08f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba341593f0 con 0x7fba340cd300 2023-10-11T06:12:36.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba3c08f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3415a340 con 0x7fba340ce0d0 2023-10-11T06:12:36.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba3c08f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba340ce4a0 con 0x7fba340adb90 2023-10-11T06:12:36.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba39e2b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fba340ce0d0 0x7fba340c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba39e2b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fba340ce0d0 0x7fba340c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54588/0 (socket says 172.21.15.84:54588) 2023-10-11T06:12:36.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba39e2b640 1 -- 172.21.15.84:0/4221336072 learned_addr learned my addr 172.21.15.84:0/4221336072 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:36.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.218+0000 7fba3962a640 1 --2- 172.21.15.84:0/4221336072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340adb90 0x7fba340caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.219+0000 7fba39e2b640 1 -- 172.21.15.84:0/4221336072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340adb90 msgr2=0x7fba340caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:36.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.219+0000 7fba39e2b640 1 --2- 172.21.15.84:0/4221336072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340adb90 0x7fba340caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.219+0000 7fba39e2b640 1 -- 172.21.15.84:0/4221336072 >> v1:172.21.15.99:6789/0 conn(0x7fba340cd300 legacy=0x7fba340b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:36.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.219+0000 7fba39e2b640 1 -- 172.21.15.84:0/4221336072 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba341596c0 con 0x7fba340ce0d0 2023-10-11T06:12:36.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.219+0000 7fba3962a640 1 --2- 172.21.15.84:0/4221336072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340adb90 0x7fba340caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:36.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.220+0000 7fba39e2b640 1 --2- 172.21.15.84:0/4221336072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340ce0d0 0x7fba340c8cc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fba3000baf0 tx=0x7fba3002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c00c5797d7b73a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:36.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.220+0000 7fba38e29640 1 -- 172.21.15.84:0/4221336072 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3002ee50 con 0x7fba340ce0d0 2023-10-11T06:12:36.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.220+0000 7fba38e29640 1 -- 172.21.15.84:0/4221336072 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30004030 con 0x7fba340ce0d0 2023-10-11T06:12:36.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.220+0000 7fba38e29640 1 -- 172.21.15.84:0/4221336072 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30004420 con 0x7fba340ce0d0 2023-10-11T06:12:36.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 -- 172.21.15.84:0/4221336072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340ce0d0 msgr2=0x7fba340c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 --2- 172.21.15.84:0/4221336072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340ce0d0 0x7fba340c8cc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fba3000baf0 tx=0x7fba3002e0a0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 -- 172.21.15.84:0/4221336072 shutdown_connections 2023-10-11T06:12:36.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 --2- 172.21.15.84:0/4221336072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340adb90 0x7fba340caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 --2- 172.21.15.84:0/4221336072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340ce0d0 0x7fba340c8cc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 -- 172.21.15.84:0/4221336072 >> 172.21.15.84:0/4221336072 conn(0x7fba34069c70 msgr2=0x7fba3406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:36.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 -- 172.21.15.84:0/4221336072 shutdown_connections 2023-10-11T06:12:36.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.221+0000 7fba3c08f640 1 -- 172.21.15.84:0/4221336072 wait complete. 2023-10-11T06:12:36.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.223+0000 7fba3c08f640 1 Processor -- start 2023-10-11T06:12:36.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.223+0000 7fba3c08f640 1 -- start start 2023-10-11T06:12:36.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba3c08f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340adb90 0x7fba341ed430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba3c08f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340cd300 0x7fba340c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba3c08f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340ce0d0 0x7fba340b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba3c08f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba340b0ed0 con 0x7fba340adb90 2023-10-11T06:12:36.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba3c08f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba340c2340 con 0x7fba340cd300 2023-10-11T06:12:36.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba3c08f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba340c4580 con 0x7fba340ce0d0 2023-10-11T06:12:36.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba39e2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340adb90 0x7fba341ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba39e2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340adb90 0x7fba341ed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54602/0 (socket says 172.21.15.84:54602) 2023-10-11T06:12:36.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.224+0000 7fba39e2b640 1 -- 172.21.15.84:0/1878126964 learned_addr learned my addr 172.21.15.84:0/1878126964 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:36.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba39e2b640 1 -- 172.21.15.84:0/1878126964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340ce0d0 msgr2=0x7fba340b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba39e2b640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340ce0d0 0x7fba340b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba39e2b640 1 -- 172.21.15.84:0/1878126964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340cd300 msgr2=0x7fba340c1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba39e2b640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340cd300 0x7fba340c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba39e2b640 1 -- 172.21.15.84:0/1878126964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3000b700 con 0x7fba340adb90 2023-10-11T06:12:36.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba39e2b640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340adb90 0x7fba341ed430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b2c0 tx=0x7fba300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:36.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba2a7fc640 1 -- 172.21.15.84:0/1878126964 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30003a20 con 0x7fba340adb90 2023-10-11T06:12:36.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba2a7fc640 1 -- 172.21.15.84:0/1878126964 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30003b80 con 0x7fba340adb90 2023-10-11T06:12:36.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba340c4860 con 0x7fba340adb90 2023-10-11T06:12:36.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba2a7fc640 1 -- 172.21.15.84:0/1878126964 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30006040 con 0x7fba340adb90 2023-10-11T06:12:36.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.225+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba340c4b40 con 0x7fba340adb90 2023-10-11T06:12:36.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.226+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba34155e60 con 0x7fba340adb90 2023-10-11T06:12:36.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.227+0000 7fba2a7fc640 1 -- 172.21.15.84:0/1878126964 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fba300061a0 con 0x7fba340adb90 2023-10-11T06:12:36.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.229+0000 7fba2a7fc640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba1806a830 0x7fba1806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.229+0000 7fba2a7fc640 1 -- 172.21.15.84:0/1878126964 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fba300ae180 con 0x7fba340adb90 2023-10-11T06:12:36.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.229+0000 7fba3962a640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba1806a830 0x7fba1806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.230+0000 7fba3962a640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba1806a830 0x7fba1806cce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba24006fd0 tx=0x7fba24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:36.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.234+0000 7fba2a7fc640 1 -- 172.21.15.84:0/1878126964 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fba300797d0 con 0x7fba340adb90 2023-10-11T06:12:36.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.425+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba34073ed0 con 0x7fba340adb90 2023-10-11T06:12:36.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.427+0000 7fba2a7fc640 1 -- 172.21.15.84:0/1878126964 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+14123 (secure 0 0 0) 0x7fba3007cdf0 con 0x7fba340adb90 2023-10-11T06:12:36.429 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:36.429 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":34,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:32.460759+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:36.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba1806a830 msgr2=0x7fba1806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba1806a830 0x7fba1806cce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba24006fd0 tx=0x7fba24008040 comp rx=0 tx=0).stop 2023-10-11T06:12:36.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340adb90 msgr2=0x7fba341ed430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340adb90 0x7fba341ed430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b2c0 tx=0x7fba300046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 shutdown_connections 2023-10-11T06:12:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba1806a830 0x7fba1806cce0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba340ce0d0 0x7fba340b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba340cd300 0x7fba340c1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 --2- 172.21.15.84:0/1878126964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba340adb90 0x7fba341ed430 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 >> 172.21.15.84:0/1878126964 conn(0x7fba34069c70 msgr2=0x7fba340cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:36.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.430+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 shutdown_connections 2023-10-11T06:12:36.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.431+0000 7fba3c08f640 1 -- 172.21.15.84:0/1878126964 wait complete. 2023-10-11T06:12:36.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:36.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:36.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.671+0000 7f61b4210640 1 Processor -- start 2023-10-11T06:12:36.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.672+0000 7f61b4210640 1 -- start start 2023-10-11T06:12:36.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.673+0000 7f61b4210640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f61ac160830 0x7f61ac0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.673+0000 7f61b4210640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61ac0c9410 0x7f61ac06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.673+0000 7f61b4210640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac0cc9e0 con 0x7f61ac0cf270 2023-10-11T06:12:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.673+0000 7f61b4210640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac06d720 con 0x7f61ac0c9410 2023-10-11T06:12:36.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.673+0000 7f61b4210640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac165410 con 0x7f61ac160830 2023-10-11T06:12:36.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b17ab640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61ac0cf270 0x7f61ac0c8d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34638/0 (socket says 172.21.15.84:34638) 2023-10-11T06:12:36.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b17ab640 1 -- 172.21.15.84:0/3239874790 learned_addr learned my addr 172.21.15.84:0/3239874790 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:36.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b27ad640 1 --2- 172.21.15.84:0/3239874790 >> v2:172.21.15.84:3300/0 conn(0x7f61ac0c9410 0x7f61ac06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b1fac640 1 --2- 172.21.15.84:0/3239874790 >> v2:172.21.15.99:3300/0 conn(0x7f61ac160830 0x7f61ac0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b0faa640 1 -- 172.21.15.84:0/3239874790 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908492951 0 0) 0x7f61ac0cc9e0 con 0x7f61ac0cf270 2023-10-11T06:12:36.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b0faa640 1 -- 172.21.15.84:0/3239874790 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6198003520 con 0x7f61ac0cf270 2023-10-11T06:12:36.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b27ad640 1 -- 172.21.15.84:0/3239874790 >> v2:172.21.15.99:3300/0 conn(0x7f61ac160830 msgr2=0x7f61ac0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:36.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b27ad640 1 --2- 172.21.15.84:0/3239874790 >> v2:172.21.15.99:3300/0 conn(0x7f61ac160830 0x7f61ac0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b27ad640 1 -- 172.21.15.84:0/3239874790 >> v1:172.21.15.84:6789/0 conn(0x7f61ac0cf270 legacy=0x7f61ac0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.674+0000 7f61b27ad640 1 -- 172.21.15.84:0/3239874790 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac06d440 con 0x7f61ac0c9410 2023-10-11T06:12:36.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.675+0000 7f61b27ad640 1 --2- 172.21.15.84:0/3239874790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac06c170 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06e380 tx=0x7f61a402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8ec5cddc7e4680 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:36.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b0faa640 1 -- 172.21.15.84:0/3239874790 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a402cbf0 con 0x7f61ac0c9410 2023-10-11T06:12:36.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b0faa640 1 -- 172.21.15.84:0/3239874790 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a402cd50 con 0x7f61ac0c9410 2023-10-11T06:12:36.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b0faa640 1 -- 172.21.15.84:0/3239874790 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4034640 con 0x7f61ac0c9410 2023-10-11T06:12:36.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 -- 172.21.15.84:0/3239874790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 msgr2=0x7f61ac06c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 --2- 172.21.15.84:0/3239874790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac06c170 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06e380 tx=0x7f61a402be40 comp rx=0 tx=0).stop 2023-10-11T06:12:36.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 -- 172.21.15.84:0/3239874790 shutdown_connections 2023-10-11T06:12:36.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 --2- 172.21.15.84:0/3239874790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac06c170 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 --2- 172.21.15.84:0/3239874790 >> v2:172.21.15.99:3300/0 conn(0x7f61ac160830 0x7f61ac0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 -- 172.21.15.84:0/3239874790 >> 172.21.15.84:0/3239874790 conn(0x7f61ac069c70 msgr2=0x7f61ac06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:36.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 -- 172.21.15.84:0/3239874790 shutdown_connections 2023-10-11T06:12:36.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.676+0000 7f61b4210640 1 -- 172.21.15.84:0/3239874790 wait complete. 2023-10-11T06:12:36.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.678+0000 7f61b4210640 1 Processor -- start 2023-10-11T06:12:36.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.678+0000 7f61b4210640 1 -- start start 2023-10-11T06:12:36.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.679+0000 7f61b4210640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac1f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.679+0000 7f61b4210640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cf270 0x7f61ac15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.679+0000 7f61b4210640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac160830 0x7f61ac15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.679+0000 7f61b4210640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac1f5cf0 con 0x7f61ac0c9410 2023-10-11T06:12:36.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.679+0000 7f61b4210640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac162e30 con 0x7f61ac0cf270 2023-10-11T06:12:36.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.679+0000 7f61b4210640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac163110 con 0x7f61ac160830 2023-10-11T06:12:36.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.679+0000 7f61b1fac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac1f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac1f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54620/0 (socket says 172.21.15.84:54620) 2023-10-11T06:12:36.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 -- 172.21.15.84:0/505594495 learned_addr learned my addr 172.21.15.84:0/505594495 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:36.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b17ab640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cf270 0x7f61ac15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b27ad640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac160830 0x7f61ac15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 -- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac160830 msgr2=0x7f61ac15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac160830 0x7f61ac15b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 -- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cf270 msgr2=0x7f61ac15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cf270 0x7f61ac15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 -- 172.21.15.84:0/505594495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a4009a70 con 0x7f61ac0c9410 2023-10-11T06:12:36.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61b1fac640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac1f57b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f61a800d800 tx=0x7f61a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:36.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.680+0000 7f61a27fc640 1 -- 172.21.15.84:0/505594495 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a800b6c0 con 0x7f61ac0c9410 2023-10-11T06:12:36.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.681+0000 7f61a27fc640 1 -- 172.21.15.84:0/505594495 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a800b820 con 0x7f61ac0c9410 2023-10-11T06:12:36.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.681+0000 7f61a27fc640 1 -- 172.21.15.84:0/505594495 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a800bbb0 con 0x7f61ac0c9410 2023-10-11T06:12:36.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.681+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac1633f0 con 0x7f61ac0c9410 2023-10-11T06:12:36.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.681+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac1639b0 con 0x7f61ac0c9410 2023-10-11T06:12:36.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.682+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac163b40 con 0x7f61ac0c9410 2023-10-11T06:12:36.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.683+0000 7f61a27fc640 1 -- 172.21.15.84:0/505594495 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f61a800bd10 con 0x7f61ac0c9410 2023-10-11T06:12:36.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.684+0000 7f61a27fc640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f619006a830 0x7f619006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:36.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.684+0000 7f61a27fc640 1 -- 172.21.15.84:0/505594495 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f61a8089140 con 0x7f61ac0c9410 2023-10-11T06:12:36.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.684+0000 7f61b17ab640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f619006a830 0x7f619006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:36.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.684+0000 7f61b17ab640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f619006a830 0x7f619006cce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06cc80 tx=0x7f619c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:36.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.688+0000 7f61a27fc640 1 -- 172.21.15.84:0/505594495 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f61a80547a0 con 0x7f61ac0c9410 2023-10-11T06:12:36.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.877+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61ac073ed0 con 0x7f61ac0c9410 2023-10-11T06:12:36.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.878+0000 7f61a27fc640 1 -- 172.21.15.84:0/505594495 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61a8057dc0 con 0x7f61ac0c9410 2023-10-11T06:12:36.879 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:36.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.880+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f619006a830 msgr2=0x7f619006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.880+0000 7f61b4210640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f619006a830 0x7f619006cce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06cc80 tx=0x7f619c009040 comp rx=0 tx=0).stop 2023-10-11T06:12:36.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.880+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 msgr2=0x7f61ac1f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:36.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.880+0000 7f61b4210640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac1f57b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f61a800d800 tx=0x7f61a800dcc0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 shutdown_connections 2023-10-11T06:12:36.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f619006a830 0x7f619006cce0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac160830 0x7f61ac15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cf270 0x7f61ac15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 --2- 172.21.15.84:0/505594495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61ac0c9410 0x7f61ac1f57b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:36.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 >> 172.21.15.84:0/505594495 conn(0x7f61ac069c70 msgr2=0x7f61ac0c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:36.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 shutdown_connections 2023-10-11T06:12:36.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:36.881+0000 7f61b4210640 1 -- 172.21.15.84:0/505594495 wait complete. 2023-10-11T06:12:36.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:37.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.109+0000 7fba62349640 1 Processor -- start 2023-10-11T06:12:37.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.110+0000 7fba62349640 1 -- start start 2023-10-11T06:12:37.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.111+0000 7fba62349640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fba5c1591b0 0x7fba5c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.111+0000 7fba62349640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0cdf90 0x7fba5c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.111+0000 7fba62349640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5c06d6e0 con 0x7fba5c15d930 2023-10-11T06:12:37.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.111+0000 7fba62349640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c159580 con 0x7fba5c1591b0 2023-10-11T06:12:37.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.111+0000 7fba62349640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c06b030 con 0x7fba5c0cdf90 2023-10-11T06:12:37.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.111+0000 7fba5bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fba5c1591b0 0x7fba5c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.112+0000 7fba5bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fba5c1591b0 0x7fba5c15d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54626/0 (socket says 172.21.15.84:54626) 2023-10-11T06:12:37.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.112+0000 7fba5bfff640 1 -- 172.21.15.84:0/4069613530 learned_addr learned my addr 172.21.15.84:0/4069613530 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:37.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.112+0000 7fba5bfff640 1 -- 172.21.15.84:0/4069613530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0cdf90 msgr2=0x7fba5c0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:37.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.112+0000 7fba608e6640 1 --2- 172.21.15.84:0/4069613530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0cdf90 0x7fba5c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.112+0000 7fba5bfff640 1 --2- 172.21.15.84:0/4069613530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0cdf90 0x7fba5c0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.112+0000 7fba5bfff640 1 -- 172.21.15.84:0/4069613530 >> v1:172.21.15.99:6789/0 conn(0x7fba5c15d930 legacy=0x7fba5c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:37.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.112+0000 7fba5bfff640 1 -- 172.21.15.84:0/4069613530 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c15ccc0 con 0x7fba5c1591b0 2023-10-11T06:12:37.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.113+0000 7fba5bfff640 1 --2- 172.21.15.84:0/4069613530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c1591b0 0x7fba5c15d3f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009e50 tx=0x7fba4c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66907b5c94e5afb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:37.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.113+0000 7fba5affd640 1 -- 172.21.15.84:0/4069613530 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c02ca70 con 0x7fba5c1591b0 2023-10-11T06:12:37.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.113+0000 7fba5affd640 1 -- 172.21.15.84:0/4069613530 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c02cbd0 con 0x7fba5c1591b0 2023-10-11T06:12:37.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba5affd640 1 -- 172.21.15.84:0/4069613530 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0343f0 con 0x7fba5c1591b0 2023-10-11T06:12:37.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 -- 172.21.15.84:0/4069613530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c1591b0 msgr2=0x7fba5c15d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 --2- 172.21.15.84:0/4069613530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c1591b0 0x7fba5c15d3f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009e50 tx=0x7fba4c02bed0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 -- 172.21.15.84:0/4069613530 shutdown_connections 2023-10-11T06:12:37.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 --2- 172.21.15.84:0/4069613530 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0cdf90 0x7fba5c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 --2- 172.21.15.84:0/4069613530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c1591b0 0x7fba5c15d3f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 -- 172.21.15.84:0/4069613530 >> 172.21.15.84:0/4069613530 conn(0x7fba5c0b16c0 msgr2=0x7fba5c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:37.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 -- 172.21.15.84:0/4069613530 shutdown_connections 2023-10-11T06:12:37.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.114+0000 7fba62349640 1 -- 172.21.15.84:0/4069613530 wait complete. 2023-10-11T06:12:37.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.116+0000 7fba62349640 1 Processor -- start 2023-10-11T06:12:37.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.116+0000 7fba62349640 1 -- start start 2023-10-11T06:12:37.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba62349640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0cdf90 0x7fba5c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba62349640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c1591b0 0x7fba5c0c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba62349640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c15d930 0x7fba5c0c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba62349640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c14efd0 con 0x7fba5c0cdf90 2023-10-11T06:12:37.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba62349640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0c3310 con 0x7fba5c15d930 2023-10-11T06:12:37.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba62349640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0c35f0 con 0x7fba5c1591b0 2023-10-11T06:12:37.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba5bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0cdf90 0x7fba5c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba5bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0cdf90 0x7fba5c14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54632/0 (socket says 172.21.15.84:54632) 2023-10-11T06:12:37.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.117+0000 7fba5bfff640 1 -- 172.21.15.84:0/3221384277 learned_addr learned my addr 172.21.15.84:0/3221384277 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:37.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba608e6640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c15d930 0x7fba5c0c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba5bfff640 1 -- 172.21.15.84:0/3221384277 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c1591b0 msgr2=0x7fba5c0c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba5bfff640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c1591b0 0x7fba5c0c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba5bfff640 1 -- 172.21.15.84:0/3221384277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c15d930 msgr2=0x7fba5c0c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba5bfff640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c15d930 0x7fba5c0c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba5bfff640 1 -- 172.21.15.84:0/3221384277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c009b30 con 0x7fba5c0cdf90 2023-10-11T06:12:37.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba5bfff640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0cdf90 0x7fba5c14ea90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009e50 tx=0x7fba4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:37.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba58ff9640 1 -- 172.21.15.84:0/3221384277 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c03e450 con 0x7fba5c0cdf90 2023-10-11T06:12:37.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.118+0000 7fba58ff9640 1 -- 172.21.15.84:0/3221384277 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c03e5f0 con 0x7fba5c0cdf90 2023-10-11T06:12:37.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.119+0000 7fba58ff9640 1 -- 172.21.15.84:0/3221384277 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c03e9a0 con 0x7fba5c0cdf90 2023-10-11T06:12:37.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.119+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c0c38d0 con 0x7fba5c0cdf90 2023-10-11T06:12:37.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.119+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c0c3e90 con 0x7fba5c0cdf90 2023-10-11T06:12:37.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.120+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5c0c3730 con 0x7fba5c0cdf90 2023-10-11T06:12:37.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.121+0000 7fba58ff9640 1 -- 172.21.15.84:0/3221384277 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fba4c03eb00 con 0x7fba5c0cdf90 2023-10-11T06:12:37.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.122+0000 7fba58ff9640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba3c06a830 0x7fba3c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.122+0000 7fba58ff9640 1 -- 172.21.15.84:0/3221384277 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fba4c0abb90 con 0x7fba5c0cdf90 2023-10-11T06:12:37.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.123+0000 7fba5b7fe640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba3c06a830 0x7fba3c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.123+0000 7fba5b7fe640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba3c06a830 0x7fba3c06cce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba5c071830 tx=0x7fba44007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:37.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.127+0000 7fba58ff9640 1 -- 172.21.15.84:0/3221384277 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fba5c0c3730 con 0x7fba5c0cdf90 2023-10-11T06:12:37.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.316+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba5c0c3730 con 0x7fba5c0cdf90 2023-10-11T06:12:37.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.317+0000 7fba58ff9640 1 -- 172.21.15.84:0/3221384277 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba5c0c3730 con 0x7fba5c0cdf90 2023-10-11T06:12:37.319 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:37.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.319+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba3c06a830 msgr2=0x7fba3c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.319+0000 7fba62349640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba3c06a830 0x7fba3c06cce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba5c071830 tx=0x7fba44007990 comp rx=0 tx=0).stop 2023-10-11T06:12:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.319+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0cdf90 msgr2=0x7fba5c14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.319+0000 7fba62349640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0cdf90 0x7fba5c14ea90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009e50 tx=0x7fba4c004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 shutdown_connections 2023-10-11T06:12:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba3c06a830 0x7fba3c06cce0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c15d930 0x7fba5c0c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c1591b0 0x7fba5c0c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 --2- 172.21.15.84:0/3221384277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0cdf90 0x7fba5c14ea90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 >> 172.21.15.84:0/3221384277 conn(0x7fba5c0b16c0 msgr2=0x7fba5c0b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:37.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 shutdown_connections 2023-10-11T06:12:37.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.320+0000 7fba62349640 1 -- 172.21.15.84:0/3221384277 wait complete. 2023-10-11T06:12:37.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T06:12:37.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.554+0000 7fdffaa77640 1 Processor -- start 2023-10-11T06:12:37.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.554+0000 7fdffaa77640 1 -- start start 2023-10-11T06:12:37.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdffaa77640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdff40d0c20 0x7fdff40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdffaa77640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff40d2a40 con 0x7fdff40d25e0 2023-10-11T06:12:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdffaa77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff406cd10 con 0x7fdff415c290 2023-10-11T06:12:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdffaa77640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff40afb60 con 0x7fdff40d0c20 2023-10-11T06:12:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdff9014640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdff40d25e0 0x7fdff40b0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34652/0 (socket says 172.21.15.84:34652) 2023-10-11T06:12:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdff9014640 1 -- 172.21.15.84:0/3197163310 learned_addr learned my addr 172.21.15.84:0/3197163310 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:37.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3197163310 >> v2:172.21.15.99:3300/0 conn(0x7fdff40d0c20 0x7fdff40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.556+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278866600 0 0) 0x7fdff40d2a40 con 0x7fdff40d25e0 2023-10-11T06:12:37.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe4003520 con 0x7fdff40d25e0 2023-10-11T06:12:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3936288127 0 0) 0x7fdfe4003520 con 0x7fdff40d25e0 2023-10-11T06:12:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 >> v2:172.21.15.99:3300/0 conn(0x7fdff40d0c20 msgr2=0x7fdff40af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 --2- 172.21.15.84:0/3197163310 >> v2:172.21.15.99:3300/0 conn(0x7fdff40d0c20 0x7fdff40af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 >> v1:172.21.15.99:6789/0 conn(0x7fdff415c290 legacy=0x7fdff40d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff40afe40 con 0x7fdff40d25e0 2023-10-11T06:12:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fdfec0019f0 con 0x7fdff40d25e0 2023-10-11T06:12:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 >> v1:172.21.15.84:6789/0 conn(0x7fdff40d25e0 legacy=0x7fdff40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdfe40039c0 0x7fdfe4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe40047c0 0x7fdfe4004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40051c0 0x7fdfe400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40afb60 con 0x7fdfe40047c0 2023-10-11T06:12:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe400c0d0 con 0x7fdfe40051c0 2023-10-11T06:12:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.557+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe400c290 con 0x7fdfe40039c0 2023-10-11T06:12:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff8813640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdfe40039c0 0x7fdfe4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff9014640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40051c0 0x7fdfe400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe40047c0 0x7fdfe4004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff3fff640 1 -- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdfe40039c0 msgr2=0x7fdfe4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:37.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdfe40039c0 0x7fdfe4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff3fff640 1 -- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40051c0 msgr2=0x7fdfe400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40051c0 0x7fdfe400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff3fff640 1 -- 172.21.15.84:0/3197163310 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4003520 con 0x7fdfe40047c0 2023-10-11T06:12:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.558+0000 7fdff9014640 1 -- 172.21.15.84:0/3197163310 reap_dead start 2023-10-11T06:12:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.559+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe40047c0 0x7fdfe4004c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009ac0 tx=0x7fdfe40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe30ff0d8cb1baec server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe002c6b0 con 0x7fdfe40047c0 2023-10-11T06:12:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe002c810 con 0x7fdfe40047c0 2023-10-11T06:12:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdff37fe640 1 -- 172.21.15.84:0/3197163310 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe002cba0 con 0x7fdfe40047c0 2023-10-11T06:12:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdffaa77640 1 -- 172.21.15.84:0/3197163310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe40047c0 msgr2=0x7fdfe4004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe40047c0 0x7fdfe4004c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009ac0 tx=0x7fdfe40057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdffaa77640 1 -- 172.21.15.84:0/3197163310 shutdown_connections 2023-10-11T06:12:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3197163310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe40047c0 0x7fdfe4004c30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdffaa77640 1 -- 172.21.15.84:0/3197163310 >> 172.21.15.84:0/3197163310 conn(0x7fdff4071cc0 msgr2=0x7fdff409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdffaa77640 1 -- 172.21.15.84:0/3197163310 shutdown_connections 2023-10-11T06:12:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.560+0000 7fdffaa77640 1 -- 172.21.15.84:0/3197163310 wait complete. 2023-10-11T06:12:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.562+0000 7fdffaa77640 1 Processor -- start 2023-10-11T06:12:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.562+0000 7fdffaa77640 1 -- start start 2023-10-11T06:12:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.563+0000 7fdffaa77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40047c0 0x7fdff40c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.563+0000 7fdffaa77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff41579e0 0x7fdff40c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.563+0000 7fdffaa77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff40c6c40 0x7fdff40c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.563+0000 7fdffaa77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff409c5e0 con 0x7fdff41579e0 2023-10-11T06:12:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.563+0000 7fdffaa77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff409cba0 con 0x7fdfe40047c0 2023-10-11T06:12:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.563+0000 7fdffaa77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41f1430 con 0x7fdff40c6c40 2023-10-11T06:12:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.563+0000 7fdff3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff41579e0 0x7fdff40c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff41579e0 0x7fdff40c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54648/0 (socket says 172.21.15.84:54648) 2023-10-11T06:12:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 -- 172.21.15.84:0/3593710045 learned_addr learned my addr 172.21.15.84:0/3593710045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff8813640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40047c0 0x7fdff40c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 -- 172.21.15.84:0/3593710045 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff40c6c40 msgr2=0x7fdff40c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff40c6c40 0x7fdff40c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 -- 172.21.15.84:0/3593710045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40047c0 msgr2=0x7fdff40c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40047c0 0x7fdff40c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 -- 172.21.15.84:0/3593710045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff409ffa0 con 0x7fdff41579e0 2023-10-11T06:12:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff3fff640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff41579e0 0x7fdff40c6700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0037040 tx=0x7fdfe0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.564+0000 7fdff17fa640 1 -- 172.21.15.84:0/3593710045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0003810 con 0x7fdff41579e0 2023-10-11T06:12:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.565+0000 7fdff17fa640 1 -- 172.21.15.84:0/3593710045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0003970 con 0x7fdff41579e0 2023-10-11T06:12:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.565+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff40a0280 con 0x7fdff41579e0 2023-10-11T06:12:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.565+0000 7fdff17fa640 1 -- 172.21.15.84:0/3593710045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0003d00 con 0x7fdff41579e0 2023-10-11T06:12:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.565+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff409fc70 con 0x7fdff41579e0 2023-10-11T06:12:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.566+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff4073ed0 con 0x7fdff41579e0 2023-10-11T06:12:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.567+0000 7fdff17fa640 1 -- 172.21.15.84:0/3593710045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdfe0040450 con 0x7fdff41579e0 2023-10-11T06:12:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.568+0000 7fdff17fa640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd806a860 0x7fdfd806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.568+0000 7fdff17fa640 1 -- 172.21.15.84:0/3593710045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdfe00aaf80 con 0x7fdff41579e0 2023-10-11T06:12:37.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.568+0000 7fdff8813640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd806a860 0x7fdfd806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:37.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.569+0000 7fdff8813640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd806a860 0x7fdfd806cd10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8005490 tx=0x7fdfe8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:37.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.572+0000 7fdff17fa640 1 -- 172.21.15.84:0/3593710045 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdfe00764b0 con 0x7fdff41579e0 2023-10-11T06:12:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.762+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdff40b03d0 con 0x7fdff41579e0 2023-10-11T06:12:37.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.763+0000 7fdff17fa640 1 -- 172.21.15.84:0/3593710045 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdfe0003e60 con 0x7fdff41579e0 2023-10-11T06:12:37.765 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:37.765 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-11T06:12:37.765 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:37.765 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-11T06:12:37.766 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-11T06:12:37.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:37.766 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:12:37.766 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 2, 2023-10-11T06:12:37.767 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "rbd" 2023-10-11T06:12:37.767 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:37.767 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-11T06:12:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd806a860 msgr2=0x7fdfd806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd806a860 0x7fdfd806cd10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8005490 tx=0x7fdfe8009040 comp rx=0 tx=0).stop 2023-10-11T06:12:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff41579e0 msgr2=0x7fdff40c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff41579e0 0x7fdff40c6700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0037040 tx=0x7fdfe0004730 comp rx=0 tx=0).stop 2023-10-11T06:12:37.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 shutdown_connections 2023-10-11T06:12:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdfd806a860 0x7fdfd806cd10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdff40c6c40 0x7fdff40c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdff41579e0 0x7fdff40c6700 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 --2- 172.21.15.84:0/3593710045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe40047c0 0x7fdff40c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 >> 172.21.15.84:0/3593710045 conn(0x7fdff4071cc0 msgr2=0x7fdff40b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:37.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 shutdown_connections 2023-10-11T06:12:37.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:37.766+0000 7fdffaa77640 1 -- 172.21.15.84:0/3593710045 wait complete. 2023-10-11T06:12:37.786 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T06:12:37.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T06:12:38.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.000+0000 7fabe4e4b640 1 Processor -- start 2023-10-11T06:12:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.001+0000 7fabe4e4b640 1 -- start start 2023-10-11T06:12:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.002+0000 7fabe4e4b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fabe015a0b0 0x7fabe00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.002+0000 7fabe4e4b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe00c8d00 0x7fabe00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.002+0000 7fabe4e4b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe00cefc0 con 0x7fabe00ceb60 2023-10-11T06:12:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.002+0000 7fabe4e4b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe006e0f0 con 0x7fabe015a0b0 2023-10-11T06:12:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabe4e4b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00cbae0 con 0x7fabe00c8d00 2023-10-11T06:12:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabde59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fabe015a0b0 0x7fabe00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabde59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fabe015a0b0 0x7fabe00cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54656/0 (socket says 172.21.15.84:54656) 2023-10-11T06:12:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabded9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fabe00ceb60 0x7fabe006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34666/0 (socket says 172.21.15.84:34666) 2023-10-11T06:12:38.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabde59c640 1 -- 172.21.15.84:0/2041854790 learned_addr learned my addr 172.21.15.84:0/2041854790 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabd7fff640 1 --2- 172.21.15.84:0/2041854790 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe00c8d00 0x7fabe00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabde59c640 1 -- 172.21.15.84:0/2041854790 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe00c8d00 msgr2=0x7fabe00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabddd9b640 1 -- 172.21.15.84:0/2041854790 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194309277 0 0) 0x7fabe00cefc0 con 0x7fabe00ceb60 2023-10-11T06:12:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabde59c640 1 --2- 172.21.15.84:0/2041854790 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe00c8d00 0x7fabe00ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabde59c640 1 -- 172.21.15.84:0/2041854790 >> v1:172.21.15.84:6789/0 conn(0x7fabe00ceb60 legacy=0x7fabe006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabde59c640 1 -- 172.21.15.84:0/2041854790 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe0071380 con 0x7fabe015a0b0 2023-10-11T06:12:38.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.003+0000 7fabd7fff640 1 --2- 172.21.15.84:0/2041854790 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe00c8d00 0x7fabe00ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.004+0000 7fabde59c640 1 --2- 172.21.15.84:0/2041854790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe015a0b0 0x7fabe00cb5a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fabd00098a0 tx=0x7fabd002b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfb100e63cbd929 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.005+0000 7fabddd9b640 1 -- 172.21.15.84:0/2041854790 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0032640 con 0x7fabe015a0b0 2023-10-11T06:12:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.005+0000 7fabddd9b640 1 -- 172.21.15.84:0/2041854790 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd002e070 con 0x7fabe015a0b0 2023-10-11T06:12:38.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.005+0000 7fabddd9b640 1 -- 172.21.15.84:0/2041854790 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00329c0 con 0x7fabe015a0b0 2023-10-11T06:12:38.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.005+0000 7fabe4e4b640 1 -- 172.21.15.84:0/2041854790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe015a0b0 msgr2=0x7fabe00cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.005+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/2041854790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe015a0b0 0x7fabe00cb5a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fabd00098a0 tx=0x7fabd002b850 comp rx=0 tx=0).stop 2023-10-11T06:12:38.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.005+0000 7fabe4e4b640 1 -- 172.21.15.84:0/2041854790 shutdown_connections 2023-10-11T06:12:38.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.006+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/2041854790 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe00c8d00 0x7fabe00ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.006+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/2041854790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe015a0b0 0x7fabe00cb5a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.006+0000 7fabe4e4b640 1 -- 172.21.15.84:0/2041854790 >> 172.21.15.84:0/2041854790 conn(0x7fabe0069c70 msgr2=0x7fabe006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.006+0000 7fabe4e4b640 1 -- 172.21.15.84:0/2041854790 shutdown_connections 2023-10-11T06:12:38.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.006+0000 7fabe4e4b640 1 -- 172.21.15.84:0/2041854790 wait complete. 2023-10-11T06:12:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.007+0000 7fabe4e4b640 1 Processor -- start 2023-10-11T06:12:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.008+0000 7fabe4e4b640 1 -- start start 2023-10-11T06:12:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.008+0000 7fabe4e4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe00c8d00 0x7fabe015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.008+0000 7fabe4e4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe00ceb60 0x7fabe015b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabe4e4b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe015a0b0 0x7fabe00b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabe4e4b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe015bb80 con 0x7fabe00c8d00 2023-10-11T06:12:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabe4e4b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01f19f0 con 0x7fabe00ceb60 2023-10-11T06:12:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabe4e4b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01f1cd0 con 0x7fabe015a0b0 2023-10-11T06:12:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe00c8d00 0x7fabe015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe00c8d00 0x7fabe015c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54672/0 (socket says 172.21.15.84:54672) 2023-10-11T06:12:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 -- 172.21.15.84:0/299234840 learned_addr learned my addr 172.21.15.84:0/299234840 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 -- 172.21.15.84:0/299234840 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe015a0b0 msgr2=0x7fabe00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe015a0b0 0x7fabe00b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 -- 172.21.15.84:0/299234840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe00ceb60 msgr2=0x7fabe015b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe00ceb60 0x7fabe015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 -- 172.21.15.84:0/299234840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00094d0 con 0x7fabe00c8d00 2023-10-11T06:12:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.009+0000 7fabde59c640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe00c8d00 0x7fabe015c410 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fabd0000c00 tx=0x7fabd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:38.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.010+0000 7fabd6ffd640 1 -- 172.21.15.84:0/299234840 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd003b800 con 0x7fabe00c8d00 2023-10-11T06:12:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.010+0000 7fabd6ffd640 1 -- 172.21.15.84:0/299234840 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd003b960 con 0x7fabe00c8d00 2023-10-11T06:12:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.010+0000 7fabd6ffd640 1 -- 172.21.15.84:0/299234840 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd003bd10 con 0x7fabe00c8d00 2023-10-11T06:12:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.010+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe01f1fb0 con 0x7fabe00c8d00 2023-10-11T06:12:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.010+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe01f2600 con 0x7fabe00c8d00 2023-10-11T06:12:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.011+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe0073ed0 con 0x7fabe00c8d00 2023-10-11T06:12:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.012+0000 7fabd6ffd640 1 -- 172.21.15.84:0/299234840 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fabd004e450 con 0x7fabe00c8d00 2023-10-11T06:12:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.013+0000 7fabd6ffd640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabc006a830 0x7fabc006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.013+0000 7fabd6ffd640 1 -- 172.21.15.84:0/299234840 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fabd00aacc0 con 0x7fabe00c8d00 2023-10-11T06:12:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.013+0000 7fabd7fff640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabc006a830 0x7fabc006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.014+0000 7fabd7fff640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabc006a830 0x7fabc006cce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.018+0000 7fabd6ffd640 1 -- 172.21.15.84:0/299234840 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fabd00763b0 con 0x7fabe00c8d00 2023-10-11T06:12:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.206+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fabe01f1e10 con 0x7fabe00c8d00 2023-10-11T06:12:38.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.508+0000 7fabd6ffd640 1 -- 172.21.15.84:0/299234840 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fabe01f1e10 con 0x7fabe00c8d00 2023-10-11T06:12:38.512 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-11T06:12:38.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabc006a830 msgr2=0x7fabc006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabc006a830 0x7fabc006cce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8008040 comp rx=0 tx=0).stop 2023-10-11T06:12:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe00c8d00 msgr2=0x7fabe015c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe00c8d00 0x7fabe015c410 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fabd0000c00 tx=0x7fabd00046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 shutdown_connections 2023-10-11T06:12:38.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fabc006a830 0x7fabc006cce0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fabe015a0b0 0x7fabe00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe00ceb60 0x7fabe015b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 --2- 172.21.15.84:0/299234840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabe00c8d00 0x7fabe015c410 secure :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fabd0000c00 tx=0x7fabd00046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.515+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 >> 172.21.15.84:0/299234840 conn(0x7fabe0069c70 msgr2=0x7fabe014ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:38.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.517+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 shutdown_connections 2023-10-11T06:12:38.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.518+0000 7fabe4e4b640 1 -- 172.21.15.84:0/299234840 wait complete. 2023-10-11T06:12:38.542 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T06:12:38.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.755+0000 7fa53c349640 1 Processor -- start 2023-10-11T06:12:38.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.755+0000 7fa53c349640 1 -- start start 2023-10-11T06:12:38.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53c349640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa53415f7e0 0x7fa5340cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53c349640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5340cd6c0 0x7fa5340d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53c349640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5340d3000 0x7fa53415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53c349640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5340d3460 con 0x7fa53415f7e0 2023-10-11T06:12:38.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53c349640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5340cf220 con 0x7fa5340cd6c0 2023-10-11T06:12:38.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53c349640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340cf500 con 0x7fa5340d3000 2023-10-11T06:12:38.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53a0e5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa53415f7e0 0x7fa5340cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53a0e5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa53415f7e0 0x7fa5340cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54674/0 (socket says 172.21.15.84:54674) 2023-10-11T06:12:38.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3501368049 learned_addr learned my addr 172.21.15.84:0/3501368049 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:38.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53a8e6640 1 --2- 172.21.15.84:0/3501368049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5340d3000 0x7fa53415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa5398e4640 1 --2- 172.21.15.84:0/3501368049 >> v2:172.21.15.99:3300/0 conn(0x7fa5340cd6c0 0x7fa5340d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.757+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3501368049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5340d3000 msgr2=0x7fa53415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:38.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.758+0000 7fa53a0e5640 1 --2- 172.21.15.84:0/3501368049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5340d3000 0x7fa53415fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.758+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3501368049 >> v2:172.21.15.99:3300/0 conn(0x7fa5340cd6c0 msgr2=0x7fa5340d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:38.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.758+0000 7fa53a0e5640 1 --2- 172.21.15.84:0/3501368049 >> v2:172.21.15.99:3300/0 conn(0x7fa5340cd6c0 0x7fa5340d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.758+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3501368049 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5340cf7e0 con 0x7fa53415f7e0 2023-10-11T06:12:38.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.759+0000 7fa53a0e5640 1 --2- 172.21.15.84:0/3501368049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53415f7e0 0x7fa5340cd180 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa53408adb0 tx=0x7fa53002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a6c18ea143add4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:38.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.759+0000 7fa5390e3640 1 -- 172.21.15.84:0/3501368049 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530031860 con 0x7fa53415f7e0 2023-10-11T06:12:38.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.759+0000 7fa5390e3640 1 -- 172.21.15.84:0/3501368049 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5300319c0 con 0x7fa53415f7e0 2023-10-11T06:12:38.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.759+0000 7fa5390e3640 1 -- 172.21.15.84:0/3501368049 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530031d50 con 0x7fa53415f7e0 2023-10-11T06:12:38.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 -- 172.21.15.84:0/3501368049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53415f7e0 msgr2=0x7fa5340cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 --2- 172.21.15.84:0/3501368049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53415f7e0 0x7fa5340cd180 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa53408adb0 tx=0x7fa53002b900 comp rx=0 tx=0).stop 2023-10-11T06:12:38.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 -- 172.21.15.84:0/3501368049 shutdown_connections 2023-10-11T06:12:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 --2- 172.21.15.84:0/3501368049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5340d3000 0x7fa53415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 --2- 172.21.15.84:0/3501368049 >> v2:172.21.15.99:3300/0 conn(0x7fa5340cd6c0 0x7fa5340d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 --2- 172.21.15.84:0/3501368049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa53415f7e0 0x7fa5340cd180 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 -- 172.21.15.84:0/3501368049 >> 172.21.15.84:0/3501368049 conn(0x7fa5340713a0 msgr2=0x7fa5340717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:38.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 -- 172.21.15.84:0/3501368049 shutdown_connections 2023-10-11T06:12:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.760+0000 7fa53c349640 1 -- 172.21.15.84:0/3501368049 wait complete. 2023-10-11T06:12:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.761+0000 7fa53c349640 1 Processor -- start 2023-10-11T06:12:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.762+0000 7fa53c349640 1 -- start start 2023-10-11T06:12:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53c349640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5340cd6c0 0x7fa5340c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53c349640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d3000 0x7fa5340c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53c349640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa53415f7e0 0x7fa534154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53c349640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53415e860 con 0x7fa5340cd6c0 2023-10-11T06:12:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53c349640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340c76e0 con 0x7fa5340d3000 2023-10-11T06:12:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53c349640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5341f1c50 con 0x7fa53415f7e0 2023-10-11T06:12:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5340cd6c0 0x7fa5340c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5340cd6c0 0x7fa5340c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54686/0 (socket says 172.21.15.84:54686) 2023-10-11T06:12:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3516414696 learned_addr learned my addr 172.21.15.84:0/3516414696 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3516414696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa53415f7e0 msgr2=0x7fa534154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa53415f7e0 0x7fa534154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3516414696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d3000 msgr2=0x7fa5340c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:38.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d3000 0x7fa5340c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:38.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.763+0000 7fa53a0e5640 1 -- 172.21.15.84:0/3516414696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300094d0 con 0x7fa5340cd6c0 2023-10-11T06:12:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.764+0000 7fa53a0e5640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5340cd6c0 0x7fa5340c6660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa53002b940 tx=0x7fa5300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.764+0000 7fa52affd640 1 -- 172.21.15.84:0/3516414696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530031850 con 0x7fa5340cd6c0 2023-10-11T06:12:38.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.764+0000 7fa52affd640 1 -- 172.21.15.84:0/3516414696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5300443f0 con 0x7fa5340cd6c0 2023-10-11T06:12:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.764+0000 7fa52affd640 1 -- 172.21.15.84:0/3516414696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300447a0 con 0x7fa5340cd6c0 2023-10-11T06:12:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.764+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5341f1f30 con 0x7fa5340cd6c0 2023-10-11T06:12:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.764+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa534154590 con 0x7fa5340cd6c0 2023-10-11T06:12:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.765+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5341f2450 con 0x7fa5340cd6c0 2023-10-11T06:12:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.767+0000 7fa52affd640 1 -- 172.21.15.84:0/3516414696 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa53004d450 con 0x7fa5340cd6c0 2023-10-11T06:12:38.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.767+0000 7fa52affd640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa51806a830 0x7fa51806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.768+0000 7fa52affd640 1 -- 172.21.15.84:0/3516414696 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fa5300ab6a0 con 0x7fa5340cd6c0 2023-10-11T06:12:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.768+0000 7fa5398e4640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa51806a830 0x7fa51806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.769+0000 7fa5398e4640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa51806a830 0x7fa51806cce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5340d04a0 tx=0x7fa524008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.772+0000 7fa52affd640 1 -- 172.21.15.84:0/3516414696 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa530076c70 con 0x7fa5340cd6c0 2023-10-11T06:12:38.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:38.962+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa5341f2450 con 0x7fa5340cd6c0 2023-10-11T06:12:39.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.524+0000 7fa52affd640 1 -- 172.21.15.84:0/3516414696 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa5300350d0 con 0x7fa5340cd6c0 2023-10-11T06:12:39.526 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-11T06:12:39.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.533+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa51806a830 msgr2=0x7fa51806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:39.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.533+0000 7fa53c349640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa51806a830 0x7fa51806cce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5340d04a0 tx=0x7fa524008040 comp rx=0 tx=0).stop 2023-10-11T06:12:39.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.533+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5340cd6c0 msgr2=0x7fa5340c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:39.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.533+0000 7fa53c349640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5340cd6c0 0x7fa5340c6660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa53002b940 tx=0x7fa5300046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.534+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 shutdown_connections 2023-10-11T06:12:39.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.534+0000 7fa53c349640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa51806a830 0x7fa51806cce0 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5340d04a0 tx=0x7fa524008040 comp rx=0 tx=0).stop 2023-10-11T06:12:39.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.534+0000 7fa53c349640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa53415f7e0 0x7fa534154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.534+0000 7fa53c349640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d3000 0x7fa5340c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.534+0000 7fa53c349640 1 --2- 172.21.15.84:0/3516414696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5340cd6c0 0x7fa5340c6660 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.534+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 >> 172.21.15.84:0/3516414696 conn(0x7fa5340713a0 msgr2=0x7fa534071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:39.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.538+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 shutdown_connections 2023-10-11T06:12:39.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.540+0000 7fa53c349640 1 -- 172.21.15.84:0/3516414696 wait complete. 2023-10-11T06:12:39.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T06:12:39.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.755+0000 7f0cae45f640 1 Processor -- start 2023-10-11T06:12:39.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.755+0000 7f0cae45f640 1 -- start start 2023-10-11T06:12:39.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.756+0000 7f0cae45f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 0x7f0ca80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0cae45f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca8071380 con 0x7f0ca80c8d00 2023-10-11T06:12:39.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0cae45f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca806de10 con 0x7f0ca80ced30 2023-10-11T06:12:39.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0cae45f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca806e0f0 con 0x7f0ca815a0b0 2023-10-11T06:12:39.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0c9ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0ca80c8d00 0x7f0ca80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48224/0 (socket says 172.21.15.84:48224) 2023-10-11T06:12:39.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0c9ffff640 1 -- 172.21.15.84:0/1174905355 learned_addr learned my addr 172.21.15.84:0/1174905355 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:39.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 0x7f0ca80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:39.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770184573 0 0) 0x7f0ca8071380 con 0x7f0ca80c8d00 2023-10-11T06:12:39.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.757+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c94003520 con 0x7f0ca80c8d00 2023-10-11T06:12:39.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1625583785 0 0) 0x7f0c94003520 con 0x7f0ca80c8d00 2023-10-11T06:12:39.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 msgr2=0x7f0ca80cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:39.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 0x7f0ca80cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 >> v1:172.21.15.99:6789/0 conn(0x7f0ca80ced30 legacy=0x7f0ca806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:39.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca8070dc0 con 0x7f0ca80c8d00 2023-10-11T06:12:39.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f0c90001670 con 0x7f0ca80c8d00 2023-10-11T06:12:39.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 >> v1:172.21.15.84:6789/0 conn(0x7f0ca80c8d00 legacy=0x7f0ca80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:39.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c940039a0 0x7f0c94003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 0x7f0c940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c94004e20 0x7f0c94005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca806e0f0 con 0x7f0c940039a0 2023-10-11T06:12:39.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9400bc20 con 0x7f0c94004e20 2023-10-11T06:12:39.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9400bde0 con 0x7f0ca815a0b0 2023-10-11T06:12:39.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c940039a0 0x7f0c94003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:39.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0c9ffff640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 0x7f0c940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:39.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.758+0000 7f0cac9fc640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c94004e20 0x7f0c94005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:39.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.759+0000 7f0ca7fff640 1 -- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 msgr2=0x7f0c940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:39.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.759+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca815a0b0 0x7f0c940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.759+0000 7f0ca7fff640 1 -- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c94004e20 msgr2=0x7f0c94005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:39.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.759+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c94004e20 0x7f0c94005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.759+0000 7f0ca7fff640 1 -- 172.21.15.84:0/1174905355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c94003520 con 0x7f0c940039a0 2023-10-11T06:12:39.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.759+0000 7f0cac9fc640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c94004e20 0x7f0c94005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:39.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.760+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c940039a0 0x7f0c94003db0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ba30 tx=0x7f0c940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a698a1c642f40f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:39.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.760+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98032640 con 0x7f0c940039a0 2023-10-11T06:12:39.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.760+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c980327a0 con 0x7f0c940039a0 2023-10-11T06:12:39.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.760+0000 7f0ca77fe640 1 -- 172.21.15.84:0/1174905355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98032b30 con 0x7f0c940039a0 2023-10-11T06:12:39.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.760+0000 7f0cae45f640 1 -- 172.21.15.84:0/1174905355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c940039a0 msgr2=0x7f0c94003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:39.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.760+0000 7f0cae45f640 1 --2- 172.21.15.84:0/1174905355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c940039a0 0x7f0c94003db0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ba30 tx=0x7f0c940057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.760+0000 7f0cac9fc640 1 -- 172.21.15.84:0/1174905355 reap_dead start 2023-10-11T06:12:39.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.761+0000 7f0cae45f640 1 -- 172.21.15.84:0/1174905355 shutdown_connections 2023-10-11T06:12:39.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.761+0000 7f0cae45f640 1 -- 172.21.15.84:0/1174905355 >> 172.21.15.84:0/1174905355 conn(0x7f0ca8069c70 msgr2=0x7f0ca806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:39.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.761+0000 7f0cae45f640 1 -- 172.21.15.84:0/1174905355 shutdown_connections 2023-10-11T06:12:39.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.761+0000 7f0cae45f640 1 -- 172.21.15.84:0/1174905355 wait complete. 2023-10-11T06:12:39.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.762+0000 7f0cae45f640 1 Processor -- start 2023-10-11T06:12:39.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.763+0000 7f0cae45f640 1 -- start start 2023-10-11T06:12:39.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.763+0000 7f0cae45f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c940039a0 0x7f0ca80cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.763+0000 7f0cae45f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca80c5780 0x7f0ca80b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.763+0000 7f0cae45f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca814e480 0x7f0ca80b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.763+0000 7f0cae45f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80ce350 con 0x7f0ca80c5780 2023-10-11T06:12:39.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.763+0000 7f0cae45f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80cd860 con 0x7f0c940039a0 2023-10-11T06:12:39.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.763+0000 7f0cae45f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80c5c80 con 0x7f0ca814e480 2023-10-11T06:12:39.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca80c5780 0x7f0ca80b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:39.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca80c5780 0x7f0ca80b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49512/0 (socket says 172.21.15.84:49512) 2023-10-11T06:12:39.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 -- 172.21.15.84:0/2103170457 learned_addr learned my addr 172.21.15.84:0/2103170457 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:39.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c940039a0 0x7f0ca80cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:39.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 -- 172.21.15.84:0/2103170457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca814e480 msgr2=0x7f0ca80b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:39.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca814e480 0x7f0ca80b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 -- 172.21.15.84:0/2103170457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c940039a0 msgr2=0x7f0ca80cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:39.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c940039a0 0x7f0ca80cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:39.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.764+0000 7f0c9ffff640 1 -- 172.21.15.84:0/2103170457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80c5f60 con 0x7f0ca80c5780 2023-10-11T06:12:39.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.765+0000 7f0c9ffff640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca80c5780 0x7f0ca80b3790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e780 tx=0x7f0c9000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:39.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.765+0000 7f0ca57fa640 1 -- 172.21.15.84:0/2103170457 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000ee30 con 0x7f0ca80c5780 2023-10-11T06:12:39.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.765+0000 7f0ca57fa640 1 -- 172.21.15.84:0/2103170457 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9000c870 con 0x7f0ca80c5780 2023-10-11T06:12:39.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.765+0000 7f0ca57fa640 1 -- 172.21.15.84:0/2103170457 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000cc20 con 0x7f0ca80c5780 2023-10-11T06:12:39.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.765+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca80c6240 con 0x7f0ca80c5780 2023-10-11T06:12:39.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.765+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca814f8c0 con 0x7f0ca80c5780 2023-10-11T06:12:39.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.766+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca8073ed0 con 0x7f0ca80c5780 2023-10-11T06:12:39.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.768+0000 7f0ca57fa640 1 -- 172.21.15.84:0/2103170457 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0c9001d460 con 0x7f0ca80c5780 2023-10-11T06:12:39.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.769+0000 7f0ca57fa640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c8806a830 0x7f0c8806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:39.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.769+0000 7f0ca57fa640 1 -- 172.21.15.84:0/2103170457 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0c90089890 con 0x7f0ca80c5780 2023-10-11T06:12:39.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.770+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c8806a830 0x7f0c8806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:39.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.770+0000 7f0ca7fff640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c8806a830 0x7f0c8806cce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ba00 tx=0x7f0c9803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:39.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:39.773+0000 7f0ca57fa640 1 -- 172.21.15.84:0/2103170457 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0c90054ce0 con 0x7f0ca80c5780 2023-10-11T06:12:40.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.000+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0ca8070760 con 0x7f0ca80c5780 2023-10-11T06:12:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 5 and data pool 6 2023-10-11T06:12:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.013+0000 7f0ca57fa640 1 -- 172.21.15.84:0/2103170457 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0c90058300 con 0x7f0ca80c5780 2023-10-11T06:12:40.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.018+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c8806a830 msgr2=0x7f0c8806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.018+0000 7f0cae45f640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c8806a830 0x7f0c8806cce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ba00 tx=0x7f0c9803d040 comp rx=0 tx=0).stop 2023-10-11T06:12:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.018+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca80c5780 msgr2=0x7f0ca80b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.018+0000 7f0cae45f640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca80c5780 0x7f0ca80b3790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e780 tx=0x7f0c9000ec40 comp rx=0 tx=0).stop 2023-10-11T06:12:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.020+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 shutdown_connections 2023-10-11T06:12:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.020+0000 7f0cae45f640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c8806a830 0x7f0c8806cce0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.020+0000 7f0cae45f640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ca814e480 0x7f0ca80b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.020+0000 7f0cae45f640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ca80c5780 0x7f0ca80b3790 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.020+0000 7f0cae45f640 1 --2- 172.21.15.84:0/2103170457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c940039a0 0x7f0ca80cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.020+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 >> 172.21.15.84:0/2103170457 conn(0x7f0ca8069c70 msgr2=0x7f0ca806c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:40.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.027+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 shutdown_connections 2023-10-11T06:12:40.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.028+0000 7f0cae45f640 1 -- 172.21.15.84:0/2103170457 wait complete. 2023-10-11T06:12:40.054 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:40.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.235+0000 7f4ce210a640 1 Processor -- start 2023-10-11T06:12:40.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.235+0000 7f4ce210a640 1 -- start start 2023-10-11T06:12:40.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.236+0000 7f4ce210a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.236+0000 7f4ce210a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4cdc0ca300 0x7f4cdc0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.236+0000 7f4ce210a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cdc071a90 con 0x7f4cdc0c9190 2023-10-11T06:12:40.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.236+0000 7f4ce210a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc072310 con 0x7f4cdc0ca300 2023-10-11T06:12:40.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.237+0000 7f4ce210a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc070280 con 0x7f4cdc15d4f0 2023-10-11T06:12:40.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.237+0000 7f4cdaffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4cdc0ca300 0x7f4cdc0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.237+0000 7f4cdb7fe640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.237+0000 7f4cdb7fe640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:48878/0 (socket says 172.21.15.84:48878) 2023-10-11T06:12:40.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.237+0000 7f4cdaffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4cdc0ca300 0x7f4cdc0c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:33290/0 (socket says 172.21.15.84:33290) 2023-10-11T06:12:40.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.237+0000 7f4cdb7fe640 1 -- 172.21.15.84:0/3463021223 learned_addr learned my addr 172.21.15.84:0/3463021223 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.238+0000 7f4cdb7fe640 1 -- 172.21.15.84:0/3463021223 >> v2:172.21.15.99:3300/0 conn(0x7f4cdc0ca300 msgr2=0x7f4cdc0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.238+0000 7f4cdb7fe640 1 --2- 172.21.15.84:0/3463021223 >> v2:172.21.15.99:3300/0 conn(0x7f4cdc0ca300 0x7f4cdc0c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.238+0000 7f4cdb7fe640 1 -- 172.21.15.84:0/3463021223 >> v1:172.21.15.99:6789/0 conn(0x7f4cdc0c9190 legacy=0x7f4cdc15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.238+0000 7f4cdb7fe640 1 -- 172.21.15.84:0/3463021223 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc071cc0 con 0x7f4cdc15d4f0 2023-10-11T06:12:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.238+0000 7f4cdaffd640 1 --2- 172.21.15.84:0/3463021223 >> v2:172.21.15.99:3300/0 conn(0x7f4cdc0ca300 0x7f4cdc0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.240+0000 7f4cdb7fe640 1 --2- 172.21.15.84:0/3463021223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0cb540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009850 tx=0x7f4ccc02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822fb9298d8e1394 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.240+0000 7f4cda7fc640 1 -- 172.21.15.84:0/3463021223 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc004230 con 0x7f4cdc15d4f0 2023-10-11T06:12:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.240+0000 7f4cda7fc640 1 -- 172.21.15.84:0/3463021223 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc004390 con 0x7f4cdc15d4f0 2023-10-11T06:12:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.240+0000 7f4cda7fc640 1 -- 172.21.15.84:0/3463021223 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc003810 con 0x7f4cdc15d4f0 2023-10-11T06:12:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 -- 172.21.15.84:0/3463021223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 msgr2=0x7f4cdc0cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 --2- 172.21.15.84:0/3463021223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0cb540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009850 tx=0x7f4ccc02b960 comp rx=0 tx=0).stop 2023-10-11T06:12:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 -- 172.21.15.84:0/3463021223 shutdown_connections 2023-10-11T06:12:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 --2- 172.21.15.84:0/3463021223 >> v2:172.21.15.99:3300/0 conn(0x7f4cdc0ca300 0x7f4cdc0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 --2- 172.21.15.84:0/3463021223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0cb540 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 -- 172.21.15.84:0/3463021223 >> 172.21.15.84:0/3463021223 conn(0x7f4cdc069c70 msgr2=0x7f4cdc06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 -- 172.21.15.84:0/3463021223 shutdown_connections 2023-10-11T06:12:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.241+0000 7f4ce210a640 1 -- 172.21.15.84:0/3463021223 wait complete. 2023-10-11T06:12:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.243+0000 7f4ce210a640 1 Processor -- start 2023-10-11T06:12:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.243+0000 7f4ce210a640 1 -- start start 2023-10-11T06:12:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4ce210a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc0c9190 0x7f4cdc1f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4ce210a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4ce210a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4ce210a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc15a530 con 0x7f4cdc0ca300 2023-10-11T06:12:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4ce210a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1f3650 con 0x7f4cdc0c9190 2023-10-11T06:12:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4ce210a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0b15d0 con 0x7f4cdc15d4f0 2023-10-11T06:12:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4cdaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4cdaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49524/0 (socket says 172.21.15.84:49524) 2023-10-11T06:12:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4cdaffd640 1 -- 172.21.15.84:0/1993531478 learned_addr learned my addr 172.21.15.84:0/1993531478 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4cdb7fe640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc0c9190 0x7f4cdc1f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.244+0000 7f4cdbfff640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdaffd640 1 -- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 msgr2=0x7f4cdc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdaffd640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdaffd640 1 -- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc0c9190 msgr2=0x7f4cdc1f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdaffd640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc0c9190 0x7f4cdc1f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdaffd640 1 -- 172.21.15.84:0/1993531478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd0009400 con 0x7f4cdc0ca300 2023-10-11T06:12:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdb7fe640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc0c9190 0x7f4cdc1f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdbfff640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cdaffd640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc159c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000e990 tx=0x7f4cd000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cc3fff640 1 -- 172.21.15.84:0/1993531478 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000c9e0 con 0x7f4cdc0ca300 2023-10-11T06:12:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cc3fff640 1 -- 172.21.15.84:0/1993531478 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd000cb40 con 0x7f4cdc0ca300 2023-10-11T06:12:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4cc3fff640 1 -- 172.21.15.84:0/1993531478 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000ced0 con 0x7f4cdc0ca300 2023-10-11T06:12:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.245+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc009460 con 0x7f4cdc0ca300 2023-10-11T06:12:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.246+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc0b1b90 con 0x7f4cdc0ca300 2023-10-11T06:12:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.247+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc073ed0 con 0x7f4cdc0ca300 2023-10-11T06:12:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.247+0000 7f4cc3fff640 1 -- 172.21.15.84:0/1993531478 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4cd0013470 con 0x7f4cdc0ca300 2023-10-11T06:12:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.248+0000 7f4cc3fff640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cc806a830 0x7f4cc806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.248+0000 7f4cc3fff640 1 -- 172.21.15.84:0/1993531478 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4cd00892a0 con 0x7f4cdc0ca300 2023-10-11T06:12:40.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.249+0000 7f4cdb7fe640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cc806a830 0x7f4cc806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.249+0000 7f4cdb7fe640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cc806a830 0x7f4cc806cce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc035000 tx=0x7f4ccc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:40.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.253+0000 7f4cc3fff640 1 -- 172.21.15.84:0/1993531478 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4cd0054520 con 0x7f4cdc0ca300 2023-10-11T06:12:40.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.436+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cdc0b09b0 con 0x7f4cdc0ca300 2023-10-11T06:12:40.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.439+0000 7f4cc3fff640 1 -- 172.21.15.84:0/1993531478 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17848 (secure 0 0 0) 0x7f4cd0057b40 con 0x7f4cdc0ca300 2023-10-11T06:12:40.441 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:40.442 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":37,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:40.001915+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cc806a830 msgr2=0x7f4cc806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cc806a830 0x7f4cc806cce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc035000 tx=0x7f4ccc008030 comp rx=0 tx=0).stop 2023-10-11T06:12:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0ca300 msgr2=0x7f4cdc159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc159c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000e990 tx=0x7f4cd000ee50 comp rx=0 tx=0).stop 2023-10-11T06:12:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 shutdown_connections 2023-10-11T06:12:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4cc806a830 0x7f4cc806cce0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc15d4f0 0x7f4cdc0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc159c50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 --2- 172.21.15.84:0/1993531478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc0c9190 0x7f4cdc1f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 >> 172.21.15.84:0/1993531478 conn(0x7f4cdc069c70 msgr2=0x7f4cdc0cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:40.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 shutdown_connections 2023-10-11T06:12:40.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.442+0000 7f4ce210a640 1 -- 172.21.15.84:0/1993531478 wait complete. 2023-10-11T06:12:40.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:40.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.662+0000 7f0715d18640 1 Processor -- start 2023-10-11T06:12:40.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.663+0000 7f0715d18640 1 -- start start 2023-10-11T06:12:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.664+0000 7f0715d18640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07101651a0 0x7f0710167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.664+0000 7f0715d18640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071015fe40 con 0x7f071015c290 2023-10-11T06:12:40.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.664+0000 7f0715d18640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071006ff30 con 0x7f07100cafb0 2023-10-11T06:12:40.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.664+0000 7f0715d18640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07100cad70 con 0x7f07101651a0 2023-10-11T06:12:40.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f071015c290 0x7f07100c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48226/0 (socket says 172.21.15.84:48226) 2023-10-11T06:12:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070f7fe640 1 -- 172.21.15.84:0/1095320440 learned_addr learned my addr 172.21.15.84:0/1095320440 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070ffff640 1 --2- 172.21.15.84:0/1095320440 >> v2:172.21.15.99:3300/0 conn(0x7f07101651a0 0x7f0710167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897438659 0 0) 0x7f071015fe40 con 0x7f071015c290 2023-10-11T06:12:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06fc003520 con 0x7f071015c290 2023-10-11T06:12:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2721248593 0 0) 0x7f06fc003520 con 0x7f071015c290 2023-10-11T06:12:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 >> v2:172.21.15.99:3300/0 conn(0x7f07101651a0 msgr2=0x7f0710167580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 --2- 172.21.15.84:0/1095320440 >> v2:172.21.15.99:3300/0 conn(0x7f07101651a0 0x7f0710167580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 >> v1:172.21.15.99:6789/0 conn(0x7f07100cafb0 legacy=0x7f07100cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07101600e0 con 0x7f071015c290 2023-10-11T06:12:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f07000017f0 con 0x7f071015c290 2023-10-11T06:12:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070ffff640 1 --2- 172.21.15.84:0/1095320440 >> v2:172.21.15.99:3300/0 conn(0x7f07101651a0 0x7f0710167580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 >> v1:172.21.15.84:6789/0 conn(0x7f071015c290 legacy=0x7f07100c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06fc0039c0 0x7f06fc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f06fc004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06fc0051c0 0x7f06fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07100cad70 con 0x7f06fc0047c0 2023-10-11T06:12:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc00c0d0 con 0x7f06fc0039c0 2023-10-11T06:12:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.665+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc00c290 con 0x7f06fc0051c0 2023-10-11T06:12:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.666+0000 7f0707fff640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f06fc004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.666+0000 7f0707fff640 1 -- 172.21.15.84:0/1095320440 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06fc0051c0 msgr2=0x7f06fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.666+0000 7f0707fff640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06fc0051c0 0x7f06fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.666+0000 7f0707fff640 1 -- 172.21.15.84:0/1095320440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06fc0039c0 msgr2=0x7f06fc003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.666+0000 7f0707fff640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06fc0039c0 0x7f06fc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.666+0000 7f0707fff640 1 -- 172.21.15.84:0/1095320440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc003520 con 0x7f06fc0047c0 2023-10-11T06:12:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.666+0000 7f070ffff640 1 -- 172.21.15.84:0/1095320440 reap_dead start 2023-10-11T06:12:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.667+0000 7f0707fff640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f06fc004c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f06f8003db0 tx=0x7f06fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bad54f6329510cc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.667+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8032600 con 0x7f06fc0047c0 2023-10-11T06:12:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.667+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f802e050 con 0x7f06fc0047c0 2023-10-11T06:12:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.667+0000 7f070effd640 1 -- 172.21.15.84:0/1095320440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f80329e0 con 0x7f06fc0047c0 2023-10-11T06:12:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.668+0000 7f0715d18640 1 -- 172.21.15.84:0/1095320440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 msgr2=0x7f06fc004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.668+0000 7f0715d18640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f06fc004c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f06f8003db0 tx=0x7f06fc0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.668+0000 7f0715d18640 1 -- 172.21.15.84:0/1095320440 shutdown_connections 2023-10-11T06:12:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.668+0000 7f0715d18640 1 --2- 172.21.15.84:0/1095320440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f06fc004c30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.668+0000 7f0715d18640 1 -- 172.21.15.84:0/1095320440 >> 172.21.15.84:0/1095320440 conn(0x7f0710069c70 msgr2=0x7f071006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.668+0000 7f0715d18640 1 -- 172.21.15.84:0/1095320440 shutdown_connections 2023-10-11T06:12:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.668+0000 7f0715d18640 1 -- 172.21.15.84:0/1095320440 wait complete. 2023-10-11T06:12:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.670+0000 7f0715d18640 1 Processor -- start 2023-10-11T06:12:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.670+0000 7f0715d18640 1 -- start start 2023-10-11T06:12:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f0715d18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f071015b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f0715d18640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f071014fd60 0x7f071015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f0715d18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f071015d730 0x7f071015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f0715d18640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07100ae040 con 0x7f06fc0047c0 2023-10-11T06:12:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f0715d18640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07100ae320 con 0x7f071015d730 2023-10-11T06:12:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f0715d18640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07101580c0 con 0x7f071014fd60 2023-10-11T06:12:40.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f070f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f071015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f070f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f071015b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49540/0 (socket says 172.21.15.84:49540) 2023-10-11T06:12:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.671+0000 7f070f7fe640 1 -- 172.21.15.84:0/3376293730 learned_addr learned my addr 172.21.15.84:0/3376293730 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070f7fe640 1 -- 172.21.15.84:0/3376293730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f071014fd60 msgr2=0x7f071015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070f7fe640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f071014fd60 0x7f071015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070f7fe640 1 -- 172.21.15.84:0/3376293730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f071015d730 msgr2=0x7f071015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070f7fe640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f071015d730 0x7f071015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070f7fe640 1 -- 172.21.15.84:0/3376293730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071015e0e0 con 0x7f06fc0047c0 2023-10-11T06:12:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070f7fe640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f071015b640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f070000c700 tx=0x7f070000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:40.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070cff9640 1 -- 172.21.15.84:0/3376293730 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000074e0 con 0x7f06fc0047c0 2023-10-11T06:12:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070cff9640 1 -- 172.21.15.84:0/3376293730 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700007640 con 0x7f06fc0047c0 2023-10-11T06:12:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f070cff9640 1 -- 172.21.15.84:0/3376293730 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000059c0 con 0x7f06fc0047c0 2023-10-11T06:12:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071015e3a0 con 0x7f06fc0047c0 2023-10-11T06:12:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.672+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071015e960 con 0x7f06fc0047c0 2023-10-11T06:12:40.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.673+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0710073ed0 con 0x7f06fc0047c0 2023-10-11T06:12:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.674+0000 7f070cff9640 1 -- 172.21.15.84:0/3376293730 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0700005b20 con 0x7f06fc0047c0 2023-10-11T06:12:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.675+0000 7f070cff9640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f006a860 0x7f06f006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.675+0000 7f070cff9640 1 -- 172.21.15.84:0/3376293730 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0700088380 con 0x7f06fc0047c0 2023-10-11T06:12:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.675+0000 7f0707fff640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f006a860 0x7f06f006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:40.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.676+0000 7f0707fff640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f006a860 0x7f06f006cd10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06f8034000 tx=0x7f06f802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:40.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.680+0000 7f070cff9640 1 -- 172.21.15.84:0/3376293730 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0700057130 con 0x7f06fc0047c0 2023-10-11T06:12:40.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.861+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07100c1c50 con 0x7f06fc0047c0 2023-10-11T06:12:40.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.866+0000 7f070cff9640 1 -- 172.21.15.84:0/3376293730 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17848 (secure 0 0 0) 0x7f0700056cc0 con 0x7f06fc0047c0 2023-10-11T06:12:40.868 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:40.868 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":37,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:40.001915+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:40.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f006a860 msgr2=0x7f06f006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f006a860 0x7f06f006cd10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06f8034000 tx=0x7f06f802f000 comp rx=0 tx=0).stop 2023-10-11T06:12:40.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 msgr2=0x7f071015b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:40.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f071015b640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f070000c700 tx=0x7f070000c7f0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 shutdown_connections 2023-10-11T06:12:40.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f06f006a860 0x7f06f006cd10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f071015d730 0x7f071015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f071014fd60 0x7f071015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 --2- 172.21.15.84:0/3376293730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06fc0047c0 0x7f071015b640 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:40.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 >> 172.21.15.84:0/3376293730 conn(0x7f0710069c70 msgr2=0x7f07101565d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:40.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 shutdown_connections 2023-10-11T06:12:40.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:40.869+0000 7f0715d18640 1 -- 172.21.15.84:0/3376293730 wait complete. 2023-10-11T06:12:40.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T06:12:41.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.095+0000 7f1ae7051640 1 Processor -- start 2023-10-11T06:12:41.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.095+0000 7f1ae7051640 1 -- start start 2023-10-11T06:12:41.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.097+0000 7f1ae7051640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1ae0150300 0x7f1ae00c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:41.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.097+0000 7f1ae7051640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae00ce510 0x7f1ae00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:41.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.097+0000 7f1ae7051640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae00c9b90 con 0x7f1ae014ff30 2023-10-11T06:12:41.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.097+0000 7f1ae7051640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae015da10 con 0x7f1ae0150300 2023-10-11T06:12:41.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.097+0000 7f1ae7051640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae00cc990 con 0x7f1ae00ce510 2023-10-11T06:12:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.097+0000 7f1ad7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1ae0150300 0x7f1ae00c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.097+0000 7f1ae55ee640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae00ce510 0x7f1ae00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.098+0000 7f1ad7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1ae0150300 0x7f1ae00c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:33322/0 (socket says 172.21.15.84:33322) 2023-10-11T06:12:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.098+0000 7f1ad7fff640 1 -- 172.21.15.84:0/3622723016 learned_addr learned my addr 172.21.15.84:0/3622723016 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.098+0000 7f1ad7fff640 1 -- 172.21.15.84:0/3622723016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae00ce510 msgr2=0x7f1ae00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.098+0000 7f1ad7fff640 1 --2- 172.21.15.84:0/3622723016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae00ce510 0x7f1ae00ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.098+0000 7f1ad7fff640 1 -- 172.21.15.84:0/3622723016 >> v1:172.21.15.99:6789/0 conn(0x7f1ae014ff30 legacy=0x7f1ae00cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.098+0000 7f1ad7fff640 1 -- 172.21.15.84:0/3622723016 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae00ceed0 con 0x7f1ae0150300 2023-10-11T06:12:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.099+0000 7f1ad7fff640 1 --2- 172.21.15.84:0/3622723016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae0150300 0x7f1ae00c9650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009870 tx=0x7f1adc02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2582852f21e2365c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.100+0000 7f1ad77fe640 1 -- 172.21.15.84:0/3622723016 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0317c0 con 0x7f1ae0150300 2023-10-11T06:12:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.100+0000 7f1ad77fe640 1 -- 172.21.15.84:0/3622723016 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc031920 con 0x7f1ae0150300 2023-10-11T06:12:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.100+0000 7f1ad77fe640 1 -- 172.21.15.84:0/3622723016 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc031cb0 con 0x7f1ae0150300 2023-10-11T06:12:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 -- 172.21.15.84:0/3622723016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae0150300 msgr2=0x7f1ae00c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3622723016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae0150300 0x7f1ae00c9650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009870 tx=0x7f1adc02b860 comp rx=0 tx=0).stop 2023-10-11T06:12:41.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 -- 172.21.15.84:0/3622723016 shutdown_connections 2023-10-11T06:12:41.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3622723016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae00ce510 0x7f1ae00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:41.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3622723016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae0150300 0x7f1ae00c9650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:41.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 -- 172.21.15.84:0/3622723016 >> 172.21.15.84:0/3622723016 conn(0x7f1ae0069c70 msgr2=0x7f1ae006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:41.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 -- 172.21.15.84:0/3622723016 shutdown_connections 2023-10-11T06:12:41.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.101+0000 7f1ae7051640 1 -- 172.21.15.84:0/3622723016 wait complete. 2023-10-11T06:12:41.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.103+0000 7f1ae7051640 1 Processor -- start 2023-10-11T06:12:41.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.103+0000 7f1ae7051640 1 -- start start 2023-10-11T06:12:41.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae7051640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae00ce510 0x7f1ae014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:41.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae7051640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae014ff30 0x7f1ae0158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:41.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae7051640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae0150300 0x7f1ae01588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:41.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae7051640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae013ca60 con 0x7f1ae0150300 2023-10-11T06:12:41.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae7051640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0139c90 con 0x7f1ae00ce510 2023-10-11T06:12:41.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae7051640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae0139f70 con 0x7f1ae014ff30 2023-10-11T06:12:41.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae55ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae0150300 0x7f1ae01588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:41.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae55ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae0150300 0x7f1ae01588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49554/0 (socket says 172.21.15.84:49554) 2023-10-11T06:12:41.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae55ee640 1 -- 172.21.15.84:0/3041661413 learned_addr learned my addr 172.21.15.84:0/3041661413 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:41.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae4ded640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae00ce510 0x7f1ae014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:41.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ad7fff640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae014ff30 0x7f1ae0158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:41.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae55ee640 1 -- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae014ff30 msgr2=0x7f1ae0158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:41.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae55ee640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae014ff30 0x7f1ae0158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:41.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.105+0000 7f1ae55ee640 1 -- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae00ce510 msgr2=0x7f1ae014e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:41.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ae55ee640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae00ce510 0x7f1ae014e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:41.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ae55ee640 1 -- 172.21.15.84:0/3041661413 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0094d0 con 0x7f1ae0150300 2023-10-11T06:12:41.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ae55ee640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae0150300 0x7f1ae01588a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8010d70 tx=0x7f1ad800e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:41.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ad57fa640 1 -- 172.21.15.84:0/3041661413 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800b930 con 0x7f1ae0150300 2023-10-11T06:12:41.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ad57fa640 1 -- 172.21.15.84:0/3041661413 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad8011660 con 0x7f1ae0150300 2023-10-11T06:12:41.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ad57fa640 1 -- 172.21.15.84:0/3041661413 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011a10 con 0x7f1ae0150300 2023-10-11T06:12:41.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae013a250 con 0x7f1ae0150300 2023-10-11T06:12:41.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.106+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae014cd00 con 0x7f1ae0150300 2023-10-11T06:12:41.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.107+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae013a770 con 0x7f1ae0150300 2023-10-11T06:12:41.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.108+0000 7f1ad57fa640 1 -- 172.21.15.84:0/3041661413 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1ad8011b90 con 0x7f1ae0150300 2023-10-11T06:12:41.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.109+0000 7f1ad57fa640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1acc06a830 0x7f1acc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:41.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.109+0000 7f1ad57fa640 1 -- 172.21.15.84:0/3041661413 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ad808bcf0 con 0x7f1ae0150300 2023-10-11T06:12:41.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.110+0000 7f1ae4ded640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1acc06a830 0x7f1acc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:41.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.110+0000 7f1ae4ded640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1acc06a830 0x7f1acc06cce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01f0840 tx=0x7f1ad0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.116+0000 7f1ad57fa640 1 -- 172.21.15.84:0/3041661413 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1ae013a770 con 0x7f1ae0150300 2023-10-11T06:12:41.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:41.327+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1ae013a770 con 0x7f1ae0150300 2023-10-11T06:12:42.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.016+0000 7f1ad57fa640 1 -- 172.21.15.84:0/3041661413 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1ae013a770 con 0x7f1ae0150300 2023-10-11T06:12:42.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1acc06a830 msgr2=0x7f1acc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:42.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1acc06a830 0x7f1acc06cce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01f0840 tx=0x7f1ad0009040 comp rx=0 tx=0).stop 2023-10-11T06:12:42.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae0150300 msgr2=0x7f1ae01588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:42.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae0150300 0x7f1ae01588a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8010d70 tx=0x7f1ad800e730 comp rx=0 tx=0).stop 2023-10-11T06:12:42.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 shutdown_connections 2023-10-11T06:12:42.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1acc06a830 0x7f1acc06cce0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ae0150300 0x7f1ae01588a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ae014ff30 0x7f1ae0158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 --2- 172.21.15.84:0/3041661413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ae00ce510 0x7f1ae014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 >> 172.21.15.84:0/3041661413 conn(0x7f1ae0069c70 msgr2=0x7f1ae00b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:42.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 shutdown_connections 2023-10-11T06:12:42.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.020+0000 7f1ae7051640 1 -- 172.21.15.84:0/3041661413 wait complete. 2023-10-11T06:12:42.041 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T06:12:42.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T06:12:42.158 INFO:tasks.ceph.mds.b.smithi099.stderr:2023-10-11T06:12:42.156+0000 7f782a45d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T06:12:42.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.255+0000 7fd8f47f5640 1 Processor -- start 2023-10-11T06:12:42.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.255+0000 7fd8f47f5640 1 -- start start 2023-10-11T06:12:42.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.256+0000 7fd8f47f5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 0x7fd8ec0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.256+0000 7fd8f47f5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec0c9b90 con 0x7fd8ec0ce510 2023-10-11T06:12:42.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.256+0000 7fd8f47f5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec15da10 con 0x7fd8ec150300 2023-10-11T06:12:42.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.256+0000 7fd8f47f5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0cc990 con 0x7fd8ec14ff30 2023-10-11T06:12:42.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.257+0000 7fd8f2d92640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd8ec0ce510 0x7fd8ec0ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48238/0 (socket says 172.21.15.84:48238) 2023-10-11T06:12:42.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.257+0000 7fd8f2d92640 1 -- 172.21.15.84:0/302754912 learned_addr learned my addr 172.21.15.84:0/302754912 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:42.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.257+0000 7fd8f2591640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 0x7fd8ec0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.257+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244558910 0 0) 0x7fd8ec0c9b90 con 0x7fd8ec0ce510 2023-10-11T06:12:42.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.257+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d8003520 con 0x7fd8ec0ce510 2023-10-11T06:12:42.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 901276968 0 0) 0x7fd8d8003520 con 0x7fd8ec0ce510 2023-10-11T06:12:42.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 msgr2=0x7fd8ec0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:42.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 0x7fd8ec0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 >> v1:172.21.15.99:6789/0 conn(0x7fd8ec150300 legacy=0x7fd8ec0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:42.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec15ce00 con 0x7fd8ec0ce510 2023-10-11T06:12:42.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fd8e40019f0 con 0x7fd8ec0ce510 2023-10-11T06:12:42.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 >> v1:172.21.15.84:6789/0 conn(0x7fd8ec0ce510 legacy=0x7fd8ec0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:42.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 0x7fd8d80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8003f70 0x7fd8d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8004d00 0x7fd8d8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0cc990 con 0x7fd8d8004d00 2023-10-11T06:12:42.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d800ff50 con 0x7fd8d8003f70 2023-10-11T06:12:42.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d8010110 con 0x7fd8ec14ff30 2023-10-11T06:12:42.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f2d92640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8004d00 0x7fd8d8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f2591640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 0x7fd8d80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f1d90640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8003f70 0x7fd8d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f2d92640 1 -- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 msgr2=0x7fd8d80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:42.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.258+0000 7fd8f2d92640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14ff30 0x7fd8d80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.259+0000 7fd8f2d92640 1 -- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8003f70 msgr2=0x7fd8d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:42.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.259+0000 7fd8f2d92640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8003f70 0x7fd8d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.259+0000 7fd8f2d92640 1 -- 172.21.15.84:0/302754912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d8003520 con 0x7fd8d8004d00 2023-10-11T06:12:42.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.260+0000 7fd8f2d92640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8004d00 0x7fd8d8005170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400a750 tx=0x7fd8d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc866172c2b8d2d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:42.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.260+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4034630 con 0x7fd8d8004d00 2023-10-11T06:12:42.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.260+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e402f070 con 0x7fd8d8004d00 2023-10-11T06:12:42.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.260+0000 7fd8f158f640 1 -- 172.21.15.84:0/302754912 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40349d0 con 0x7fd8d8004d00 2023-10-11T06:12:42.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.261+0000 7fd8f47f5640 1 -- 172.21.15.84:0/302754912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8004d00 msgr2=0x7fd8d8005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:42.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.261+0000 7fd8f47f5640 1 --2- 172.21.15.84:0/302754912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8004d00 0x7fd8d8005170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400a750 tx=0x7fd8d80057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.261+0000 7fd8f2d92640 1 -- 172.21.15.84:0/302754912 reap_dead start 2023-10-11T06:12:42.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.261+0000 7fd8f47f5640 1 -- 172.21.15.84:0/302754912 shutdown_connections 2023-10-11T06:12:42.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.261+0000 7fd8f47f5640 1 -- 172.21.15.84:0/302754912 >> 172.21.15.84:0/302754912 conn(0x7fd8ec069c70 msgr2=0x7fd8ec06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:42.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.261+0000 7fd8f47f5640 1 -- 172.21.15.84:0/302754912 shutdown_connections 2023-10-11T06:12:42.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.261+0000 7fd8f47f5640 1 -- 172.21.15.84:0/302754912 wait complete. 2023-10-11T06:12:42.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.263+0000 7fd8f47f5640 1 Processor -- start 2023-10-11T06:12:42.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.263+0000 7fd8f47f5640 1 -- start start 2023-10-11T06:12:42.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.264+0000 7fd8f47f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8004d00 0x7fd8ec0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.264+0000 7fd8f47f5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14d600 0x7fd8ec14ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.264+0000 7fd8f47f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ec0b2580 0x7fd8ec150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.264+0000 7fd8f47f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b3b50 con 0x7fd8ec0b2580 2023-10-11T06:12:42.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.264+0000 7fd8f47f5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b38d0 con 0x7fd8d8004d00 2023-10-11T06:12:42.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.264+0000 7fd8f47f5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec14d3d0 con 0x7fd8ec14d600 2023-10-11T06:12:42.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ec0b2580 0x7fd8ec150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ec0b2580 0x7fd8ec150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49582/0 (socket says 172.21.15.84:49582) 2023-10-11T06:12:42.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 -- 172.21.15.84:0/2731246403 learned_addr learned my addr 172.21.15.84:0/2731246403 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:42.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2591640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8004d00 0x7fd8ec0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f1d90640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14d600 0x7fd8ec14ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 -- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14d600 msgr2=0x7fd8ec14ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:42.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14d600 0x7fd8ec14ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 -- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8004d00 msgr2=0x7fd8ec0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:42.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8004d00 0x7fd8ec0b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:42.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2d92640 1 -- 172.21.15.84:0/2731246403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0b3ea0 con 0x7fd8ec0b2580 2023-10-11T06:12:42.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.265+0000 7fd8f2591640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8004d00 0x7fd8ec0b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:42.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.266+0000 7fd8f2d92640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ec0b2580 0x7fd8ec150700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd8e402caa0 tx=0x7fd8ec159670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:42.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.266+0000 7fd8e2ffd640 1 -- 172.21.15.84:0/2731246403 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4034630 con 0x7fd8ec0b2580 2023-10-11T06:12:42.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.266+0000 7fd8e2ffd640 1 -- 172.21.15.84:0/2731246403 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e4034790 con 0x7fd8ec0b2580 2023-10-11T06:12:42.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.266+0000 7fd8e2ffd640 1 -- 172.21.15.84:0/2731246403 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4034b00 con 0x7fd8ec0b2580 2023-10-11T06:12:42.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.266+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec0b4180 con 0x7fd8ec0b2580 2023-10-11T06:12:42.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.266+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec0b29f0 con 0x7fd8ec0b2580 2023-10-11T06:12:42.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.267+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec15d680 con 0x7fd8ec0b2580 2023-10-11T06:12:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.268+0000 7fd8e2ffd640 1 -- 172.21.15.84:0/2731246403 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd8e4031030 con 0x7fd8ec0b2580 2023-10-11T06:12:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.269+0000 7fd8e2ffd640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd8d006a830 0x7fd8d006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.269+0000 7fd8e2ffd640 1 -- 172.21.15.84:0/2731246403 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd8e40aad90 con 0x7fd8ec0b2580 2023-10-11T06:12:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.269+0000 7fd8f2591640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd8d006a830 0x7fd8d006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:42.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.270+0000 7fd8f2591640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd8d006a830 0x7fd8d006cce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009a00 tx=0x7fd8e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:42.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.273+0000 7fd8e2ffd640 1 -- 172.21.15.84:0/2731246403 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd8ec15d680 con 0x7fd8ec0b2580 2023-10-11T06:12:42.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:42.498+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd8ec15d680 con 0x7fd8ec0b2580 2023-10-11T06:12:43.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.019+0000 7fd8e2ffd640 1 -- 172.21.15.84:0/2731246403 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8ec15d680 con 0x7fd8ec0b2580 2023-10-11T06:12:43.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd8d006a830 msgr2=0x7fd8d006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd8d006a830 0x7fd8d006cce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009a00 tx=0x7fd8e8008040 comp rx=0 tx=0).stop 2023-10-11T06:12:43.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ec0b2580 msgr2=0x7fd8ec150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ec0b2580 0x7fd8ec150700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd8e402caa0 tx=0x7fd8ec159670 comp rx=0 tx=0).stop 2023-10-11T06:12:43.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 shutdown_connections 2023-10-11T06:12:43.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd8d006a830 0x7fd8d006cce0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8ec0b2580 0x7fd8ec150700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8ec14d600 0x7fd8ec14ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 --2- 172.21.15.84:0/2731246403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8d8004d00 0x7fd8ec0b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.023+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 >> 172.21.15.84:0/2731246403 conn(0x7fd8ec069c70 msgr2=0x7fd8ec06ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:43.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.024+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 shutdown_connections 2023-10-11T06:12:43.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.024+0000 7fd8f47f5640 1 -- 172.21.15.84:0/2731246403 wait complete. 2023-10-11T06:12:43.045 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T06:12:43.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:12:43.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.262+0000 7f0d1e241640 1 Processor -- start 2023-10-11T06:12:43.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.262+0000 7f0d1e241640 1 -- start start 2023-10-11T06:12:43.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d1e241640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d1815d4f0 0x7f0d180cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d1e241640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 0x7f0d180c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d1e241640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d180c9190 0x7f0d1815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d1e241640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d18072310 con 0x7f0d180c9190 2023-10-11T06:12:43.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d1e241640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d18071a90 con 0x7f0d1815d4f0 2023-10-11T06:12:43.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d1e241640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18070280 con 0x7f0d180ca300 2023-10-11T06:12:43.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d17fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d180c9190 0x7f0d1815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d17fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d180c9190 0x7f0d1815dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49588/0 (socket says 172.21.15.84:49588) 2023-10-11T06:12:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d17fff640 1 -- 172.21.15.84:0/3118425684 learned_addr learned my addr 172.21.15.84:0/3118425684 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d177fe640 1 --2- 172.21.15.84:0/3118425684 >> v2:172.21.15.99:3300/0 conn(0x7f0d1815d4f0 0x7f0d180cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.264+0000 7f0d0ffff640 1 --2- 172.21.15.84:0/3118425684 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 0x7f0d180c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.265+0000 7f0d17fff640 1 -- 172.21.15.84:0/3118425684 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 msgr2=0x7f0d180c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.265+0000 7f0d17fff640 1 --2- 172.21.15.84:0/3118425684 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 0x7f0d180c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.265+0000 7f0d17fff640 1 -- 172.21.15.84:0/3118425684 >> v2:172.21.15.99:3300/0 conn(0x7f0d1815d4f0 msgr2=0x7f0d180cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.265+0000 7f0d17fff640 1 --2- 172.21.15.84:0/3118425684 >> v2:172.21.15.99:3300/0 conn(0x7f0d1815d4f0 0x7f0d180cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.265+0000 7f0d17fff640 1 -- 172.21.15.84:0/3118425684 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d18071cc0 con 0x7f0d180c9190 2023-10-11T06:12:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.266+0000 7f0d17fff640 1 --2- 172.21.15.84:0/3118425684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d1815dbc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d180cf6a0 tx=0x7f0d1002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d542b776c940cf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.266+0000 7f0d16ffd640 1 -- 172.21.15.84:0/3118425684 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1002ca70 con 0x7f0d180c9190 2023-10-11T06:12:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.266+0000 7f0d16ffd640 1 -- 172.21.15.84:0/3118425684 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1002cbd0 con 0x7f0d180c9190 2023-10-11T06:12:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.266+0000 7f0d16ffd640 1 -- 172.21.15.84:0/3118425684 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100343f0 con 0x7f0d180c9190 2023-10-11T06:12:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 -- 172.21.15.84:0/3118425684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 msgr2=0x7f0d1815dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 --2- 172.21.15.84:0/3118425684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d1815dbc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0d180cf6a0 tx=0x7f0d1002bed0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 -- 172.21.15.84:0/3118425684 shutdown_connections 2023-10-11T06:12:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 --2- 172.21.15.84:0/3118425684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d1815dbc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 --2- 172.21.15.84:0/3118425684 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 0x7f0d180c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 --2- 172.21.15.84:0/3118425684 >> v2:172.21.15.99:3300/0 conn(0x7f0d1815d4f0 0x7f0d180cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 -- 172.21.15.84:0/3118425684 >> 172.21.15.84:0/3118425684 conn(0x7f0d18069c70 msgr2=0x7f0d1806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:43.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 -- 172.21.15.84:0/3118425684 shutdown_connections 2023-10-11T06:12:43.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.267+0000 7f0d1e241640 1 -- 172.21.15.84:0/3118425684 wait complete. 2023-10-11T06:12:43.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.268+0000 7f0d1e241640 1 Processor -- start 2023-10-11T06:12:43.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.269+0000 7f0d1e241640 1 -- start start 2023-10-11T06:12:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d1e241640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d181f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d1e241640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 0x7f0d18154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d1e241640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1815d4f0 0x7f0d1814ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d1e241640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181531c0 con 0x7f0d180c9190 2023-10-11T06:12:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d1e241640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181548d0 con 0x7f0d1815d4f0 2023-10-11T06:12:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d1e241640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18154b00 con 0x7f0d180ca300 2023-10-11T06:12:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d181f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d181f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49598/0 (socket says 172.21.15.84:49598) 2023-10-11T06:12:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 -- 172.21.15.84:0/309309913 learned_addr learned my addr 172.21.15.84:0/309309913 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 -- 172.21.15.84:0/309309913 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 msgr2=0x7f0d18154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 0x7f0d18154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 -- 172.21.15.84:0/309309913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1815d4f0 msgr2=0x7f0d1814ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d17fff640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1815d4f0 0x7f0d1814ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1815d4f0 0x7f0d1814ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.270+0000 7f0d177fe640 1 -- 172.21.15.84:0/309309913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10009b30 con 0x7f0d180c9190 2023-10-11T06:12:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.271+0000 7f0d177fe640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d181f07d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800d7d0 tx=0x7f0d0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.271+0000 7f0d14ff9640 1 -- 172.21.15.84:0/309309913 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080096c0 con 0x7f0d180c9190 2023-10-11T06:12:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.271+0000 7f0d14ff9640 1 -- 172.21.15.84:0/309309913 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d08009820 con 0x7f0d180c9190 2023-10-11T06:12:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.271+0000 7f0d14ff9640 1 -- 172.21.15.84:0/309309913 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08009bd0 con 0x7f0d180c9190 2023-10-11T06:12:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.271+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d18154de0 con 0x7f0d180c9190 2023-10-11T06:12:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.271+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d18152de0 con 0x7f0d180c9190 2023-10-11T06:12:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.272+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18073ed0 con 0x7f0d180c9190 2023-10-11T06:12:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.273+0000 7f0d14ff9640 1 -- 172.21.15.84:0/309309913 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0d08009d30 con 0x7f0d180c9190 2023-10-11T06:12:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.274+0000 7f0d14ff9640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cf806a830 0x7f0cf806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.274+0000 7f0d14ff9640 1 -- 172.21.15.84:0/309309913 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d080897f0 con 0x7f0d180c9190 2023-10-11T06:12:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.274+0000 7f0d0ffff640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cf806a830 0x7f0cf806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.275+0000 7f0d0ffff640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cf806a830 0x7f0cf806cce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0d180c86a0 tx=0x7f0d00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:43.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.278+0000 7f0d14ff9640 1 -- 172.21.15.84:0/309309913 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0d08054b00 con 0x7f0d180c9190 2023-10-11T06:12:43.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.508+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d180b09b0 con 0x7f0d180c9190 2023-10-11T06:12:43.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.510+0000 7f0d14ff9640 1 -- 172.21.15.84:0/309309913 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f0d08058120 con 0x7f0d180c9190 2023-10-11T06:12:43.511 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:43.512 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:12:40.002715+0000","modified":"2023-10-11T06:12:43.012925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5334":{"gid":5334,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/2172719017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2172719017},{"type":"v1","addr":"172.21.15.99:6835","nonce":2172719017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T06:12:43.512 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 15 2023-10-11T06:12:43.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.512+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cf806a830 msgr2=0x7f0cf806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.512+0000 7f0d1e241640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cf806a830 0x7f0cf806cce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0d180c86a0 tx=0x7f0d00008040 comp rx=0 tx=0).stop 2023-10-11T06:12:43.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.512+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 msgr2=0x7f0d181f07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.512+0000 7f0d1e241640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d181f07d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800d7d0 tx=0x7f0d0800dc90 comp rx=0 tx=0).stop 2023-10-11T06:12:43.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 shutdown_connections 2023-10-11T06:12:43.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cf806a830 0x7f0cf806cce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1815d4f0 0x7f0d1814ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d180ca300 0x7f0d18154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 --2- 172.21.15.84:0/309309913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d180c9190 0x7f0d181f07d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 >> 172.21.15.84:0/309309913 conn(0x7f0d18069c70 msgr2=0x7f0d180ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:43.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 shutdown_connections 2023-10-11T06:12:43.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.513+0000 7f0d1e241640 1 -- 172.21.15.84:0/309309913 wait complete. 2023-10-11T06:12:43.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:43.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.735+0000 7f6309f27640 1 Processor -- start 2023-10-11T06:12:43.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.735+0000 7f6309f27640 1 -- start start 2023-10-11T06:12:43.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f6309f27640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f630415d4f0 0x7f63040cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f6309f27640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63040ca300 0x7f63040c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f6309f27640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6304071a90 con 0x7f63040c9190 2023-10-11T06:12:43.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f6309f27640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6304072310 con 0x7f630415d4f0 2023-10-11T06:12:43.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f6309f27640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6304070280 con 0x7f63040ca300 2023-10-11T06:12:43.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f63037fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f630415d4f0 0x7f63040cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f63037fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f630415d4f0 0x7f63040cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49614/0 (socket says 172.21.15.84:49614) 2023-10-11T06:12:43.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.737+0000 7f63037fe640 1 -- 172.21.15.84:0/1917376256 learned_addr learned my addr 172.21.15.84:0/1917376256 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:43.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.738+0000 7f62fbfff640 1 --2- 172.21.15.84:0/1917376256 >> v2:172.21.15.99:3300/0 conn(0x7f63040ca300 0x7f63040c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.738+0000 7f63037fe640 1 -- 172.21.15.84:0/1917376256 >> v2:172.21.15.99:3300/0 conn(0x7f63040ca300 msgr2=0x7f63040c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:43.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.738+0000 7f63037fe640 1 --2- 172.21.15.84:0/1917376256 >> v2:172.21.15.99:3300/0 conn(0x7f63040ca300 0x7f63040c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.738+0000 7f63037fe640 1 -- 172.21.15.84:0/1917376256 >> v1:172.21.15.99:6789/0 conn(0x7f63040c9190 legacy=0x7f630415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:43.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.738+0000 7f63037fe640 1 -- 172.21.15.84:0/1917376256 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304071cc0 con 0x7f630415d4f0 2023-10-11T06:12:43.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.738+0000 7f62fbfff640 1 --2- 172.21.15.84:0/1917376256 >> v2:172.21.15.99:3300/0 conn(0x7f63040ca300 0x7f63040c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:43.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.739+0000 7f63037fe640 1 --2- 172.21.15.84:0/1917376256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f630415d4f0 0x7f63040cb540 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f63040b0980 tx=0x7f62f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2393834344cfe228 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:43.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.739+0000 7f6302ffd640 1 -- 172.21.15.84:0/1917376256 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4004230 con 0x7f630415d4f0 2023-10-11T06:12:43.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.739+0000 7f6302ffd640 1 -- 172.21.15.84:0/1917376256 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4004390 con 0x7f630415d4f0 2023-10-11T06:12:43.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.739+0000 7f6302ffd640 1 -- 172.21.15.84:0/1917376256 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4003810 con 0x7f630415d4f0 2023-10-11T06:12:43.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 -- 172.21.15.84:0/1917376256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f630415d4f0 msgr2=0x7f63040cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 --2- 172.21.15.84:0/1917376256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f630415d4f0 0x7f63040cb540 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f63040b0980 tx=0x7f62f402b8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 -- 172.21.15.84:0/1917376256 shutdown_connections 2023-10-11T06:12:43.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 --2- 172.21.15.84:0/1917376256 >> v2:172.21.15.99:3300/0 conn(0x7f63040ca300 0x7f63040c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 --2- 172.21.15.84:0/1917376256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f630415d4f0 0x7f63040cb540 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 -- 172.21.15.84:0/1917376256 >> 172.21.15.84:0/1917376256 conn(0x7f6304069c70 msgr2=0x7f630406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:43.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 -- 172.21.15.84:0/1917376256 shutdown_connections 2023-10-11T06:12:43.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.740+0000 7f6309f27640 1 -- 172.21.15.84:0/1917376256 wait complete. 2023-10-11T06:12:43.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.742+0000 7f6309f27640 1 Processor -- start 2023-10-11T06:12:43.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.742+0000 7f6309f27640 1 -- start start 2023-10-11T06:12:43.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6309f27640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63040c9190 0x7f63040b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6309f27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63040ca300 0x7f630415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6309f27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630415d4f0 0x7f6304156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6309f27640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630415ad10 con 0x7f63040ca300 2023-10-11T06:12:43.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6309f27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63041561e0 con 0x7f630415d4f0 2023-10-11T06:12:43.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6309f27640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6304155370 con 0x7f63040c9190 2023-10-11T06:12:43.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63040ca300 0x7f630415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63040ca300 0x7f630415be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49626/0 (socket says 172.21.15.84:49626) 2023-10-11T06:12:43.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 -- 172.21.15.84:0/847717577 learned_addr learned my addr 172.21.15.84:0/847717577 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:43.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 -- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63040c9190 msgr2=0x7f63040b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63040c9190 0x7f63040b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 -- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630415d4f0 msgr2=0x7f6304156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6303fff640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630415d4f0 0x7f6304156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630415d4f0 0x7f6304156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f62fbfff640 1 -- 172.21.15.84:0/847717577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4009460 con 0x7f63040ca300 2023-10-11T06:12:43.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.743+0000 7f6303fff640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630415d4f0 0x7f6304156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:43.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.744+0000 7f62fbfff640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63040ca300 0x7f630415be50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00c7c0 tx=0x7f62ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:43.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.744+0000 7f6300ff9640 1 -- 172.21.15.84:0/847717577 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec007820 con 0x7f63040ca300 2023-10-11T06:12:43.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.744+0000 7f6300ff9640 1 -- 172.21.15.84:0/847717577 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec007980 con 0x7f63040ca300 2023-10-11T06:12:43.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.744+0000 7f6300ff9640 1 -- 172.21.15.84:0/847717577 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec007d10 con 0x7f63040ca300 2023-10-11T06:12:43.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.744+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304155610 con 0x7f63040ca300 2023-10-11T06:12:43.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.744+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6304155b90 con 0x7f63040ca300 2023-10-11T06:12:43.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.745+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6304073ed0 con 0x7f63040ca300 2023-10-11T06:12:43.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.746+0000 7f6300ff9640 1 -- 172.21.15.84:0/847717577 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f62ec0113f0 con 0x7f63040ca300 2023-10-11T06:12:43.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.747+0000 7f6300ff9640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62e406a860 0x7f62e406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:43.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.747+0000 7f6300ff9640 1 -- 172.21.15.84:0/847717577 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62ec0890e0 con 0x7f63040ca300 2023-10-11T06:12:43.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.748+0000 7f63037fe640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62e406a860 0x7f62e406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:43.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.748+0000 7f63037fe640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62e406a860 0x7f62e406cd10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62f4035000 tx=0x7f62f4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:43.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.751+0000 7f6300ff9640 1 -- 172.21.15.84:0/847717577 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f62ec0542c0 con 0x7f63040ca300 2023-10-11T06:12:43.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.939+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63040c8f70 con 0x7f63040ca300 2023-10-11T06:12:43.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.941+0000 7f6300ff9640 1 -- 172.21.15.84:0/847717577 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17835 (secure 0 0 0) 0x7f62ec0578e0 con 0x7f63040ca300 2023-10-11T06:12:43.943 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:43.944 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":38,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:41.006502+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:43.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62e406a860 msgr2=0x7f62e406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62e406a860 0x7f62e406cd10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62f4035000 tx=0x7f62f4030000 comp rx=0 tx=0).stop 2023-10-11T06:12:43.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63040ca300 msgr2=0x7f630415be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:43.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63040ca300 0x7f630415be50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00c7c0 tx=0x7f62ec00cc80 comp rx=0 tx=0).stop 2023-10-11T06:12:43.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 shutdown_connections 2023-10-11T06:12:43.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62e406a860 0x7f62e406cd10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f630415d4f0 0x7f6304156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63040ca300 0x7f630415be50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 --2- 172.21.15.84:0/847717577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63040c9190 0x7f63040b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:43.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 >> 172.21.15.84:0/847717577 conn(0x7f6304069c70 msgr2=0x7f63040ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:43.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 shutdown_connections 2023-10-11T06:12:43.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:43.944+0000 7f6309f27640 1 -- 172.21.15.84:0/847717577 wait complete. 2023-10-11T06:12:43.963 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T06:12:43.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:12:44.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.173+0000 7ff91f4e7640 1 Processor -- start 2023-10-11T06:12:44.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.173+0000 7ff91f4e7640 1 -- start start 2023-10-11T06:12:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff91f4e7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 0x7ff9180cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff91f4e7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff918071a90 con 0x7ff91815d4f0 2023-10-11T06:12:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff91f4e7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff918070280 con 0x7ff9180c9360 2023-10-11T06:12:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff91f4e7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff918072310 con 0x7ff9180ca300 2023-10-11T06:12:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff91d283640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff91815d4f0 0x7ff9180c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48254/0 (socket says 172.21.15.84:48254) 2023-10-11T06:12:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff91d283640 1 -- 172.21.15.84:0/213366822 learned_addr learned my addr 172.21.15.84:0/213366822 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118300487 0 0) 0x7ff918071a90 con 0x7ff91815d4f0 2023-10-11T06:12:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff904003520 con 0x7ff91815d4f0 2023-10-11T06:12:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.175+0000 7ff91ca82640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 0x7ff9180cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 676690192 0 0) 0x7ff904003520 con 0x7ff91815d4f0 2023-10-11T06:12:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 msgr2=0x7ff9180cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 0x7ff9180cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 >> v1:172.21.15.99:6789/0 conn(0x7ff9180c9360 legacy=0x7ff9180ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918071cc0 con 0x7ff91815d4f0 2023-10-11T06:12:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7ff9140017f0 con 0x7ff91815d4f0 2023-10-11T06:12:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 >> v1:172.21.15.84:6789/0 conn(0x7ff91815d4f0 legacy=0x7ff9180c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9040039a0 0x7ff904003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 0x7ff9040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904004d00 0x7ff904005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff918072310 con 0x7ff904004d00 2023-10-11T06:12:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90400bb10 con 0x7ff9040039a0 2023-10-11T06:12:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90400bca0 con 0x7ff9180ca300 2023-10-11T06:12:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff91da84640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904004d00 0x7ff904005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff91d283640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9040039a0 0x7ff904003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.176+0000 7ff91ca82640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 0x7ff9040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.177+0000 7ff91da84640 1 -- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 msgr2=0x7ff9040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.177+0000 7ff91da84640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9180ca300 0x7ff9040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.177+0000 7ff91da84640 1 -- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9040039a0 msgr2=0x7ff904003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.177+0000 7ff91da84640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9040039a0 0x7ff904003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.177+0000 7ff91da84640 1 -- 172.21.15.84:0/213366822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff904003520 con 0x7ff904004d00 2023-10-11T06:12:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.178+0000 7ff91da84640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904004d00 0x7ff904005150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff910001e50 tx=0x7ff9040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f453f6ad9b320c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.178+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910003b70 con 0x7ff904004d00 2023-10-11T06:12:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.178+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910003cd0 con 0x7ff904004d00 2023-10-11T06:12:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.178+0000 7ff90ffff640 1 -- 172.21.15.84:0/213366822 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910005310 con 0x7ff904004d00 2023-10-11T06:12:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.178+0000 7ff91f4e7640 1 -- 172.21.15.84:0/213366822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904004d00 msgr2=0x7ff904005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.178+0000 7ff91f4e7640 1 --2- 172.21.15.84:0/213366822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff904004d00 0x7ff904005150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff910001e50 tx=0x7ff9040057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.178+0000 7ff91da84640 1 -- 172.21.15.84:0/213366822 reap_dead start 2023-10-11T06:12:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.179+0000 7ff91f4e7640 1 -- 172.21.15.84:0/213366822 shutdown_connections 2023-10-11T06:12:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.179+0000 7ff91f4e7640 1 -- 172.21.15.84:0/213366822 >> 172.21.15.84:0/213366822 conn(0x7ff918069c70 msgr2=0x7ff91806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.179+0000 7ff91f4e7640 1 -- 172.21.15.84:0/213366822 shutdown_connections 2023-10-11T06:12:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.179+0000 7ff91f4e7640 1 -- 172.21.15.84:0/213366822 wait complete. 2023-10-11T06:12:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.180+0000 7ff91f4e7640 1 Processor -- start 2023-10-11T06:12:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.180+0000 7ff91f4e7640 1 -- start start 2023-10-11T06:12:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.181+0000 7ff91f4e7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904004d00 0x7ff918156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.181+0000 7ff91f4e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff918159920 0x7ff9181570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.181+0000 7ff91f4e7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff91815b070 0x7ff9180c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.181+0000 7ff91f4e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180c56d0 con 0x7ff918159920 2023-10-11T06:12:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.181+0000 7ff91f4e7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9181f1890 con 0x7ff904004d00 2023-10-11T06:12:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.181+0000 7ff91f4e7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180b3920 con 0x7ff91815b070 2023-10-11T06:12:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff918159920 0x7ff9181570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff918159920 0x7ff9181570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49646/0 (socket says 172.21.15.84:49646) 2023-10-11T06:12:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 -- 172.21.15.84:0/1657491578 learned_addr learned my addr 172.21.15.84:0/1657491578 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 -- 172.21.15.84:0/1657491578 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff91815b070 msgr2=0x7ff9180c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff91815b070 0x7ff9180c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91d283640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904004d00 0x7ff918156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 -- 172.21.15.84:0/1657491578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904004d00 msgr2=0x7ff918156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904004d00 0x7ff918156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 -- 172.21.15.84:0/1657491578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180b3c00 con 0x7ff918159920 2023-10-11T06:12:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.182+0000 7ff91ca82640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff918159920 0x7ff9181570e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff90800f500 tx=0x7ff90800f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.183+0000 7ff90dffb640 1 -- 172.21.15.84:0/1657491578 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908014870 con 0x7ff918159920 2023-10-11T06:12:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.183+0000 7ff90dffb640 1 -- 172.21.15.84:0/1657491578 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9080149d0 con 0x7ff918159920 2023-10-11T06:12:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.183+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180c5920 con 0x7ff918159920 2023-10-11T06:12:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.183+0000 7ff90dffb640 1 -- 172.21.15.84:0/1657491578 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908014d80 con 0x7ff918159920 2023-10-11T06:12:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.183+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9181566d0 con 0x7ff918159920 2023-10-11T06:12:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.184+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff918073ed0 con 0x7ff918159920 2023-10-11T06:12:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.185+0000 7ff90dffb640 1 -- 172.21.15.84:0/1657491578 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff908015410 con 0x7ff918159920 2023-10-11T06:12:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.186+0000 7ff90dffb640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8fc06a830 0x7ff8fc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.186+0000 7ff90dffb640 1 -- 172.21.15.84:0/1657491578 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff908089a30 con 0x7ff918159920 2023-10-11T06:12:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.186+0000 7ff91d283640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8fc06a830 0x7ff8fc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.187+0000 7ff91d283640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8fc06a830 0x7ff8fc06cce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff9140011d0 tx=0x7ff914009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.190+0000 7ff90dffb640 1 -- 172.21.15.84:0/1657491578 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff908054c20 con 0x7ff918159920 2023-10-11T06:12:44.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.360+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff91815d8c0 con 0x7ff8fc06a830 2023-10-11T06:12:44.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.376+0000 7ff90dffb640 1 -- 172.21.15.84:0/1657491578 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308754 (secure 0 0 0) 0x7ff914006dc0 con 0x7ff8fc06a830 2023-10-11T06:12:44.380 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:44.382 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:12:44.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8fc06a830 msgr2=0x7ff8fc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8fc06a830 0x7ff8fc06cce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff9140011d0 tx=0x7ff914009040 comp rx=0 tx=0).stop 2023-10-11T06:12:44.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff918159920 msgr2=0x7ff9181570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff918159920 0x7ff9181570e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff90800f500 tx=0x7ff90800f9c0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 shutdown_connections 2023-10-11T06:12:44.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8fc06a830 0x7ff8fc06cce0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff91815b070 0x7ff9180c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff918159920 0x7ff9181570e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 --2- 172.21.15.84:0/1657491578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904004d00 0x7ff918156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 >> 172.21.15.84:0/1657491578 conn(0x7ff918069c70 msgr2=0x7ff9180c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:44.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.380+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 shutdown_connections 2023-10-11T06:12:44.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.381+0000 7ff91f4e7640 1 -- 172.21.15.84:0/1657491578 wait complete. 2023-10-11T06:12:44.398 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-10-11T06:12:42.799635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219912,"kb_used_data":4296,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754808,"statfs":{"total":773094113280,"available":772868923392,"internally_reserved":0,"allocated":4399104,"data_stored":2472296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.294174"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039012+0000","last_change":"2023-10-11T06:12:40.033024+0000","last_active":"2023-10-11T06:12:40.039012+0000","last_peered":"2023-10-11T06:12:40.039012+0000","last_clean":"2023-10-11T06:12:40.039012+0000","last_became_active":"2023-10-11T06:12:40.032747+0000","last_became_peered":"2023-10-11T06:12:40.032747+0000","last_unstale":"2023-10-11T06:12:40.039012+0000","last_undegraded":"2023-10-11T06:12:40.039012+0000","last_fullsized":"2023-10-11T06:12:40.039012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:54:42.612535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.027138+0000","last_change":"2023-10-11T06:12:40.021323+0000","last_active":"2023-10-11T06:12:40.027138+0000","last_peered":"2023-10-11T06:12:40.027138+0000","last_clean":"2023-10-11T06:12:40.027138+0000","last_became_active":"2023-10-11T06:12:40.020825+0000","last_became_peered":"2023-10-11T06:12:40.020825+0000","last_unstale":"2023-10-11T06:12:40.027138+0000","last_undegraded":"2023-10-11T06:12:40.027138+0000","last_fullsized":"2023-10-11T06:12:40.027138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:39.940351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042556+0000","last_change":"2023-10-11T06:12:40.039917+0000","last_active":"2023-10-11T06:12:40.042556+0000","last_peered":"2023-10-11T06:12:40.042556+0000","last_clean":"2023-10-11T06:12:40.042556+0000","last_became_active":"2023-10-11T06:12:40.038852+0000","last_became_peered":"2023-10-11T06:12:40.038852+0000","last_unstale":"2023-10-11T06:12:40.042556+0000","last_undegraded":"2023-10-11T06:12:40.042556+0000","last_fullsized":"2023-10-11T06:12:40.042556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:20:04.064920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036105+0000","last_change":"2023-10-11T06:12:40.030013+0000","last_active":"2023-10-11T06:12:40.036105+0000","last_peered":"2023-10-11T06:12:40.036105+0000","last_clean":"2023-10-11T06:12:40.036105+0000","last_became_active":"2023-10-11T06:12:40.029604+0000","last_became_peered":"2023-10-11T06:12:40.029604+0000","last_unstale":"2023-10-11T06:12:40.036105+0000","last_undegraded":"2023-10-11T06:12:40.036105+0000","last_fullsized":"2023-10-11T06:12:40.036105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:03.888918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036807+0000","last_change":"2023-10-11T06:12:40.030973+0000","last_active":"2023-10-11T06:12:40.036807+0000","last_peered":"2023-10-11T06:12:40.036807+0000","last_clean":"2023-10-11T06:12:40.036807+0000","last_became_active":"2023-10-11T06:12:40.030645+0000","last_became_peered":"2023-10-11T06:12:40.030645+0000","last_unstale":"2023-10-11T06:12:40.036807+0000","last_undegraded":"2023-10-11T06:12:40.036807+0000","last_fullsized":"2023-10-11T06:12:40.036807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:25:39.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.032603+0000","last_change":"2023-10-11T06:12:40.026094+0000","last_active":"2023-10-11T06:12:40.032603+0000","last_peered":"2023-10-11T06:12:40.032603+0000","last_clean":"2023-10-11T06:12:40.032603+0000","last_became_active":"2023-10-11T06:12:40.025612+0000","last_became_peered":"2023-10-11T06:12:40.025612+0000","last_unstale":"2023-10-11T06:12:40.032603+0000","last_undegraded":"2023-10-11T06:12:40.032603+0000","last_fullsized":"2023-10-11T06:12:40.032603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:23:59.597403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041827+0000","last_change":"2023-10-11T06:12:40.036422+0000","last_active":"2023-10-11T06:12:40.041827+0000","last_peered":"2023-10-11T06:12:40.041827+0000","last_clean":"2023-10-11T06:12:40.041827+0000","last_became_active":"2023-10-11T06:12:40.032519+0000","last_became_peered":"2023-10-11T06:12:40.032519+0000","last_unstale":"2023-10-11T06:12:40.041827+0000","last_undegraded":"2023-10-11T06:12:40.041827+0000","last_fullsized":"2023-10-11T06:12:40.041827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:39:41.279503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042573+0000","last_change":"2023-10-11T06:12:40.040112+0000","last_active":"2023-10-11T06:12:40.042573+0000","last_peered":"2023-10-11T06:12:40.042573+0000","last_clean":"2023-10-11T06:12:40.042573+0000","last_became_active":"2023-10-11T06:12:40.039582+0000","last_became_peered":"2023-10-11T06:12:40.039582+0000","last_unstale":"2023-10-11T06:12:40.042573+0000","last_undegraded":"2023-10-11T06:12:40.042573+0000","last_fullsized":"2023-10-11T06:12:40.042573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:39.281451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042977+0000","last_change":"2023-10-11T06:12:40.039844+0000","last_active":"2023-10-11T06:12:40.042977+0000","last_peered":"2023-10-11T06:12:40.042977+0000","last_clean":"2023-10-11T06:12:40.042977+0000","last_became_active":"2023-10-11T06:12:40.039317+0000","last_became_peered":"2023-10-11T06:12:40.039317+0000","last_unstale":"2023-10-11T06:12:40.042977+0000","last_undegraded":"2023-10-11T06:12:40.042977+0000","last_fullsized":"2023-10-11T06:12:40.042977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:12:30.177239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.052885+0000","last_change":"2023-10-11T06:12:40.047527+0000","last_active":"2023-10-11T06:12:40.052885+0000","last_peered":"2023-10-11T06:12:40.052885+0000","last_clean":"2023-10-11T06:12:40.052885+0000","last_became_active":"2023-10-11T06:12:40.047237+0000","last_became_peered":"2023-10-11T06:12:40.047237+0000","last_unstale":"2023-10-11T06:12:40.052885+0000","last_undegraded":"2023-10-11T06:12:40.052885+0000","last_fullsized":"2023-10-11T06:12:40.052885+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:39.281451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044427+0000","last_change":"2023-10-11T06:12:40.040062+0000","last_active":"2023-10-11T06:12:40.044427+0000","last_peered":"2023-10-11T06:12:40.044427+0000","last_clean":"2023-10-11T06:12:40.044427+0000","last_became_active":"2023-10-11T06:12:40.039747+0000","last_became_peered":"2023-10-11T06:12:40.039747+0000","last_unstale":"2023-10-11T06:12:40.044427+0000","last_undegraded":"2023-10-11T06:12:40.044427+0000","last_fullsized":"2023-10-11T06:12:40.044427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:38.377463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.033588+0000","last_change":"2023-10-11T06:12:40.029578+0000","last_active":"2023-10-11T06:12:40.033588+0000","last_peered":"2023-10-11T06:12:40.033588+0000","last_clean":"2023-10-11T06:12:40.033588+0000","last_became_active":"2023-10-11T06:12:40.029312+0000","last_became_peered":"2023-10-11T06:12:40.029312+0000","last_unstale":"2023-10-11T06:12:40.033588+0000","last_undegraded":"2023-10-11T06:12:40.033588+0000","last_fullsized":"2023-10-11T06:12:40.033588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:34:05.011734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.052917+0000","last_change":"2023-10-11T06:12:40.047603+0000","last_active":"2023-10-11T06:12:40.052917+0000","last_peered":"2023-10-11T06:12:40.052917+0000","last_clean":"2023-10-11T06:12:40.052917+0000","last_became_active":"2023-10-11T06:12:40.047238+0000","last_became_peered":"2023-10-11T06:12:40.047238+0000","last_unstale":"2023-10-11T06:12:40.052917+0000","last_undegraded":"2023-10-11T06:12:40.052917+0000","last_fullsized":"2023-10-11T06:12:40.052917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.028716+0000","last_change":"2023-10-11T06:12:40.024432+0000","last_active":"2023-10-11T06:12:40.028716+0000","last_peered":"2023-10-11T06:12:40.028716+0000","last_clean":"2023-10-11T06:12:40.028716+0000","last_became_active":"2023-10-11T06:12:40.024199+0000","last_became_peered":"2023-10-11T06:12:40.024199+0000","last_unstale":"2023-10-11T06:12:40.028716+0000","last_undegraded":"2023-10-11T06:12:40.028716+0000","last_fullsized":"2023-10-11T06:12:40.028716+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:09:17.938055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.038749+0000","last_change":"2023-10-11T06:12:40.032742+0000","last_active":"2023-10-11T06:12:40.038749+0000","last_peered":"2023-10-11T06:12:40.038749+0000","last_clean":"2023-10-11T06:12:40.038749+0000","last_became_active":"2023-10-11T06:12:40.032471+0000","last_became_peered":"2023-10-11T06:12:40.032471+0000","last_unstale":"2023-10-11T06:12:40.038749+0000","last_undegraded":"2023-10-11T06:12:40.038749+0000","last_fullsized":"2023-10-11T06:12:40.038749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:38.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049310+0000","last_change":"2023-10-11T06:12:40.046216+0000","last_active":"2023-10-11T06:12:40.049310+0000","last_peered":"2023-10-11T06:12:40.049310+0000","last_clean":"2023-10-11T06:12:40.049310+0000","last_became_active":"2023-10-11T06:12:40.046006+0000","last_became_peered":"2023-10-11T06:12:40.046006+0000","last_unstale":"2023-10-11T06:12:40.049310+0000","last_undegraded":"2023-10-11T06:12:40.049310+0000","last_fullsized":"2023-10-11T06:12:40.049310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048999+0000","last_change":"2023-10-11T06:12:40.045218+0000","last_active":"2023-10-11T06:12:40.048999+0000","last_peered":"2023-10-11T06:12:40.048999+0000","last_clean":"2023-10-11T06:12:40.048999+0000","last_became_active":"2023-10-11T06:12:40.044718+0000","last_became_peered":"2023-10-11T06:12:40.044718+0000","last_unstale":"2023-10-11T06:12:40.048999+0000","last_undegraded":"2023-10-11T06:12:40.048999+0000","last_fullsized":"2023-10-11T06:12:40.048999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:54:42.612535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.029410+0000","last_change":"2023-10-11T06:12:40.026253+0000","last_active":"2023-10-11T06:12:40.029410+0000","last_peered":"2023-10-11T06:12:40.029410+0000","last_clean":"2023-10-11T06:12:40.029410+0000","last_became_active":"2023-10-11T06:12:40.025962+0000","last_became_peered":"2023-10-11T06:12:40.025962+0000","last_unstale":"2023-10-11T06:12:40.029410+0000","last_undegraded":"2023-10-11T06:12:40.029410+0000","last_fullsized":"2023-10-11T06:12:40.029410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:47.514739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039322+0000","last_change":"2023-10-11T06:12:40.034197+0000","last_active":"2023-10-11T06:12:40.039322+0000","last_peered":"2023-10-11T06:12:40.039322+0000","last_clean":"2023-10-11T06:12:40.039322+0000","last_became_active":"2023-10-11T06:12:40.033933+0000","last_became_peered":"2023-10-11T06:12:40.033933+0000","last_unstale":"2023-10-11T06:12:40.039322+0000","last_undegraded":"2023-10-11T06:12:40.039322+0000","last_fullsized":"2023-10-11T06:12:40.039322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:02.844683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.029441+0000","last_change":"2023-10-11T06:12:40.026331+0000","last_active":"2023-10-11T06:12:40.029441+0000","last_peered":"2023-10-11T06:12:40.029441+0000","last_clean":"2023-10-11T06:12:40.029441+0000","last_became_active":"2023-10-11T06:12:40.026040+0000","last_became_peered":"2023-10-11T06:12:40.026040+0000","last_unstale":"2023-10-11T06:12:40.029441+0000","last_undegraded":"2023-10-11T06:12:40.029441+0000","last_fullsized":"2023-10-11T06:12:40.029441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:02.844683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044893+0000","last_change":"2023-10-11T06:12:40.040250+0000","last_active":"2023-10-11T06:12:40.044893+0000","last_peered":"2023-10-11T06:12:40.044893+0000","last_clean":"2023-10-11T06:12:40.044893+0000","last_became_active":"2023-10-11T06:12:40.039987+0000","last_became_peered":"2023-10-11T06:12:40.039987+0000","last_unstale":"2023-10-11T06:12:40.044893+0000","last_undegraded":"2023-10-11T06:12:40.044893+0000","last_fullsized":"2023-10-11T06:12:40.044893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:59:40.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034263+0000","last_change":"2023-10-11T06:12:40.028636+0000","last_active":"2023-10-11T06:12:40.034263+0000","last_peered":"2023-10-11T06:12:40.034263+0000","last_clean":"2023-10-11T06:12:40.034263+0000","last_became_active":"2023-10-11T06:12:40.028377+0000","last_became_peered":"2023-10-11T06:12:40.028377+0000","last_unstale":"2023-10-11T06:12:40.034263+0000","last_undegraded":"2023-10-11T06:12:40.034263+0000","last_fullsized":"2023-10-11T06:12:40.034263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:39.281451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.040886+0000","last_change":"2023-10-11T06:12:40.034169+0000","last_active":"2023-10-11T06:12:40.040886+0000","last_peered":"2023-10-11T06:12:40.040886+0000","last_clean":"2023-10-11T06:12:40.040886+0000","last_became_active":"2023-10-11T06:12:40.033921+0000","last_became_peered":"2023-10-11T06:12:40.033921+0000","last_unstale":"2023-10-11T06:12:40.040886+0000","last_undegraded":"2023-10-11T06:12:40.040886+0000","last_fullsized":"2023-10-11T06:12:40.040886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:57:55.747144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042602+0000","last_change":"2023-10-11T06:12:40.040719+0000","last_active":"2023-10-11T06:12:40.042602+0000","last_peered":"2023-10-11T06:12:40.042602+0000","last_clean":"2023-10-11T06:12:40.042602+0000","last_became_active":"2023-10-11T06:12:40.040439+0000","last_became_peered":"2023-10-11T06:12:40.040439+0000","last_unstale":"2023-10-11T06:12:40.042602+0000","last_undegraded":"2023-10-11T06:12:40.042602+0000","last_fullsized":"2023-10-11T06:12:40.042602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:47.514739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.030615+0000","last_change":"2023-10-11T06:12:40.028358+0000","last_active":"2023-10-11T06:12:40.030615+0000","last_peered":"2023-10-11T06:12:40.030615+0000","last_clean":"2023-10-11T06:12:40.030615+0000","last_became_active":"2023-10-11T06:12:40.028122+0000","last_became_peered":"2023-10-11T06:12:40.028122+0000","last_unstale":"2023-10-11T06:12:40.030615+0000","last_undegraded":"2023-10-11T06:12:40.030615+0000","last_fullsized":"2023-10-11T06:12:40.030615+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:49.725336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.047686+0000","last_change":"2023-10-11T06:12:40.043689+0000","last_active":"2023-10-11T06:12:40.047686+0000","last_peered":"2023-10-11T06:12:40.047686+0000","last_clean":"2023-10-11T06:12:40.047686+0000","last_became_active":"2023-10-11T06:12:40.043108+0000","last_became_peered":"2023-10-11T06:12:40.043108+0000","last_unstale":"2023-10-11T06:12:40.047686+0000","last_undegraded":"2023-10-11T06:12:40.047686+0000","last_fullsized":"2023-10-11T06:12:40.047686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:37:44.742549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034373+0000","last_change":"2023-10-11T06:12:40.032026+0000","last_active":"2023-10-11T06:12:40.034373+0000","last_peered":"2023-10-11T06:12:40.034373+0000","last_clean":"2023-10-11T06:12:40.034373+0000","last_became_active":"2023-10-11T06:12:40.029679+0000","last_became_peered":"2023-10-11T06:12:40.029679+0000","last_unstale":"2023-10-11T06:12:40.034373+0000","last_undegraded":"2023-10-11T06:12:40.034373+0000","last_fullsized":"2023-10-11T06:12:40.034373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:11:41.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044408+0000","last_change":"2023-10-11T06:12:40.039392+0000","last_active":"2023-10-11T06:12:40.044408+0000","last_peered":"2023-10-11T06:12:40.044408+0000","last_clean":"2023-10-11T06:12:40.044408+0000","last_became_active":"2023-10-11T06:12:40.039137+0000","last_became_peered":"2023-10-11T06:12:40.039137+0000","last_unstale":"2023-10-11T06:12:40.044408+0000","last_undegraded":"2023-10-11T06:12:40.044408+0000","last_fullsized":"2023-10-11T06:12:40.044408+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:14.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.038312+0000","last_change":"2023-10-11T06:12:40.034551+0000","last_active":"2023-10-11T06:12:40.038312+0000","last_peered":"2023-10-11T06:12:40.038312+0000","last_clean":"2023-10-11T06:12:40.038312+0000","last_became_active":"2023-10-11T06:12:40.034297+0000","last_became_peered":"2023-10-11T06:12:40.034297+0000","last_unstale":"2023-10-11T06:12:40.038312+0000","last_undegraded":"2023-10-11T06:12:40.038312+0000","last_fullsized":"2023-10-11T06:12:40.038312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:18.159878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034403+0000","last_change":"2023-10-11T06:12:40.031746+0000","last_active":"2023-10-11T06:12:40.034403+0000","last_peered":"2023-10-11T06:12:40.034403+0000","last_clean":"2023-10-11T06:12:40.034403+0000","last_became_active":"2023-10-11T06:12:40.031477+0000","last_became_peered":"2023-10-11T06:12:40.031477+0000","last_unstale":"2023-10-11T06:12:40.034403+0000","last_undegraded":"2023-10-11T06:12:40.034403+0000","last_fullsized":"2023-10-11T06:12:40.034403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:28:43.683955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048241+0000","last_change":"2023-10-11T06:12:40.041007+0000","last_active":"2023-10-11T06:12:40.048241+0000","last_peered":"2023-10-11T06:12:40.048241+0000","last_clean":"2023-10-11T06:12:40.048241+0000","last_became_active":"2023-10-11T06:12:40.040315+0000","last_became_peered":"2023-10-11T06:12:40.040315+0000","last_unstale":"2023-10-11T06:12:40.048241+0000","last_undegraded":"2023-10-11T06:12:40.048241+0000","last_fullsized":"2023-10-11T06:12:40.048241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:23:18.593586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034602+0000","last_change":"2023-10-11T06:12:40.031001+0000","last_active":"2023-10-11T06:12:40.034602+0000","last_peered":"2023-10-11T06:12:40.034602+0000","last_clean":"2023-10-11T06:12:40.034602+0000","last_became_active":"2023-10-11T06:12:40.030659+0000","last_became_peered":"2023-10-11T06:12:40.030659+0000","last_unstale":"2023-10-11T06:12:40.034602+0000","last_undegraded":"2023-10-11T06:12:40.034602+0000","last_fullsized":"2023-10-11T06:12:40.034602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:21:29.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048486+0000","last_change":"2023-10-11T06:12:40.042664+0000","last_active":"2023-10-11T06:12:40.048486+0000","last_peered":"2023-10-11T06:12:40.048486+0000","last_clean":"2023-10-11T06:12:40.048486+0000","last_became_active":"2023-10-11T06:12:40.042347+0000","last_became_peered":"2023-10-11T06:12:40.042347+0000","last_unstale":"2023-10-11T06:12:40.048486+0000","last_undegraded":"2023-10-11T06:12:40.048486+0000","last_fullsized":"2023-10-11T06:12:40.048486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:12.851212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039603+0000","last_change":"2023-10-11T06:12:39.551526+0000","last_active":"2023-10-11T06:12:40.039603+0000","last_peered":"2023-10-11T06:12:40.039603+0000","last_clean":"2023-10-11T06:12:40.039603+0000","last_became_active":"2023-10-11T06:12:39.551259+0000","last_became_peered":"2023-10-11T06:12:39.551259+0000","last_unstale":"2023-10-11T06:12:40.039603+0000","last_undegraded":"2023-10-11T06:12:40.039603+0000","last_fullsized":"2023-10-11T06:12:40.039603+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:35:59.834474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044712+0000","last_change":"2023-10-11T06:12:40.040728+0000","last_active":"2023-10-11T06:12:40.044712+0000","last_peered":"2023-10-11T06:12:40.044712+0000","last_clean":"2023-10-11T06:12:40.044712+0000","last_became_active":"2023-10-11T06:12:40.040455+0000","last_became_peered":"2023-10-11T06:12:40.040455+0000","last_unstale":"2023-10-11T06:12:40.044712+0000","last_undegraded":"2023-10-11T06:12:40.044712+0000","last_fullsized":"2023-10-11T06:12:40.044712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:57:53.399147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.047210+0000","last_change":"2023-10-11T06:12:39.548879+0000","last_active":"2023-10-11T06:12:40.047210+0000","last_peered":"2023-10-11T06:12:40.047210+0000","last_clean":"2023-10-11T06:12:40.047210+0000","last_became_active":"2023-10-11T06:12:39.548594+0000","last_became_peered":"2023-10-11T06:12:39.548594+0000","last_unstale":"2023-10-11T06:12:40.047210+0000","last_undegraded":"2023-10-11T06:12:40.047210+0000","last_fullsized":"2023-10-11T06:12:40.047210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:28:42.668410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.028422+0000","last_change":"2023-10-11T06:12:40.024267+0000","last_active":"2023-10-11T06:12:40.028422+0000","last_peered":"2023-10-11T06:12:40.028422+0000","last_clean":"2023-10-11T06:12:40.028422+0000","last_became_active":"2023-10-11T06:12:40.023987+0000","last_became_peered":"2023-10-11T06:12:40.023987+0000","last_unstale":"2023-10-11T06:12:40.028422+0000","last_undegraded":"2023-10-11T06:12:40.028422+0000","last_fullsized":"2023-10-11T06:12:40.028422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:59:40.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035031+0000","last_change":"2023-10-11T06:12:39.551964+0000","last_active":"2023-10-11T06:12:40.035031+0000","last_peered":"2023-10-11T06:12:40.035031+0000","last_clean":"2023-10-11T06:12:40.035031+0000","last_became_active":"2023-10-11T06:12:39.551658+0000","last_became_peered":"2023-10-11T06:12:39.551658+0000","last_unstale":"2023-10-11T06:12:40.035031+0000","last_undegraded":"2023-10-11T06:12:40.035031+0000","last_fullsized":"2023-10-11T06:12:40.035031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:08:03.104559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.029632+0000","last_change":"2023-10-11T06:12:40.026295+0000","last_active":"2023-10-11T06:12:40.029632+0000","last_peered":"2023-10-11T06:12:40.029632+0000","last_clean":"2023-10-11T06:12:40.029632+0000","last_became_active":"2023-10-11T06:12:40.026039+0000","last_became_peered":"2023-10-11T06:12:40.026039+0000","last_unstale":"2023-10-11T06:12:40.029632+0000","last_undegraded":"2023-10-11T06:12:40.029632+0000","last_fullsized":"2023-10-11T06:12:40.029632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:57:55.747144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045382+0000","last_change":"2023-10-11T06:12:39.552520+0000","last_active":"2023-10-11T06:12:40.045382+0000","last_peered":"2023-10-11T06:12:40.045382+0000","last_clean":"2023-10-11T06:12:40.045382+0000","last_became_active":"2023-10-11T06:12:39.552149+0000","last_became_peered":"2023-10-11T06:12:39.552149+0000","last_unstale":"2023-10-11T06:12:40.045382+0000","last_undegraded":"2023-10-11T06:12:40.045382+0000","last_fullsized":"2023-10-11T06:12:40.045382+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:37:43.727004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041819+0000","last_change":"2023-10-11T06:12:40.035838+0000","last_active":"2023-10-11T06:12:40.041819+0000","last_peered":"2023-10-11T06:12:40.041819+0000","last_clean":"2023-10-11T06:12:40.041819+0000","last_became_active":"2023-10-11T06:12:40.035577+0000","last_became_peered":"2023-10-11T06:12:40.035577+0000","last_unstale":"2023-10-11T06:12:40.041819+0000","last_undegraded":"2023-10-11T06:12:40.041819+0000","last_fullsized":"2023-10-11T06:12:40.041819+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:00.886095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036203+0000","last_change":"2023-10-11T06:12:39.548700+0000","last_active":"2023-10-11T06:12:40.036203+0000","last_peered":"2023-10-11T06:12:40.036203+0000","last_clean":"2023-10-11T06:12:40.036203+0000","last_became_active":"2023-10-11T06:12:39.547969+0000","last_became_peered":"2023-10-11T06:12:39.547969+0000","last_unstale":"2023-10-11T06:12:40.036203+0000","last_undegraded":"2023-10-11T06:12:40.036203+0000","last_fullsized":"2023-10-11T06:12:40.036203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:23:58.581858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048697+0000","last_change":"2023-10-11T06:12:40.041198+0000","last_active":"2023-10-11T06:12:40.048697+0000","last_peered":"2023-10-11T06:12:40.048697+0000","last_clean":"2023-10-11T06:12:40.048697+0000","last_became_active":"2023-10-11T06:12:40.040759+0000","last_became_peered":"2023-10-11T06:12:40.040759+0000","last_unstale":"2023-10-11T06:12:40.048697+0000","last_undegraded":"2023-10-11T06:12:40.048697+0000","last_fullsized":"2023-10-11T06:12:40.048697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:09:17.938055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.046191+0000","last_change":"2023-10-11T06:12:39.551972+0000","last_active":"2023-10-11T06:12:40.046191+0000","last_peered":"2023-10-11T06:12:40.046191+0000","last_clean":"2023-10-11T06:12:40.046191+0000","last_became_active":"2023-10-11T06:12:39.551591+0000","last_became_peered":"2023-10-11T06:12:39.551591+0000","last_unstale":"2023-10-11T06:12:40.046191+0000","last_undegraded":"2023-10-11T06:12:40.046191+0000","last_fullsized":"2023-10-11T06:12:40.046191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:01.829138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.043797+0000","last_change":"2023-10-11T06:12:40.042009+0000","last_active":"2023-10-11T06:12:40.043797+0000","last_peered":"2023-10-11T06:12:40.043797+0000","last_clean":"2023-10-11T06:12:40.043797+0000","last_became_active":"2023-10-11T06:12:40.041459+0000","last_became_peered":"2023-10-11T06:12:40.041459+0000","last_unstale":"2023-10-11T06:12:40.043797+0000","last_undegraded":"2023-10-11T06:12:40.043797+0000","last_fullsized":"2023-10-11T06:12:40.043797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:59.927845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.015446+0000","last_change":"2023-10-11T06:12:39.561706+0000","last_active":"2023-10-11T06:12:40.015446+0000","last_peered":"2023-10-11T06:12:40.015446+0000","last_clean":"2023-10-11T06:12:40.015446+0000","last_became_active":"2023-10-11T06:12:39.561447+0000","last_became_peered":"2023-10-11T06:12:39.561447+0000","last_unstale":"2023-10-11T06:12:40.015446+0000","last_undegraded":"2023-10-11T06:12:40.015446+0000","last_fullsized":"2023-10-11T06:12:40.015446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:11:40.142203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041741+0000","last_change":"2023-10-11T06:12:40.038003+0000","last_active":"2023-10-11T06:12:40.041741+0000","last_peered":"2023-10-11T06:12:40.041741+0000","last_clean":"2023-10-11T06:12:40.041741+0000","last_became_active":"2023-10-11T06:12:40.037727+0000","last_became_peered":"2023-10-11T06:12:40.037727+0000","last_unstale":"2023-10-11T06:12:40.041741+0000","last_undegraded":"2023-10-11T06:12:40.041741+0000","last_fullsized":"2023-10-11T06:12:40.041741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:47.911862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.013918+0000","last_change":"2023-10-11T06:12:39.554563+0000","last_active":"2023-10-11T06:12:40.013918+0000","last_peered":"2023-10-11T06:12:40.013918+0000","last_clean":"2023-10-11T06:12:40.013918+0000","last_became_active":"2023-10-11T06:12:39.554191+0000","last_became_peered":"2023-10-11T06:12:39.554191+0000","last_unstale":"2023-10-11T06:12:40.013918+0000","last_undegraded":"2023-10-11T06:12:40.013918+0000","last_fullsized":"2023-10-11T06:12:40.013918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:34:03.996188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036125+0000","last_change":"2023-10-11T06:12:40.030876+0000","last_active":"2023-10-11T06:12:40.036125+0000","last_peered":"2023-10-11T06:12:40.036125+0000","last_clean":"2023-10-11T06:12:40.036125+0000","last_became_active":"2023-10-11T06:12:40.030565+0000","last_became_peered":"2023-10-11T06:12:40.030565+0000","last_unstale":"2023-10-11T06:12:40.036125+0000","last_undegraded":"2023-10-11T06:12:40.036125+0000","last_fullsized":"2023-10-11T06:12:40.036125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:15:23.236778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.011580+0000","last_change":"2023-10-11T06:12:39.543157+0000","last_active":"2023-10-11T06:12:40.011580+0000","last_peered":"2023-10-11T06:12:40.011580+0000","last_clean":"2023-10-11T06:12:40.011580+0000","last_became_active":"2023-10-11T06:12:39.542683+0000","last_became_peered":"2023-10-11T06:12:39.542683+0000","last_unstale":"2023-10-11T06:12:40.011580+0000","last_undegraded":"2023-10-11T06:12:40.011580+0000","last_fullsized":"2023-10-11T06:12:40.011580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:15:22.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049280+0000","last_change":"2023-10-11T06:12:40.046079+0000","last_active":"2023-10-11T06:12:40.049280+0000","last_peered":"2023-10-11T06:12:40.049280+0000","last_clean":"2023-10-11T06:12:40.049280+0000","last_became_active":"2023-10-11T06:12:40.045787+0000","last_became_peered":"2023-10-11T06:12:40.045787+0000","last_unstale":"2023-10-11T06:12:40.049280+0000","last_undegraded":"2023-10-11T06:12:40.049280+0000","last_fullsized":"2023-10-11T06:12:40.049280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:38.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.027651+0000","last_change":"2023-10-11T06:12:39.557411+0000","last_active":"2023-10-11T06:12:40.027651+0000","last_peered":"2023-10-11T06:12:40.027651+0000","last_clean":"2023-10-11T06:12:40.027651+0000","last_became_active":"2023-10-11T06:12:39.557119+0000","last_became_peered":"2023-10-11T06:12:39.557119+0000","last_unstale":"2023-10-11T06:12:40.027651+0000","last_undegraded":"2023-10-11T06:12:40.027651+0000","last_fullsized":"2023-10-11T06:12:40.027651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:15:22.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048249+0000","last_change":"2023-10-11T06:12:40.047564+0000","last_active":"2023-10-11T06:12:40.048249+0000","last_peered":"2023-10-11T06:12:40.048249+0000","last_clean":"2023-10-11T06:12:40.048249+0000","last_became_active":"2023-10-11T06:12:40.047250+0000","last_became_peered":"2023-10-11T06:12:40.047250+0000","last_unstale":"2023-10-11T06:12:40.048249+0000","last_undegraded":"2023-10-11T06:12:40.048249+0000","last_fullsized":"2023-10-11T06:12:40.048249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:53.963222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045796+0000","last_change":"2023-10-11T06:12:39.557044+0000","last_active":"2023-10-11T06:12:40.045796+0000","last_peered":"2023-10-11T06:12:40.045796+0000","last_clean":"2023-10-11T06:12:40.045796+0000","last_became_active":"2023-10-11T06:12:39.556558+0000","last_became_peered":"2023-10-11T06:12:39.556558+0000","last_unstale":"2023-10-11T06:12:40.045796+0000","last_undegraded":"2023-10-11T06:12:40.045796+0000","last_fullsized":"2023-10-11T06:12:40.045796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:21:28.975805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034232+0000","last_change":"2023-10-11T06:12:40.029136+0000","last_active":"2023-10-11T06:12:40.034232+0000","last_peered":"2023-10-11T06:12:40.034232+0000","last_clean":"2023-10-11T06:12:40.034232+0000","last_became_active":"2023-10-11T06:12:40.028800+0000","last_became_peered":"2023-10-11T06:12:40.028800+0000","last_unstale":"2023-10-11T06:12:40.034232+0000","last_undegraded":"2023-10-11T06:12:40.034232+0000","last_fullsized":"2023-10-11T06:12:40.034232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.053493+0000","last_change":"2023-10-11T06:12:39.544686+0000","last_active":"2023-10-11T06:12:40.053493+0000","last_peered":"2023-10-11T06:12:40.053493+0000","last_clean":"2023-10-11T06:12:40.053493+0000","last_became_active":"2023-10-11T06:12:39.544374+0000","last_became_peered":"2023-10-11T06:12:39.544374+0000","last_unstale":"2023-10-11T06:12:40.053493+0000","last_undegraded":"2023-10-11T06:12:40.053493+0000","last_fullsized":"2023-10-11T06:12:40.053493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:46.499194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042584+0000","last_change":"2023-10-11T06:12:40.036601+0000","last_active":"2023-10-11T06:12:40.042584+0000","last_peered":"2023-10-11T06:12:40.042584+0000","last_clean":"2023-10-11T06:12:40.042584+0000","last_became_active":"2023-10-11T06:12:40.036362+0000","last_became_peered":"2023-10-11T06:12:40.036362+0000","last_unstale":"2023-10-11T06:12:40.042584+0000","last_undegraded":"2023-10-11T06:12:40.042584+0000","last_fullsized":"2023-10-11T06:12:40.042584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:02.744270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.033051+0000","last_change":"2023-10-11T06:12:39.542882+0000","last_active":"2023-10-11T06:12:40.033051+0000","last_peered":"2023-10-11T06:12:40.033051+0000","last_clean":"2023-10-11T06:12:40.033051+0000","last_became_active":"2023-10-11T06:12:39.542603+0000","last_became_peered":"2023-10-11T06:12:39.542603+0000","last_unstale":"2023-10-11T06:12:40.033051+0000","last_undegraded":"2023-10-11T06:12:40.033051+0000","last_fullsized":"2023-10-11T06:12:40.033051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:02.873373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035537+0000","last_change":"2023-10-11T06:12:40.032457+0000","last_active":"2023-10-11T06:12:40.035537+0000","last_peered":"2023-10-11T06:12:40.035537+0000","last_clean":"2023-10-11T06:12:40.035537+0000","last_became_active":"2023-10-11T06:12:40.031951+0000","last_became_peered":"2023-10-11T06:12:40.031951+0000","last_unstale":"2023-10-11T06:12:40.035537+0000","last_undegraded":"2023-10-11T06:12:40.035537+0000","last_fullsized":"2023-10-11T06:12:40.035537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:12.851212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.015216+0000","last_change":"2023-10-11T06:12:39.561583+0000","last_active":"2023-10-11T06:12:40.015216+0000","last_peered":"2023-10-11T06:12:40.015216+0000","last_clean":"2023-10-11T06:12:40.015216+0000","last_became_active":"2023-10-11T06:12:39.561362+0000","last_became_peered":"2023-10-11T06:12:39.561362+0000","last_unstale":"2023-10-11T06:12:40.015216+0000","last_undegraded":"2023-10-11T06:12:40.015216+0000","last_fullsized":"2023-10-11T06:12:40.015216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:31:04.013960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035951+0000","last_change":"2023-10-11T06:12:40.030734+0000","last_active":"2023-10-11T06:12:40.035951+0000","last_peered":"2023-10-11T06:12:40.035951+0000","last_clean":"2023-10-11T06:12:40.035951+0000","last_became_active":"2023-10-11T06:12:40.030476+0000","last_became_peered":"2023-10-11T06:12:40.030476+0000","last_unstale":"2023-10-11T06:12:40.035951+0000","last_undegraded":"2023-10-11T06:12:40.035951+0000","last_fullsized":"2023-10-11T06:12:40.035951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:20:04.064920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.018975+0000","last_change":"2023-10-11T06:12:39.554445+0000","last_active":"2023-10-11T06:12:40.018975+0000","last_peered":"2023-10-11T06:12:40.018975+0000","last_clean":"2023-10-11T06:12:40.018975+0000","last_became_active":"2023-10-11T06:12:39.553941+0000","last_became_peered":"2023-10-11T06:12:39.553941+0000","last_unstale":"2023-10-11T06:12:40.018975+0000","last_undegraded":"2023-10-11T06:12:40.018975+0000","last_fullsized":"2023-10-11T06:12:40.018975+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:03:30.855024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.037453+0000","last_change":"2023-10-11T06:12:40.032116+0000","last_active":"2023-10-11T06:12:40.037453+0000","last_peered":"2023-10-11T06:12:40.037453+0000","last_clean":"2023-10-11T06:12:40.037453+0000","last_became_active":"2023-10-11T06:12:40.031858+0000","last_became_peered":"2023-10-11T06:12:40.031858+0000","last_unstale":"2023-10-11T06:12:40.037453+0000","last_undegraded":"2023-10-11T06:12:40.037453+0000","last_fullsized":"2023-10-11T06:12:40.037453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:38.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.050598+0000","last_change":"2023-10-11T06:12:39.551905+0000","last_active":"2023-10-11T06:12:40.050598+0000","last_peered":"2023-10-11T06:12:40.050598+0000","last_clean":"2023-10-11T06:12:40.050598+0000","last_became_active":"2023-10-11T06:12:39.551604+0000","last_became_peered":"2023-10-11T06:12:39.551604+0000","last_unstale":"2023-10-11T06:12:40.050598+0000","last_undegraded":"2023-10-11T06:12:40.050598+0000","last_fullsized":"2023-10-11T06:12:40.050598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:48.709791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.032766+0000","last_change":"2023-10-11T06:12:40.029691+0000","last_active":"2023-10-11T06:12:40.032766+0000","last_peered":"2023-10-11T06:12:40.032766+0000","last_clean":"2023-10-11T06:12:40.032766+0000","last_became_active":"2023-10-11T06:12:40.029412+0000","last_became_peered":"2023-10-11T06:12:40.029412+0000","last_unstale":"2023-10-11T06:12:40.032766+0000","last_undegraded":"2023-10-11T06:12:40.032766+0000","last_fullsized":"2023-10-11T06:12:40.032766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045980+0000","last_change":"2023-10-11T06:12:39.554535+0000","last_active":"2023-10-11T06:12:40.045980+0000","last_peered":"2023-10-11T06:12:40.045980+0000","last_clean":"2023-10-11T06:12:40.045980+0000","last_became_active":"2023-10-11T06:12:39.554254+0000","last_became_peered":"2023-10-11T06:12:39.554254+0000","last_unstale":"2023-10-11T06:12:40.045980+0000","last_undegraded":"2023-10-11T06:12:40.045980+0000","last_fullsized":"2023-10-11T06:12:40.045980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:27:59.870550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048901+0000","last_change":"2023-10-11T06:12:40.042131+0000","last_active":"2023-10-11T06:12:40.048901+0000","last_peered":"2023-10-11T06:12:40.048901+0000","last_clean":"2023-10-11T06:12:40.048901+0000","last_became_active":"2023-10-11T06:12:40.041881+0000","last_became_peered":"2023-10-11T06:12:40.041881+0000","last_unstale":"2023-10-11T06:12:40.048901+0000","last_undegraded":"2023-10-11T06:12:40.048901+0000","last_fullsized":"2023-10-11T06:12:40.048901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:59.927845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049940+0000","last_change":"2023-10-11T06:12:39.560892+0000","last_active":"2023-10-11T06:12:40.049940+0000","last_peered":"2023-10-11T06:12:40.049940+0000","last_clean":"2023-10-11T06:12:40.049940+0000","last_became_active":"2023-10-11T06:12:39.560546+0000","last_became_peered":"2023-10-11T06:12:39.560546+0000","last_unstale":"2023-10-11T06:12:40.049940+0000","last_undegraded":"2023-10-11T06:12:40.049940+0000","last_fullsized":"2023-10-11T06:12:40.049940+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:01.728725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042347+0000","last_change":"2023-10-11T06:12:40.035526+0000","last_active":"2023-10-11T06:12:40.042347+0000","last_peered":"2023-10-11T06:12:40.042347+0000","last_clean":"2023-10-11T06:12:40.042347+0000","last_became_active":"2023-10-11T06:12:40.035264+0000","last_became_peered":"2023-10-11T06:12:40.035264+0000","last_unstale":"2023-10-11T06:12:40.042347+0000","last_undegraded":"2023-10-11T06:12:40.042347+0000","last_fullsized":"2023-10-11T06:12:40.042347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:38.377463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.047959+0000","last_change":"2023-10-11T06:12:39.547719+0000","last_active":"2023-10-11T06:12:40.047959+0000","last_peered":"2023-10-11T06:12:40.047959+0000","last_clean":"2023-10-11T06:12:40.047959+0000","last_became_active":"2023-10-11T06:12:39.547374+0000","last_became_peered":"2023-10-11T06:12:39.547374+0000","last_unstale":"2023-10-11T06:12:40.047959+0000","last_undegraded":"2023-10-11T06:12:40.047959+0000","last_fullsized":"2023-10-11T06:12:40.047959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:31:56.222312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.032187+0000","last_change":"2023-10-11T06:12:40.027222+0000","last_active":"2023-10-11T06:12:40.032187+0000","last_peered":"2023-10-11T06:12:40.032187+0000","last_clean":"2023-10-11T06:12:40.032187+0000","last_became_active":"2023-10-11T06:12:40.026946+0000","last_became_peered":"2023-10-11T06:12:40.026946+0000","last_unstale":"2023-10-11T06:12:40.032187+0000","last_undegraded":"2023-10-11T06:12:40.032187+0000","last_fullsized":"2023-10-11T06:12:40.032187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:49.725336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.016327+0000","last_change":"2023-10-11T06:12:39.554020+0000","last_active":"2023-10-11T06:12:40.016327+0000","last_peered":"2023-10-11T06:12:40.016327+0000","last_clean":"2023-10-11T06:12:40.016327+0000","last_became_active":"2023-10-11T06:12:39.553595+0000","last_became_peered":"2023-10-11T06:12:39.553595+0000","last_unstale":"2023-10-11T06:12:40.016327+0000","last_undegraded":"2023-10-11T06:12:40.016327+0000","last_fullsized":"2023-10-11T06:12:40.016327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:28:42.668410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048945+0000","last_change":"2023-10-11T06:12:40.043914+0000","last_active":"2023-10-11T06:12:40.048945+0000","last_peered":"2023-10-11T06:12:40.048945+0000","last_clean":"2023-10-11T06:12:40.048945+0000","last_became_active":"2023-10-11T06:12:40.043610+0000","last_became_peered":"2023-10-11T06:12:40.043610+0000","last_unstale":"2023-10-11T06:12:40.048945+0000","last_undegraded":"2023-10-11T06:12:40.048945+0000","last_fullsized":"2023-10-11T06:12:40.048945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:39.940351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039800+0000","last_change":"2023-10-11T06:12:39.541852+0000","last_active":"2023-10-11T06:12:40.039800+0000","last_peered":"2023-10-11T06:12:40.039800+0000","last_clean":"2023-10-11T06:12:40.039800+0000","last_became_active":"2023-10-11T06:12:39.541512+0000","last_became_peered":"2023-10-11T06:12:39.541512+0000","last_unstale":"2023-10-11T06:12:40.039800+0000","last_undegraded":"2023-10-11T06:12:40.039800+0000","last_fullsized":"2023-10-11T06:12:40.039800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:35:59.834474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049937+0000","last_change":"2023-10-11T06:12:40.047718+0000","last_active":"2023-10-11T06:12:40.049937+0000","last_peered":"2023-10-11T06:12:40.049937+0000","last_clean":"2023-10-11T06:12:40.049937+0000","last_became_active":"2023-10-11T06:12:40.047257+0000","last_became_peered":"2023-10-11T06:12:40.047257+0000","last_unstale":"2023-10-11T06:12:40.049937+0000","last_undegraded":"2023-10-11T06:12:40.049937+0000","last_fullsized":"2023-10-11T06:12:40.049937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:37:44.742549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.016233+0000","last_change":"2023-10-11T06:12:39.554743+0000","last_active":"2023-10-11T06:12:40.016233+0000","last_peered":"2023-10-11T06:12:40.016233+0000","last_clean":"2023-10-11T06:12:40.016233+0000","last_became_active":"2023-10-11T06:12:39.554421+0000","last_became_peered":"2023-10-11T06:12:39.554421+0000","last_unstale":"2023-10-11T06:12:40.016233+0000","last_undegraded":"2023-10-11T06:12:40.016233+0000","last_fullsized":"2023-10-11T06:12:40.016233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:08:11.398571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.038869+0000","last_change":"2023-10-11T06:12:40.033658+0000","last_active":"2023-10-11T06:12:40.038869+0000","last_peered":"2023-10-11T06:12:40.038869+0000","last_clean":"2023-10-11T06:12:40.038869+0000","last_became_active":"2023-10-11T06:12:40.033375+0000","last_became_peered":"2023-10-11T06:12:40.033375+0000","last_unstale":"2023-10-11T06:12:40.038869+0000","last_undegraded":"2023-10-11T06:12:40.038869+0000","last_fullsized":"2023-10-11T06:12:40.038869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:16:46.060438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.024742+0000","last_change":"2023-10-11T06:12:39.552614+0000","last_active":"2023-10-11T06:12:40.024742+0000","last_peered":"2023-10-11T06:12:40.024742+0000","last_clean":"2023-10-11T06:12:40.024742+0000","last_became_active":"2023-10-11T06:12:39.552263+0000","last_became_peered":"2023-10-11T06:12:39.552263+0000","last_unstale":"2023-10-11T06:12:40.024742+0000","last_undegraded":"2023-10-11T06:12:40.024742+0000","last_fullsized":"2023-10-11T06:12:40.024742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:46.499194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041760+0000","last_change":"2023-10-11T06:12:40.038045+0000","last_active":"2023-10-11T06:12:40.041760+0000","last_peered":"2023-10-11T06:12:40.041760+0000","last_clean":"2023-10-11T06:12:40.041760+0000","last_became_active":"2023-10-11T06:12:40.037703+0000","last_became_peered":"2023-10-11T06:12:40.037703+0000","last_unstale":"2023-10-11T06:12:40.041760+0000","last_undegraded":"2023-10-11T06:12:40.041760+0000","last_fullsized":"2023-10-11T06:12:40.041760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:16:46.060438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.015493+0000","last_change":"2023-10-11T06:12:39.546913+0000","last_active":"2023-10-11T06:12:40.015493+0000","last_peered":"2023-10-11T06:12:40.015493+0000","last_clean":"2023-10-11T06:12:40.015493+0000","last_became_active":"2023-10-11T06:12:39.546628+0000","last_became_peered":"2023-10-11T06:12:39.546628+0000","last_unstale":"2023-10-11T06:12:40.015493+0000","last_undegraded":"2023-10-11T06:12:40.015493+0000","last_fullsized":"2023-10-11T06:12:40.015493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:03:30.855024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044454+0000","last_change":"2023-10-11T06:12:40.038791+0000","last_active":"2023-10-11T06:12:40.044454+0000","last_peered":"2023-10-11T06:12:40.044454+0000","last_clean":"2023-10-11T06:12:40.044454+0000","last_became_active":"2023-10-11T06:12:40.038515+0000","last_became_peered":"2023-10-11T06:12:40.038515+0000","last_unstale":"2023-10-11T06:12:40.044454+0000","last_undegraded":"2023-10-11T06:12:40.044454+0000","last_fullsized":"2023-10-11T06:12:40.044454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:50.254018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.022718+0000","last_change":"2023-10-11T06:12:39.551032+0000","last_active":"2023-10-11T06:12:40.022718+0000","last_peered":"2023-10-11T06:12:40.022718+0000","last_clean":"2023-10-11T06:12:40.022718+0000","last_became_active":"2023-10-11T06:12:39.550720+0000","last_became_peered":"2023-10-11T06:12:39.550720+0000","last_unstale":"2023-10-11T06:12:40.022718+0000","last_undegraded":"2023-10-11T06:12:40.022718+0000","last_fullsized":"2023-10-11T06:12:40.022718+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:53.089945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.541742+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:12:39.541742+0000","last_peered":"2023-10-11T06:12:39.541742+0000","last_clean":"2023-10-11T06:12:39.541742+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:12:39.541742+0000","last_undegraded":"2023-10-11T06:12:39.541742+0000","last_fullsized":"2023-10-11T06:12:39.541742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.530698+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:12:39.530698+0000","last_peered":"2023-10-11T06:12:39.530698+0000","last_clean":"2023-10-11T06:12:39.530698+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:12:39.530698+0000","last_undegraded":"2023-10-11T06:12:39.530698+0000","last_fullsized":"2023-10-11T06:12:39.530698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.043232+0000","last_change":"2023-10-11T06:12:40.040049+0000","last_active":"2023-10-11T06:12:40.043232+0000","last_peered":"2023-10-11T06:12:40.043232+0000","last_clean":"2023-10-11T06:12:40.043232+0000","last_became_active":"2023-10-11T06:12:40.039823+0000","last_became_peered":"2023-10-11T06:12:40.039823+0000","last_unstale":"2023-10-11T06:12:40.043232+0000","last_undegraded":"2023-10-11T06:12:40.043232+0000","last_fullsized":"2023-10-11T06:12:40.043232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:53.963222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045569+0000","last_change":"2023-10-11T06:12:39.557053+0000","last_active":"2023-10-11T06:12:40.045569+0000","last_peered":"2023-10-11T06:12:40.045569+0000","last_clean":"2023-10-11T06:12:40.045569+0000","last_became_active":"2023-10-11T06:12:39.556539+0000","last_became_peered":"2023-10-11T06:12:39.556539+0000","last_unstale":"2023-10-11T06:12:40.045569+0000","last_undegraded":"2023-10-11T06:12:40.045569+0000","last_fullsized":"2023-10-11T06:12:40.045569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:02.873373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.541540+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:12:39.541540+0000","last_peered":"2023-10-11T06:12:39.541540+0000","last_clean":"2023-10-11T06:12:39.541540+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:12:39.541540+0000","last_undegraded":"2023-10-11T06:12:39.541540+0000","last_fullsized":"2023-10-11T06:12:39.541540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049301+0000","last_change":"2023-10-11T06:12:40.046735+0000","last_active":"2023-10-11T06:12:40.049301+0000","last_peered":"2023-10-11T06:12:40.049301+0000","last_clean":"2023-10-11T06:12:40.049301+0000","last_became_active":"2023-10-11T06:12:40.046484+0000","last_became_peered":"2023-10-11T06:12:40.046484+0000","last_unstale":"2023-10-11T06:12:40.049301+0000","last_undegraded":"2023-10-11T06:12:40.049301+0000","last_fullsized":"2023-10-11T06:12:40.049301+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:47.911862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.016069+0000","last_change":"2023-10-11T06:12:39.545340+0000","last_active":"2023-10-11T06:12:40.016069+0000","last_peered":"2023-10-11T06:12:40.016069+0000","last_clean":"2023-10-11T06:12:40.016069+0000","last_became_active":"2023-10-11T06:12:39.545003+0000","last_became_peered":"2023-10-11T06:12:39.545003+0000","last_unstale":"2023-10-11T06:12:40.016069+0000","last_undegraded":"2023-10-11T06:12:40.016069+0000","last_fullsized":"2023-10-11T06:12:40.016069+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:35:33.188382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.530797+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:12:39.530797+0000","last_peered":"2023-10-11T06:12:39.530797+0000","last_clean":"2023-10-11T06:12:39.530797+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:12:39.530797+0000","last_undegraded":"2023-10-11T06:12:39.530797+0000","last_fullsized":"2023-10-11T06:12:39.530797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034916+0000","last_change":"2023-10-11T06:12:40.030168+0000","last_active":"2023-10-11T06:12:40.034916+0000","last_peered":"2023-10-11T06:12:40.034916+0000","last_clean":"2023-10-11T06:12:40.034916+0000","last_became_active":"2023-10-11T06:12:40.029886+0000","last_became_peered":"2023-10-11T06:12:40.029886+0000","last_unstale":"2023-10-11T06:12:40.034916+0000","last_undegraded":"2023-10-11T06:12:40.034916+0000","last_fullsized":"2023-10-11T06:12:40.034916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:38.377463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.023534+0000","last_change":"2023-10-11T06:12:39.547036+0000","last_active":"2023-10-11T06:12:40.023534+0000","last_peered":"2023-10-11T06:12:40.023534+0000","last_clean":"2023-10-11T06:12:40.023534+0000","last_became_active":"2023-10-11T06:12:39.546743+0000","last_became_peered":"2023-10-11T06:12:39.546743+0000","last_unstale":"2023-10-11T06:12:40.023534+0000","last_undegraded":"2023-10-11T06:12:40.023534+0000","last_fullsized":"2023-10-11T06:12:40.023534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:17.144333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.529092+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:12:39.529092+0000","last_peered":"2023-10-11T06:12:39.529092+0000","last_clean":"2023-10-11T06:12:39.529092+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:12:39.529092+0000","last_undegraded":"2023-10-11T06:12:39.529092+0000","last_fullsized":"2023-10-11T06:12:39.529092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035914+0000","last_change":"2023-10-11T06:12:40.034223+0000","last_active":"2023-10-11T06:12:40.035914+0000","last_peered":"2023-10-11T06:12:40.035914+0000","last_clean":"2023-10-11T06:12:40.035914+0000","last_became_active":"2023-10-11T06:12:40.033974+0000","last_became_peered":"2023-10-11T06:12:40.033974+0000","last_unstale":"2023-10-11T06:12:40.035914+0000","last_undegraded":"2023-10-11T06:12:40.035914+0000","last_fullsized":"2023-10-11T06:12:40.035914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:57:53.399147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.024888+0000","last_change":"2023-10-11T06:12:39.549813+0000","last_active":"2023-10-11T06:12:40.024888+0000","last_peered":"2023-10-11T06:12:40.024888+0000","last_clean":"2023-10-11T06:12:40.024888+0000","last_became_active":"2023-10-11T06:12:39.549504+0000","last_became_peered":"2023-10-11T06:12:39.549504+0000","last_unstale":"2023-10-11T06:12:40.024888+0000","last_undegraded":"2023-10-11T06:12:40.024888+0000","last_fullsized":"2023-10-11T06:12:40.024888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:11:40.142203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.535901+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:12:39.535901+0000","last_peered":"2023-10-11T06:12:39.535901+0000","last_clean":"2023-10-11T06:12:39.535901+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:12:39.535901+0000","last_undegraded":"2023-10-11T06:12:39.535901+0000","last_fullsized":"2023-10-11T06:12:39.535901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035507+0000","last_change":"2023-10-11T06:12:40.032462+0000","last_active":"2023-10-11T06:12:40.035507+0000","last_peered":"2023-10-11T06:12:40.035507+0000","last_clean":"2023-10-11T06:12:40.035507+0000","last_became_active":"2023-10-11T06:12:40.032192+0000","last_became_peered":"2023-10-11T06:12:40.032192+0000","last_unstale":"2023-10-11T06:12:40.035507+0000","last_undegraded":"2023-10-11T06:12:40.035507+0000","last_fullsized":"2023-10-11T06:12:40.035507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:00.886095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.017027+0000","last_change":"2023-10-11T06:12:39.545407+0000","last_active":"2023-10-11T06:12:40.017027+0000","last_peered":"2023-10-11T06:12:40.017027+0000","last_clean":"2023-10-11T06:12:40.017027+0000","last_became_active":"2023-10-11T06:12:39.544922+0000","last_became_peered":"2023-10-11T06:12:39.544922+0000","last_unstale":"2023-10-11T06:12:40.017027+0000","last_undegraded":"2023-10-11T06:12:40.017027+0000","last_fullsized":"2023-10-11T06:12:40.017027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:31:56.222312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.543577+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:12:39.543577+0000","last_peered":"2023-10-11T06:12:39.543577+0000","last_clean":"2023-10-11T06:12:39.543577+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:12:39.543577+0000","last_undegraded":"2023-10-11T06:12:39.543577+0000","last_fullsized":"2023-10-11T06:12:39.543577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.030126+0000","last_change":"2023-10-11T06:12:40.026712+0000","last_active":"2023-10-11T06:12:40.030126+0000","last_peered":"2023-10-11T06:12:40.030126+0000","last_clean":"2023-10-11T06:12:40.030126+0000","last_became_active":"2023-10-11T06:12:40.026379+0000","last_became_peered":"2023-10-11T06:12:40.026379+0000","last_unstale":"2023-10-11T06:12:40.030126+0000","last_undegraded":"2023-10-11T06:12:40.030126+0000","last_fullsized":"2023-10-11T06:12:40.030126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:57:55.747144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.014217+0000","last_change":"2023-10-11T06:12:39.552035+0000","last_active":"2023-10-11T06:12:40.014217+0000","last_peered":"2023-10-11T06:12:40.014217+0000","last_clean":"2023-10-11T06:12:40.014217+0000","last_became_active":"2023-10-11T06:12:39.551708+0000","last_became_peered":"2023-10-11T06:12:39.551708+0000","last_unstale":"2023-10-11T06:12:40.014217+0000","last_undegraded":"2023-10-11T06:12:40.014217+0000","last_fullsized":"2023-10-11T06:12:40.014217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:52.947677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.533268+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:12:39.533268+0000","last_peered":"2023-10-11T06:12:39.533268+0000","last_clean":"2023-10-11T06:12:39.533268+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:12:39.533268+0000","last_undegraded":"2023-10-11T06:12:39.533268+0000","last_fullsized":"2023-10-11T06:12:39.533268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045064+0000","last_change":"2023-10-11T06:12:40.035782+0000","last_active":"2023-10-11T06:12:40.045064+0000","last_peered":"2023-10-11T06:12:40.045064+0000","last_clean":"2023-10-11T06:12:40.045064+0000","last_became_active":"2023-10-11T06:12:40.035462+0000","last_became_peered":"2023-10-11T06:12:40.035462+0000","last_unstale":"2023-10-11T06:12:40.045064+0000","last_undegraded":"2023-10-11T06:12:40.045064+0000","last_fullsized":"2023-10-11T06:12:40.045064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:16:46.060438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.011431+0000","last_change":"2023-10-11T06:12:39.542539+0000","last_active":"2023-10-11T06:12:40.011431+0000","last_peered":"2023-10-11T06:12:40.011431+0000","last_clean":"2023-10-11T06:12:40.011431+0000","last_became_active":"2023-10-11T06:12:39.542184+0000","last_became_peered":"2023-10-11T06:12:39.542184+0000","last_unstale":"2023-10-11T06:12:40.011431+0000","last_undegraded":"2023-10-11T06:12:40.011431+0000","last_fullsized":"2023-10-11T06:12:40.011431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:25:38.498778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.530223+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:12:39.530223+0000","last_peered":"2023-10-11T06:12:39.530223+0000","last_clean":"2023-10-11T06:12:39.530223+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:12:39.530223+0000","last_undegraded":"2023-10-11T06:12:39.530223+0000","last_fullsized":"2023-10-11T06:12:39.530223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":750018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.427}]},{"osd":1,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.94199999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.355}]},{"osd":3,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.135},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.96499999999999997}]},{"osd":4,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.5659999999999998,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.0269999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.2},{"interface":"front","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.161}]},{"osd":6,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.0950000000000002,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.246}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.997},{"interface":"front","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.119},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.194}]},{"osd":3,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":3.6000000000000001,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.7170000000000001,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.236}]},{"osd":5,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.347},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.1000000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.6669999999999998,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.395}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.81499999999999995}]},{"osd":1,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.87}]},{"osd":2,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.069}]},{"osd":3,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":1.27},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.97699999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.371}]},{"osd":6,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.468}]},{"osd":7,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":1.304},{"interface":"front","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3440000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":174010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.401}]},{"osd":1,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.091},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0369999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.2230000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.163},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.524}]},{"osd":5,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.306},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.353}]},{"osd":6,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.575},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.474}]},{"osd":7,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":1.129,"5min":1.129,"15min":1.129},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.647}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.439},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.32300000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.56399999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":0.753},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.8780000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.77100000000000002}]},{"osd":5,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.1280000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":5.1100000000000003,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.88100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.73099999999999998}]},{"osd":1,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.286},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.032,"5min":3.032,"15min":3.032},"last":0.998}]},{"osd":4,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.218},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":1.1759999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.139},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.9119999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.2729999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.369},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.2030000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.25}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.79300000000000004}]},{"osd":4,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.268}]},{"osd":5,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.357}]},{"osd":6,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.2389999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.4279999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.405},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.0940000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":750018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.64800000000000002}]},{"osd":2,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":0.83999999999999997}]},{"osd":3,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0549999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.202}]},{"osd":5,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.671}]},{"osd":6,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.573},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.1339999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.2120000000000002,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.3799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:12:44.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:12:44.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.649+0000 7f1a284b6640 1 Processor -- start 2023-10-11T06:12:44.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.650+0000 7f1a284b6640 1 -- start start 2023-10-11T06:12:44.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.651+0000 7f1a284b6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a20159cd0 0x7f1a200b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.651+0000 7f1a284b6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a2015d320 0x7f1a200cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.651+0000 7f1a284b6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a200ceda0 con 0x7f1a20069c70 2023-10-11T06:12:44.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.651+0000 7f1a284b6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2015a0a0 con 0x7f1a2015d320 2023-10-11T06:12:44.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.651+0000 7f1a284b6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200cf020 con 0x7f1a20159cd0 2023-10-11T06:12:44.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.652+0000 7f1a25a51640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a2015d320 0x7f1a200cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.652+0000 7f1a26252640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a20159cd0 0x7f1a200b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.652+0000 7f1a25a51640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a2015d320 0x7f1a200cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:33406/0 (socket says 172.21.15.84:33406) 2023-10-11T06:12:44.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.652+0000 7f1a25a51640 1 -- 172.21.15.84:0/3997514237 learned_addr learned my addr 172.21.15.84:0/3997514237 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:44.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.653+0000 7f1a25a51640 1 -- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a20159cd0 msgr2=0x7f1a200b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:44.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.653+0000 7f1a25a51640 1 --2- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a20159cd0 0x7f1a200b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.653+0000 7f1a25a51640 1 -- 172.21.15.84:0/3997514237 >> v1:172.21.15.99:6789/0 conn(0x7f1a20069c70 legacy=0x7f1a200710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.653+0000 7f1a25a51640 1 -- 172.21.15.84:0/3997514237 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a200cf300 con 0x7f1a2015d320 2023-10-11T06:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.653+0000 7f1a25250640 1 -- 172.21.15.84:0/3997514237 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710410738 0 0) 0x7f1a200ceda0 con 0x7f1a20069c70 2023-10-11T06:12:44.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.653+0000 7f1a26252640 1 --2- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a20159cd0 0x7f1a200b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.654+0000 7f1a25a51640 1 --2- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a2015d320 0x7f1a200cd2d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009870 tx=0x7f1a0c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3872191eb3fdd7c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.655+0000 7f1a25250640 1 -- 172.21.15.84:0/3997514237 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c030840 con 0x7f1a2015d320 2023-10-11T06:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.655+0000 7f1a25250640 1 -- 172.21.15.84:0/3997514237 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c0309a0 con 0x7f1a2015d320 2023-10-11T06:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.655+0000 7f1a25250640 1 -- 172.21.15.84:0/3997514237 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c030d30 con 0x7f1a2015d320 2023-10-11T06:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 -- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a2015d320 msgr2=0x7f1a200cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 --2- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a2015d320 0x7f1a200cd2d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009870 tx=0x7f1a0c02b980 comp rx=0 tx=0).stop 2023-10-11T06:12:44.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 -- 172.21.15.84:0/3997514237 shutdown_connections 2023-10-11T06:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 --2- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a2015d320 0x7f1a200cd2d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 --2- 172.21.15.84:0/3997514237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a20159cd0 0x7f1a200b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 -- 172.21.15.84:0/3997514237 >> 172.21.15.84:0/3997514237 conn(0x7f1a2006b9b0 msgr2=0x7f1a2006d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 -- 172.21.15.84:0/3997514237 shutdown_connections 2023-10-11T06:12:44.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.656+0000 7f1a284b6640 1 -- 172.21.15.84:0/3997514237 wait complete. 2023-10-11T06:12:44.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.658+0000 7f1a284b6640 1 Processor -- start 2023-10-11T06:12:44.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.658+0000 7f1a284b6640 1 -- start start 2023-10-11T06:12:44.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.659+0000 7f1a284b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a20069c70 0x7f1a200ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.659+0000 7f1a284b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a20159cd0 0x7f1a200c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.659+0000 7f1a284b6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a2015d320 0x7f1a200af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.659+0000 7f1a284b6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200ae820 con 0x7f1a20159cd0 2023-10-11T06:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.659+0000 7f1a284b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200c4ab0 con 0x7f1a20069c70 2023-10-11T06:12:44.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.659+0000 7f1a284b6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200c4d90 con 0x7f1a2015d320 2023-10-11T06:12:44.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a20159cd0 0x7f1a200c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a20159cd0 0x7f1a200c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49658/0 (socket says 172.21.15.84:49658) 2023-10-11T06:12:44.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 -- 172.21.15.84:0/1507245524 learned_addr learned my addr 172.21.15.84:0/1507245524 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:44.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a26252640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a20069c70 0x7f1a200ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a26a53640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a2015d320 0x7f1a200af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 -- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a2015d320 msgr2=0x7f1a200af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a2015d320 0x7f1a200af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 -- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a20069c70 msgr2=0x7f1a200ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a20069c70 0x7f1a200ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.660+0000 7f1a25a51640 1 -- 172.21.15.84:0/1507245524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14009a70 con 0x7f1a20159cd0 2023-10-11T06:12:44.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.661+0000 7f1a25a51640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a20159cd0 0x7f1a200c2db0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000c00 tx=0x7f1a0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:44.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.661+0000 7f1a1effd640 1 -- 172.21.15.84:0/1507245524 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c039c90 con 0x7f1a20159cd0 2023-10-11T06:12:44.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.661+0000 7f1a1effd640 1 -- 172.21.15.84:0/1507245524 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c039df0 con 0x7f1a20159cd0 2023-10-11T06:12:44.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.661+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c0094d0 con 0x7f1a20159cd0 2023-10-11T06:12:44.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.661+0000 7f1a1effd640 1 -- 172.21.15.84:0/1507245524 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c043640 con 0x7f1a20159cd0 2023-10-11T06:12:44.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.661+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a200c5350 con 0x7f1a20159cd0 2023-10-11T06:12:44.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.663+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a20067090 con 0x7f1a20159cd0 2023-10-11T06:12:44.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.663+0000 7f1a1effd640 1 -- 172.21.15.84:0/1507245524 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1a0c0437a0 con 0x7f1a20159cd0 2023-10-11T06:12:44.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.664+0000 7f1a1effd640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1a0406a860 0x7f1a0406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:44.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.664+0000 7f1a1effd640 1 -- 172.21.15.84:0/1507245524 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a0c03e070 con 0x7f1a20159cd0 2023-10-11T06:12:44.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.665+0000 7f1a26252640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1a0406a860 0x7f1a0406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:44.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.665+0000 7f1a26252640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1a0406a860 0x7f1a0406cd10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009fd0 tx=0x7f1a1400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:44.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.669+0000 7f1a1effd640 1 -- 172.21.15.84:0/1507245524 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1a0c0af030 con 0x7f1a20159cd0 2023-10-11T06:12:44.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.855+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a200b15e0 con 0x7f1a0406a860 2023-10-11T06:12:44.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.879+0000 7f1a1effd640 1 -- 172.21.15.84:0/1507245524 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308754 (secure 0 0 0) 0x7f1a1400f040 con 0x7f1a0406a860 2023-10-11T06:12:44.884 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:44.886 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:12:44.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1a0406a860 msgr2=0x7f1a0406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1a0406a860 0x7f1a0406cd10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009fd0 tx=0x7f1a1400b040 comp rx=0 tx=0).stop 2023-10-11T06:12:44.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a20159cd0 msgr2=0x7f1a200c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:44.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a20159cd0 0x7f1a200c2db0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000c00 tx=0x7f1a0c004730 comp rx=0 tx=0).stop 2023-10-11T06:12:44.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 shutdown_connections 2023-10-11T06:12:44.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1a0406a860 0x7f1a0406cd10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a2015d320 0x7f1a200af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a20159cd0 0x7f1a200c2db0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 --2- 172.21.15.84:0/1507245524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a20069c70 0x7f1a200ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:44.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 >> 172.21.15.84:0/1507245524 conn(0x7f1a2006b9b0 msgr2=0x7f1a20070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:44.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 shutdown_connections 2023-10-11T06:12:44.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:44.886+0000 7f1a284b6640 1 -- 172.21.15.84:0/1507245524 wait complete. 2023-10-11T06:12:44.907 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-10-11T06:12:44.800403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219912,"kb_used_data":4296,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754808,"statfs":{"total":773094113280,"available":772868923392,"internally_reserved":0,"allocated":4399104,"data_stored":2472296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002032"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039012+0000","last_change":"2023-10-11T06:12:40.033024+0000","last_active":"2023-10-11T06:12:40.039012+0000","last_peered":"2023-10-11T06:12:40.039012+0000","last_clean":"2023-10-11T06:12:40.039012+0000","last_became_active":"2023-10-11T06:12:40.032747+0000","last_became_peered":"2023-10-11T06:12:40.032747+0000","last_unstale":"2023-10-11T06:12:40.039012+0000","last_undegraded":"2023-10-11T06:12:40.039012+0000","last_fullsized":"2023-10-11T06:12:40.039012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:54:42.612535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.027138+0000","last_change":"2023-10-11T06:12:40.021323+0000","last_active":"2023-10-11T06:12:40.027138+0000","last_peered":"2023-10-11T06:12:40.027138+0000","last_clean":"2023-10-11T06:12:40.027138+0000","last_became_active":"2023-10-11T06:12:40.020825+0000","last_became_peered":"2023-10-11T06:12:40.020825+0000","last_unstale":"2023-10-11T06:12:40.027138+0000","last_undegraded":"2023-10-11T06:12:40.027138+0000","last_fullsized":"2023-10-11T06:12:40.027138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:39.940351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042556+0000","last_change":"2023-10-11T06:12:40.039917+0000","last_active":"2023-10-11T06:12:40.042556+0000","last_peered":"2023-10-11T06:12:40.042556+0000","last_clean":"2023-10-11T06:12:40.042556+0000","last_became_active":"2023-10-11T06:12:40.038852+0000","last_became_peered":"2023-10-11T06:12:40.038852+0000","last_unstale":"2023-10-11T06:12:40.042556+0000","last_undegraded":"2023-10-11T06:12:40.042556+0000","last_fullsized":"2023-10-11T06:12:40.042556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:20:04.064920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036105+0000","last_change":"2023-10-11T06:12:40.030013+0000","last_active":"2023-10-11T06:12:40.036105+0000","last_peered":"2023-10-11T06:12:40.036105+0000","last_clean":"2023-10-11T06:12:40.036105+0000","last_became_active":"2023-10-11T06:12:40.029604+0000","last_became_peered":"2023-10-11T06:12:40.029604+0000","last_unstale":"2023-10-11T06:12:40.036105+0000","last_undegraded":"2023-10-11T06:12:40.036105+0000","last_fullsized":"2023-10-11T06:12:40.036105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:03.888918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036807+0000","last_change":"2023-10-11T06:12:40.030973+0000","last_active":"2023-10-11T06:12:40.036807+0000","last_peered":"2023-10-11T06:12:40.036807+0000","last_clean":"2023-10-11T06:12:40.036807+0000","last_became_active":"2023-10-11T06:12:40.030645+0000","last_became_peered":"2023-10-11T06:12:40.030645+0000","last_unstale":"2023-10-11T06:12:40.036807+0000","last_undegraded":"2023-10-11T06:12:40.036807+0000","last_fullsized":"2023-10-11T06:12:40.036807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:25:39.514323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.032603+0000","last_change":"2023-10-11T06:12:40.026094+0000","last_active":"2023-10-11T06:12:40.032603+0000","last_peered":"2023-10-11T06:12:40.032603+0000","last_clean":"2023-10-11T06:12:40.032603+0000","last_became_active":"2023-10-11T06:12:40.025612+0000","last_became_peered":"2023-10-11T06:12:40.025612+0000","last_unstale":"2023-10-11T06:12:40.032603+0000","last_undegraded":"2023-10-11T06:12:40.032603+0000","last_fullsized":"2023-10-11T06:12:40.032603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:23:59.597403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041827+0000","last_change":"2023-10-11T06:12:40.036422+0000","last_active":"2023-10-11T06:12:40.041827+0000","last_peered":"2023-10-11T06:12:40.041827+0000","last_clean":"2023-10-11T06:12:40.041827+0000","last_became_active":"2023-10-11T06:12:40.032519+0000","last_became_peered":"2023-10-11T06:12:40.032519+0000","last_unstale":"2023-10-11T06:12:40.041827+0000","last_undegraded":"2023-10-11T06:12:40.041827+0000","last_fullsized":"2023-10-11T06:12:40.041827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:39:41.279503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042573+0000","last_change":"2023-10-11T06:12:40.040112+0000","last_active":"2023-10-11T06:12:40.042573+0000","last_peered":"2023-10-11T06:12:40.042573+0000","last_clean":"2023-10-11T06:12:40.042573+0000","last_became_active":"2023-10-11T06:12:40.039582+0000","last_became_peered":"2023-10-11T06:12:40.039582+0000","last_unstale":"2023-10-11T06:12:40.042573+0000","last_undegraded":"2023-10-11T06:12:40.042573+0000","last_fullsized":"2023-10-11T06:12:40.042573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:39.281451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042977+0000","last_change":"2023-10-11T06:12:40.039844+0000","last_active":"2023-10-11T06:12:40.042977+0000","last_peered":"2023-10-11T06:12:40.042977+0000","last_clean":"2023-10-11T06:12:40.042977+0000","last_became_active":"2023-10-11T06:12:40.039317+0000","last_became_peered":"2023-10-11T06:12:40.039317+0000","last_unstale":"2023-10-11T06:12:40.042977+0000","last_undegraded":"2023-10-11T06:12:40.042977+0000","last_fullsized":"2023-10-11T06:12:40.042977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:12:30.177239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.052885+0000","last_change":"2023-10-11T06:12:40.047527+0000","last_active":"2023-10-11T06:12:40.052885+0000","last_peered":"2023-10-11T06:12:40.052885+0000","last_clean":"2023-10-11T06:12:40.052885+0000","last_became_active":"2023-10-11T06:12:40.047237+0000","last_became_peered":"2023-10-11T06:12:40.047237+0000","last_unstale":"2023-10-11T06:12:40.052885+0000","last_undegraded":"2023-10-11T06:12:40.052885+0000","last_fullsized":"2023-10-11T06:12:40.052885+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:39.281451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044427+0000","last_change":"2023-10-11T06:12:40.040062+0000","last_active":"2023-10-11T06:12:40.044427+0000","last_peered":"2023-10-11T06:12:40.044427+0000","last_clean":"2023-10-11T06:12:40.044427+0000","last_became_active":"2023-10-11T06:12:40.039747+0000","last_became_peered":"2023-10-11T06:12:40.039747+0000","last_unstale":"2023-10-11T06:12:40.044427+0000","last_undegraded":"2023-10-11T06:12:40.044427+0000","last_fullsized":"2023-10-11T06:12:40.044427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:38.377463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.033588+0000","last_change":"2023-10-11T06:12:40.029578+0000","last_active":"2023-10-11T06:12:40.033588+0000","last_peered":"2023-10-11T06:12:40.033588+0000","last_clean":"2023-10-11T06:12:40.033588+0000","last_became_active":"2023-10-11T06:12:40.029312+0000","last_became_peered":"2023-10-11T06:12:40.029312+0000","last_unstale":"2023-10-11T06:12:40.033588+0000","last_undegraded":"2023-10-11T06:12:40.033588+0000","last_fullsized":"2023-10-11T06:12:40.033588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:34:05.011734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.052917+0000","last_change":"2023-10-11T06:12:40.047603+0000","last_active":"2023-10-11T06:12:40.052917+0000","last_peered":"2023-10-11T06:12:40.052917+0000","last_clean":"2023-10-11T06:12:40.052917+0000","last_became_active":"2023-10-11T06:12:40.047238+0000","last_became_peered":"2023-10-11T06:12:40.047238+0000","last_unstale":"2023-10-11T06:12:40.052917+0000","last_undegraded":"2023-10-11T06:12:40.052917+0000","last_fullsized":"2023-10-11T06:12:40.052917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.028716+0000","last_change":"2023-10-11T06:12:40.024432+0000","last_active":"2023-10-11T06:12:40.028716+0000","last_peered":"2023-10-11T06:12:40.028716+0000","last_clean":"2023-10-11T06:12:40.028716+0000","last_became_active":"2023-10-11T06:12:40.024199+0000","last_became_peered":"2023-10-11T06:12:40.024199+0000","last_unstale":"2023-10-11T06:12:40.028716+0000","last_undegraded":"2023-10-11T06:12:40.028716+0000","last_fullsized":"2023-10-11T06:12:40.028716+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:09:17.938055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.038749+0000","last_change":"2023-10-11T06:12:40.032742+0000","last_active":"2023-10-11T06:12:40.038749+0000","last_peered":"2023-10-11T06:12:40.038749+0000","last_clean":"2023-10-11T06:12:40.038749+0000","last_became_active":"2023-10-11T06:12:40.032471+0000","last_became_peered":"2023-10-11T06:12:40.032471+0000","last_unstale":"2023-10-11T06:12:40.038749+0000","last_undegraded":"2023-10-11T06:12:40.038749+0000","last_fullsized":"2023-10-11T06:12:40.038749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:38.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049310+0000","last_change":"2023-10-11T06:12:40.046216+0000","last_active":"2023-10-11T06:12:40.049310+0000","last_peered":"2023-10-11T06:12:40.049310+0000","last_clean":"2023-10-11T06:12:40.049310+0000","last_became_active":"2023-10-11T06:12:40.046006+0000","last_became_peered":"2023-10-11T06:12:40.046006+0000","last_unstale":"2023-10-11T06:12:40.049310+0000","last_undegraded":"2023-10-11T06:12:40.049310+0000","last_fullsized":"2023-10-11T06:12:40.049310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048999+0000","last_change":"2023-10-11T06:12:40.045218+0000","last_active":"2023-10-11T06:12:40.048999+0000","last_peered":"2023-10-11T06:12:40.048999+0000","last_clean":"2023-10-11T06:12:40.048999+0000","last_became_active":"2023-10-11T06:12:40.044718+0000","last_became_peered":"2023-10-11T06:12:40.044718+0000","last_unstale":"2023-10-11T06:12:40.048999+0000","last_undegraded":"2023-10-11T06:12:40.048999+0000","last_fullsized":"2023-10-11T06:12:40.048999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:54:42.612535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.029410+0000","last_change":"2023-10-11T06:12:40.026253+0000","last_active":"2023-10-11T06:12:40.029410+0000","last_peered":"2023-10-11T06:12:40.029410+0000","last_clean":"2023-10-11T06:12:40.029410+0000","last_became_active":"2023-10-11T06:12:40.025962+0000","last_became_peered":"2023-10-11T06:12:40.025962+0000","last_unstale":"2023-10-11T06:12:40.029410+0000","last_undegraded":"2023-10-11T06:12:40.029410+0000","last_fullsized":"2023-10-11T06:12:40.029410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:47.514739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039322+0000","last_change":"2023-10-11T06:12:40.034197+0000","last_active":"2023-10-11T06:12:40.039322+0000","last_peered":"2023-10-11T06:12:40.039322+0000","last_clean":"2023-10-11T06:12:40.039322+0000","last_became_active":"2023-10-11T06:12:40.033933+0000","last_became_peered":"2023-10-11T06:12:40.033933+0000","last_unstale":"2023-10-11T06:12:40.039322+0000","last_undegraded":"2023-10-11T06:12:40.039322+0000","last_fullsized":"2023-10-11T06:12:40.039322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:02.844683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.029441+0000","last_change":"2023-10-11T06:12:40.026331+0000","last_active":"2023-10-11T06:12:40.029441+0000","last_peered":"2023-10-11T06:12:40.029441+0000","last_clean":"2023-10-11T06:12:40.029441+0000","last_became_active":"2023-10-11T06:12:40.026040+0000","last_became_peered":"2023-10-11T06:12:40.026040+0000","last_unstale":"2023-10-11T06:12:40.029441+0000","last_undegraded":"2023-10-11T06:12:40.029441+0000","last_fullsized":"2023-10-11T06:12:40.029441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:02.844683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044893+0000","last_change":"2023-10-11T06:12:40.040250+0000","last_active":"2023-10-11T06:12:40.044893+0000","last_peered":"2023-10-11T06:12:40.044893+0000","last_clean":"2023-10-11T06:12:40.044893+0000","last_became_active":"2023-10-11T06:12:40.039987+0000","last_became_peered":"2023-10-11T06:12:40.039987+0000","last_unstale":"2023-10-11T06:12:40.044893+0000","last_undegraded":"2023-10-11T06:12:40.044893+0000","last_fullsized":"2023-10-11T06:12:40.044893+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:59:40.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034263+0000","last_change":"2023-10-11T06:12:40.028636+0000","last_active":"2023-10-11T06:12:40.034263+0000","last_peered":"2023-10-11T06:12:40.034263+0000","last_clean":"2023-10-11T06:12:40.034263+0000","last_became_active":"2023-10-11T06:12:40.028377+0000","last_became_peered":"2023-10-11T06:12:40.028377+0000","last_unstale":"2023-10-11T06:12:40.034263+0000","last_undegraded":"2023-10-11T06:12:40.034263+0000","last_fullsized":"2023-10-11T06:12:40.034263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T18:00:39.281451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.040886+0000","last_change":"2023-10-11T06:12:40.034169+0000","last_active":"2023-10-11T06:12:40.040886+0000","last_peered":"2023-10-11T06:12:40.040886+0000","last_clean":"2023-10-11T06:12:40.040886+0000","last_became_active":"2023-10-11T06:12:40.033921+0000","last_became_peered":"2023-10-11T06:12:40.033921+0000","last_unstale":"2023-10-11T06:12:40.040886+0000","last_undegraded":"2023-10-11T06:12:40.040886+0000","last_fullsized":"2023-10-11T06:12:40.040886+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:57:55.747144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042602+0000","last_change":"2023-10-11T06:12:40.040719+0000","last_active":"2023-10-11T06:12:40.042602+0000","last_peered":"2023-10-11T06:12:40.042602+0000","last_clean":"2023-10-11T06:12:40.042602+0000","last_became_active":"2023-10-11T06:12:40.040439+0000","last_became_peered":"2023-10-11T06:12:40.040439+0000","last_unstale":"2023-10-11T06:12:40.042602+0000","last_undegraded":"2023-10-11T06:12:40.042602+0000","last_fullsized":"2023-10-11T06:12:40.042602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:47.514739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.030615+0000","last_change":"2023-10-11T06:12:40.028358+0000","last_active":"2023-10-11T06:12:40.030615+0000","last_peered":"2023-10-11T06:12:40.030615+0000","last_clean":"2023-10-11T06:12:40.030615+0000","last_became_active":"2023-10-11T06:12:40.028122+0000","last_became_peered":"2023-10-11T06:12:40.028122+0000","last_unstale":"2023-10-11T06:12:40.030615+0000","last_undegraded":"2023-10-11T06:12:40.030615+0000","last_fullsized":"2023-10-11T06:12:40.030615+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:49.725336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.047686+0000","last_change":"2023-10-11T06:12:40.043689+0000","last_active":"2023-10-11T06:12:40.047686+0000","last_peered":"2023-10-11T06:12:40.047686+0000","last_clean":"2023-10-11T06:12:40.047686+0000","last_became_active":"2023-10-11T06:12:40.043108+0000","last_became_peered":"2023-10-11T06:12:40.043108+0000","last_unstale":"2023-10-11T06:12:40.047686+0000","last_undegraded":"2023-10-11T06:12:40.047686+0000","last_fullsized":"2023-10-11T06:12:40.047686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:37:44.742549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034373+0000","last_change":"2023-10-11T06:12:40.032026+0000","last_active":"2023-10-11T06:12:40.034373+0000","last_peered":"2023-10-11T06:12:40.034373+0000","last_clean":"2023-10-11T06:12:40.034373+0000","last_became_active":"2023-10-11T06:12:40.029679+0000","last_became_peered":"2023-10-11T06:12:40.029679+0000","last_unstale":"2023-10-11T06:12:40.034373+0000","last_undegraded":"2023-10-11T06:12:40.034373+0000","last_fullsized":"2023-10-11T06:12:40.034373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:11:41.157748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044408+0000","last_change":"2023-10-11T06:12:40.039392+0000","last_active":"2023-10-11T06:12:40.044408+0000","last_peered":"2023-10-11T06:12:40.044408+0000","last_clean":"2023-10-11T06:12:40.044408+0000","last_became_active":"2023-10-11T06:12:40.039137+0000","last_became_peered":"2023-10-11T06:12:40.039137+0000","last_unstale":"2023-10-11T06:12:40.044408+0000","last_undegraded":"2023-10-11T06:12:40.044408+0000","last_fullsized":"2023-10-11T06:12:40.044408+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:14.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.038312+0000","last_change":"2023-10-11T06:12:40.034551+0000","last_active":"2023-10-11T06:12:40.038312+0000","last_peered":"2023-10-11T06:12:40.038312+0000","last_clean":"2023-10-11T06:12:40.038312+0000","last_became_active":"2023-10-11T06:12:40.034297+0000","last_became_peered":"2023-10-11T06:12:40.034297+0000","last_unstale":"2023-10-11T06:12:40.038312+0000","last_undegraded":"2023-10-11T06:12:40.038312+0000","last_fullsized":"2023-10-11T06:12:40.038312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:18.159878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034403+0000","last_change":"2023-10-11T06:12:40.031746+0000","last_active":"2023-10-11T06:12:40.034403+0000","last_peered":"2023-10-11T06:12:40.034403+0000","last_clean":"2023-10-11T06:12:40.034403+0000","last_became_active":"2023-10-11T06:12:40.031477+0000","last_became_peered":"2023-10-11T06:12:40.031477+0000","last_unstale":"2023-10-11T06:12:40.034403+0000","last_undegraded":"2023-10-11T06:12:40.034403+0000","last_fullsized":"2023-10-11T06:12:40.034403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:28:43.683955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048241+0000","last_change":"2023-10-11T06:12:40.041007+0000","last_active":"2023-10-11T06:12:40.048241+0000","last_peered":"2023-10-11T06:12:40.048241+0000","last_clean":"2023-10-11T06:12:40.048241+0000","last_became_active":"2023-10-11T06:12:40.040315+0000","last_became_peered":"2023-10-11T06:12:40.040315+0000","last_unstale":"2023-10-11T06:12:40.048241+0000","last_undegraded":"2023-10-11T06:12:40.048241+0000","last_fullsized":"2023-10-11T06:12:40.048241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:23:18.593586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034602+0000","last_change":"2023-10-11T06:12:40.031001+0000","last_active":"2023-10-11T06:12:40.034602+0000","last_peered":"2023-10-11T06:12:40.034602+0000","last_clean":"2023-10-11T06:12:40.034602+0000","last_became_active":"2023-10-11T06:12:40.030659+0000","last_became_peered":"2023-10-11T06:12:40.030659+0000","last_unstale":"2023-10-11T06:12:40.034602+0000","last_undegraded":"2023-10-11T06:12:40.034602+0000","last_fullsized":"2023-10-11T06:12:40.034602+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:21:29.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048486+0000","last_change":"2023-10-11T06:12:40.042664+0000","last_active":"2023-10-11T06:12:40.048486+0000","last_peered":"2023-10-11T06:12:40.048486+0000","last_clean":"2023-10-11T06:12:40.048486+0000","last_became_active":"2023-10-11T06:12:40.042347+0000","last_became_peered":"2023-10-11T06:12:40.042347+0000","last_unstale":"2023-10-11T06:12:40.048486+0000","last_undegraded":"2023-10-11T06:12:40.048486+0000","last_fullsized":"2023-10-11T06:12:40.048486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:12.851212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039603+0000","last_change":"2023-10-11T06:12:39.551526+0000","last_active":"2023-10-11T06:12:40.039603+0000","last_peered":"2023-10-11T06:12:40.039603+0000","last_clean":"2023-10-11T06:12:40.039603+0000","last_became_active":"2023-10-11T06:12:39.551259+0000","last_became_peered":"2023-10-11T06:12:39.551259+0000","last_unstale":"2023-10-11T06:12:40.039603+0000","last_undegraded":"2023-10-11T06:12:40.039603+0000","last_fullsized":"2023-10-11T06:12:40.039603+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:35:59.834474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044712+0000","last_change":"2023-10-11T06:12:40.040728+0000","last_active":"2023-10-11T06:12:40.044712+0000","last_peered":"2023-10-11T06:12:40.044712+0000","last_clean":"2023-10-11T06:12:40.044712+0000","last_became_active":"2023-10-11T06:12:40.040455+0000","last_became_peered":"2023-10-11T06:12:40.040455+0000","last_unstale":"2023-10-11T06:12:40.044712+0000","last_undegraded":"2023-10-11T06:12:40.044712+0000","last_fullsized":"2023-10-11T06:12:40.044712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:57:53.399147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.047210+0000","last_change":"2023-10-11T06:12:39.548879+0000","last_active":"2023-10-11T06:12:40.047210+0000","last_peered":"2023-10-11T06:12:40.047210+0000","last_clean":"2023-10-11T06:12:40.047210+0000","last_became_active":"2023-10-11T06:12:39.548594+0000","last_became_peered":"2023-10-11T06:12:39.548594+0000","last_unstale":"2023-10-11T06:12:40.047210+0000","last_undegraded":"2023-10-11T06:12:40.047210+0000","last_fullsized":"2023-10-11T06:12:40.047210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:28:42.668410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.028422+0000","last_change":"2023-10-11T06:12:40.024267+0000","last_active":"2023-10-11T06:12:40.028422+0000","last_peered":"2023-10-11T06:12:40.028422+0000","last_clean":"2023-10-11T06:12:40.028422+0000","last_became_active":"2023-10-11T06:12:40.023987+0000","last_became_peered":"2023-10-11T06:12:40.023987+0000","last_unstale":"2023-10-11T06:12:40.028422+0000","last_undegraded":"2023-10-11T06:12:40.028422+0000","last_fullsized":"2023-10-11T06:12:40.028422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:59:40.031937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035031+0000","last_change":"2023-10-11T06:12:39.551964+0000","last_active":"2023-10-11T06:12:40.035031+0000","last_peered":"2023-10-11T06:12:40.035031+0000","last_clean":"2023-10-11T06:12:40.035031+0000","last_became_active":"2023-10-11T06:12:39.551658+0000","last_became_peered":"2023-10-11T06:12:39.551658+0000","last_unstale":"2023-10-11T06:12:40.035031+0000","last_undegraded":"2023-10-11T06:12:40.035031+0000","last_fullsized":"2023-10-11T06:12:40.035031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:08:03.104559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.029632+0000","last_change":"2023-10-11T06:12:40.026295+0000","last_active":"2023-10-11T06:12:40.029632+0000","last_peered":"2023-10-11T06:12:40.029632+0000","last_clean":"2023-10-11T06:12:40.029632+0000","last_became_active":"2023-10-11T06:12:40.026039+0000","last_became_peered":"2023-10-11T06:12:40.026039+0000","last_unstale":"2023-10-11T06:12:40.029632+0000","last_undegraded":"2023-10-11T06:12:40.029632+0000","last_fullsized":"2023-10-11T06:12:40.029632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:57:55.747144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045382+0000","last_change":"2023-10-11T06:12:39.552520+0000","last_active":"2023-10-11T06:12:40.045382+0000","last_peered":"2023-10-11T06:12:40.045382+0000","last_clean":"2023-10-11T06:12:40.045382+0000","last_became_active":"2023-10-11T06:12:39.552149+0000","last_became_peered":"2023-10-11T06:12:39.552149+0000","last_unstale":"2023-10-11T06:12:40.045382+0000","last_undegraded":"2023-10-11T06:12:40.045382+0000","last_fullsized":"2023-10-11T06:12:40.045382+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:37:43.727004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041819+0000","last_change":"2023-10-11T06:12:40.035838+0000","last_active":"2023-10-11T06:12:40.041819+0000","last_peered":"2023-10-11T06:12:40.041819+0000","last_clean":"2023-10-11T06:12:40.041819+0000","last_became_active":"2023-10-11T06:12:40.035577+0000","last_became_peered":"2023-10-11T06:12:40.035577+0000","last_unstale":"2023-10-11T06:12:40.041819+0000","last_undegraded":"2023-10-11T06:12:40.041819+0000","last_fullsized":"2023-10-11T06:12:40.041819+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:00.886095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036203+0000","last_change":"2023-10-11T06:12:39.548700+0000","last_active":"2023-10-11T06:12:40.036203+0000","last_peered":"2023-10-11T06:12:40.036203+0000","last_clean":"2023-10-11T06:12:40.036203+0000","last_became_active":"2023-10-11T06:12:39.547969+0000","last_became_peered":"2023-10-11T06:12:39.547969+0000","last_unstale":"2023-10-11T06:12:40.036203+0000","last_undegraded":"2023-10-11T06:12:40.036203+0000","last_fullsized":"2023-10-11T06:12:40.036203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:23:58.581858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048697+0000","last_change":"2023-10-11T06:12:40.041198+0000","last_active":"2023-10-11T06:12:40.048697+0000","last_peered":"2023-10-11T06:12:40.048697+0000","last_clean":"2023-10-11T06:12:40.048697+0000","last_became_active":"2023-10-11T06:12:40.040759+0000","last_became_peered":"2023-10-11T06:12:40.040759+0000","last_unstale":"2023-10-11T06:12:40.048697+0000","last_undegraded":"2023-10-11T06:12:40.048697+0000","last_fullsized":"2023-10-11T06:12:40.048697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:09:17.938055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.046191+0000","last_change":"2023-10-11T06:12:39.551972+0000","last_active":"2023-10-11T06:12:40.046191+0000","last_peered":"2023-10-11T06:12:40.046191+0000","last_clean":"2023-10-11T06:12:40.046191+0000","last_became_active":"2023-10-11T06:12:39.551591+0000","last_became_peered":"2023-10-11T06:12:39.551591+0000","last_unstale":"2023-10-11T06:12:40.046191+0000","last_undegraded":"2023-10-11T06:12:40.046191+0000","last_fullsized":"2023-10-11T06:12:40.046191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:43:01.829138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.043797+0000","last_change":"2023-10-11T06:12:40.042009+0000","last_active":"2023-10-11T06:12:40.043797+0000","last_peered":"2023-10-11T06:12:40.043797+0000","last_clean":"2023-10-11T06:12:40.043797+0000","last_became_active":"2023-10-11T06:12:40.041459+0000","last_became_peered":"2023-10-11T06:12:40.041459+0000","last_unstale":"2023-10-11T06:12:40.043797+0000","last_undegraded":"2023-10-11T06:12:40.043797+0000","last_fullsized":"2023-10-11T06:12:40.043797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:59.927845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.015446+0000","last_change":"2023-10-11T06:12:39.561706+0000","last_active":"2023-10-11T06:12:40.015446+0000","last_peered":"2023-10-11T06:12:40.015446+0000","last_clean":"2023-10-11T06:12:40.015446+0000","last_became_active":"2023-10-11T06:12:39.561447+0000","last_became_peered":"2023-10-11T06:12:39.561447+0000","last_unstale":"2023-10-11T06:12:40.015446+0000","last_undegraded":"2023-10-11T06:12:40.015446+0000","last_fullsized":"2023-10-11T06:12:40.015446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:11:40.142203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041741+0000","last_change":"2023-10-11T06:12:40.038003+0000","last_active":"2023-10-11T06:12:40.041741+0000","last_peered":"2023-10-11T06:12:40.041741+0000","last_clean":"2023-10-11T06:12:40.041741+0000","last_became_active":"2023-10-11T06:12:40.037727+0000","last_became_peered":"2023-10-11T06:12:40.037727+0000","last_unstale":"2023-10-11T06:12:40.041741+0000","last_undegraded":"2023-10-11T06:12:40.041741+0000","last_fullsized":"2023-10-11T06:12:40.041741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:47.911862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.013918+0000","last_change":"2023-10-11T06:12:39.554563+0000","last_active":"2023-10-11T06:12:40.013918+0000","last_peered":"2023-10-11T06:12:40.013918+0000","last_clean":"2023-10-11T06:12:40.013918+0000","last_became_active":"2023-10-11T06:12:39.554191+0000","last_became_peered":"2023-10-11T06:12:39.554191+0000","last_unstale":"2023-10-11T06:12:40.013918+0000","last_undegraded":"2023-10-11T06:12:40.013918+0000","last_fullsized":"2023-10-11T06:12:40.013918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:34:03.996188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.036125+0000","last_change":"2023-10-11T06:12:40.030876+0000","last_active":"2023-10-11T06:12:40.036125+0000","last_peered":"2023-10-11T06:12:40.036125+0000","last_clean":"2023-10-11T06:12:40.036125+0000","last_became_active":"2023-10-11T06:12:40.030565+0000","last_became_peered":"2023-10-11T06:12:40.030565+0000","last_unstale":"2023-10-11T06:12:40.036125+0000","last_undegraded":"2023-10-11T06:12:40.036125+0000","last_fullsized":"2023-10-11T06:12:40.036125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:15:23.236778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.011580+0000","last_change":"2023-10-11T06:12:39.543157+0000","last_active":"2023-10-11T06:12:40.011580+0000","last_peered":"2023-10-11T06:12:40.011580+0000","last_clean":"2023-10-11T06:12:40.011580+0000","last_became_active":"2023-10-11T06:12:39.542683+0000","last_became_peered":"2023-10-11T06:12:39.542683+0000","last_unstale":"2023-10-11T06:12:40.011580+0000","last_undegraded":"2023-10-11T06:12:40.011580+0000","last_fullsized":"2023-10-11T06:12:40.011580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:15:22.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049280+0000","last_change":"2023-10-11T06:12:40.046079+0000","last_active":"2023-10-11T06:12:40.049280+0000","last_peered":"2023-10-11T06:12:40.049280+0000","last_clean":"2023-10-11T06:12:40.049280+0000","last_became_active":"2023-10-11T06:12:40.045787+0000","last_became_peered":"2023-10-11T06:12:40.045787+0000","last_unstale":"2023-10-11T06:12:40.049280+0000","last_undegraded":"2023-10-11T06:12:40.049280+0000","last_fullsized":"2023-10-11T06:12:40.049280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:38.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.027651+0000","last_change":"2023-10-11T06:12:39.557411+0000","last_active":"2023-10-11T06:12:40.027651+0000","last_peered":"2023-10-11T06:12:40.027651+0000","last_clean":"2023-10-11T06:12:40.027651+0000","last_became_active":"2023-10-11T06:12:39.557119+0000","last_became_peered":"2023-10-11T06:12:39.557119+0000","last_unstale":"2023-10-11T06:12:40.027651+0000","last_undegraded":"2023-10-11T06:12:40.027651+0000","last_fullsized":"2023-10-11T06:12:40.027651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:15:22.221232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048249+0000","last_change":"2023-10-11T06:12:40.047564+0000","last_active":"2023-10-11T06:12:40.048249+0000","last_peered":"2023-10-11T06:12:40.048249+0000","last_clean":"2023-10-11T06:12:40.048249+0000","last_became_active":"2023-10-11T06:12:40.047250+0000","last_became_peered":"2023-10-11T06:12:40.047250+0000","last_unstale":"2023-10-11T06:12:40.048249+0000","last_undegraded":"2023-10-11T06:12:40.048249+0000","last_fullsized":"2023-10-11T06:12:40.048249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:53.963222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045796+0000","last_change":"2023-10-11T06:12:39.557044+0000","last_active":"2023-10-11T06:12:40.045796+0000","last_peered":"2023-10-11T06:12:40.045796+0000","last_clean":"2023-10-11T06:12:40.045796+0000","last_became_active":"2023-10-11T06:12:39.556558+0000","last_became_peered":"2023-10-11T06:12:39.556558+0000","last_unstale":"2023-10-11T06:12:40.045796+0000","last_undegraded":"2023-10-11T06:12:40.045796+0000","last_fullsized":"2023-10-11T06:12:40.045796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:21:28.975805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034232+0000","last_change":"2023-10-11T06:12:40.029136+0000","last_active":"2023-10-11T06:12:40.034232+0000","last_peered":"2023-10-11T06:12:40.034232+0000","last_clean":"2023-10-11T06:12:40.034232+0000","last_became_active":"2023-10-11T06:12:40.028800+0000","last_became_peered":"2023-10-11T06:12:40.028800+0000","last_unstale":"2023-10-11T06:12:40.034232+0000","last_undegraded":"2023-10-11T06:12:40.034232+0000","last_fullsized":"2023-10-11T06:12:40.034232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.053493+0000","last_change":"2023-10-11T06:12:39.544686+0000","last_active":"2023-10-11T06:12:40.053493+0000","last_peered":"2023-10-11T06:12:40.053493+0000","last_clean":"2023-10-11T06:12:40.053493+0000","last_became_active":"2023-10-11T06:12:39.544374+0000","last_became_peered":"2023-10-11T06:12:39.544374+0000","last_unstale":"2023-10-11T06:12:40.053493+0000","last_undegraded":"2023-10-11T06:12:40.053493+0000","last_fullsized":"2023-10-11T06:12:40.053493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:46.499194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042584+0000","last_change":"2023-10-11T06:12:40.036601+0000","last_active":"2023-10-11T06:12:40.042584+0000","last_peered":"2023-10-11T06:12:40.042584+0000","last_clean":"2023-10-11T06:12:40.042584+0000","last_became_active":"2023-10-11T06:12:40.036362+0000","last_became_peered":"2023-10-11T06:12:40.036362+0000","last_unstale":"2023-10-11T06:12:40.042584+0000","last_undegraded":"2023-10-11T06:12:40.042584+0000","last_fullsized":"2023-10-11T06:12:40.042584+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:02.744270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.033051+0000","last_change":"2023-10-11T06:12:39.542882+0000","last_active":"2023-10-11T06:12:40.033051+0000","last_peered":"2023-10-11T06:12:40.033051+0000","last_clean":"2023-10-11T06:12:40.033051+0000","last_became_active":"2023-10-11T06:12:39.542603+0000","last_became_peered":"2023-10-11T06:12:39.542603+0000","last_unstale":"2023-10-11T06:12:40.033051+0000","last_undegraded":"2023-10-11T06:12:40.033051+0000","last_fullsized":"2023-10-11T06:12:40.033051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:02.873373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035537+0000","last_change":"2023-10-11T06:12:40.032457+0000","last_active":"2023-10-11T06:12:40.035537+0000","last_peered":"2023-10-11T06:12:40.035537+0000","last_clean":"2023-10-11T06:12:40.035537+0000","last_became_active":"2023-10-11T06:12:40.031951+0000","last_became_peered":"2023-10-11T06:12:40.031951+0000","last_unstale":"2023-10-11T06:12:40.035537+0000","last_undegraded":"2023-10-11T06:12:40.035537+0000","last_fullsized":"2023-10-11T06:12:40.035537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:42:12.851212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.015216+0000","last_change":"2023-10-11T06:12:39.561583+0000","last_active":"2023-10-11T06:12:40.015216+0000","last_peered":"2023-10-11T06:12:40.015216+0000","last_clean":"2023-10-11T06:12:40.015216+0000","last_became_active":"2023-10-11T06:12:39.561362+0000","last_became_peered":"2023-10-11T06:12:39.561362+0000","last_unstale":"2023-10-11T06:12:40.015216+0000","last_undegraded":"2023-10-11T06:12:40.015216+0000","last_fullsized":"2023-10-11T06:12:40.015216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:31:04.013960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035951+0000","last_change":"2023-10-11T06:12:40.030734+0000","last_active":"2023-10-11T06:12:40.035951+0000","last_peered":"2023-10-11T06:12:40.035951+0000","last_clean":"2023-10-11T06:12:40.035951+0000","last_became_active":"2023-10-11T06:12:40.030476+0000","last_became_peered":"2023-10-11T06:12:40.030476+0000","last_unstale":"2023-10-11T06:12:40.035951+0000","last_undegraded":"2023-10-11T06:12:40.035951+0000","last_fullsized":"2023-10-11T06:12:40.035951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:20:04.064920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.018975+0000","last_change":"2023-10-11T06:12:39.554445+0000","last_active":"2023-10-11T06:12:40.018975+0000","last_peered":"2023-10-11T06:12:40.018975+0000","last_clean":"2023-10-11T06:12:40.018975+0000","last_became_active":"2023-10-11T06:12:39.553941+0000","last_became_peered":"2023-10-11T06:12:39.553941+0000","last_unstale":"2023-10-11T06:12:40.018975+0000","last_undegraded":"2023-10-11T06:12:40.018975+0000","last_fullsized":"2023-10-11T06:12:40.018975+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:03:30.855024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.037453+0000","last_change":"2023-10-11T06:12:40.032116+0000","last_active":"2023-10-11T06:12:40.037453+0000","last_peered":"2023-10-11T06:12:40.037453+0000","last_clean":"2023-10-11T06:12:40.037453+0000","last_became_active":"2023-10-11T06:12:40.031858+0000","last_became_peered":"2023-10-11T06:12:40.031858+0000","last_unstale":"2023-10-11T06:12:40.037453+0000","last_undegraded":"2023-10-11T06:12:40.037453+0000","last_fullsized":"2023-10-11T06:12:40.037453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:07:38.427753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.050598+0000","last_change":"2023-10-11T06:12:39.551905+0000","last_active":"2023-10-11T06:12:40.050598+0000","last_peered":"2023-10-11T06:12:40.050598+0000","last_clean":"2023-10-11T06:12:40.050598+0000","last_became_active":"2023-10-11T06:12:39.551604+0000","last_became_peered":"2023-10-11T06:12:39.551604+0000","last_unstale":"2023-10-11T06:12:40.050598+0000","last_undegraded":"2023-10-11T06:12:40.050598+0000","last_fullsized":"2023-10-11T06:12:40.050598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:48.709791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.032766+0000","last_change":"2023-10-11T06:12:40.029691+0000","last_active":"2023-10-11T06:12:40.032766+0000","last_peered":"2023-10-11T06:12:40.032766+0000","last_clean":"2023-10-11T06:12:40.032766+0000","last_became_active":"2023-10-11T06:12:40.029412+0000","last_became_peered":"2023-10-11T06:12:40.029412+0000","last_unstale":"2023-10-11T06:12:40.032766+0000","last_undegraded":"2023-10-11T06:12:40.032766+0000","last_fullsized":"2023-10-11T06:12:40.032766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:05:14.664582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045980+0000","last_change":"2023-10-11T06:12:39.554535+0000","last_active":"2023-10-11T06:12:40.045980+0000","last_peered":"2023-10-11T06:12:40.045980+0000","last_clean":"2023-10-11T06:12:40.045980+0000","last_became_active":"2023-10-11T06:12:39.554254+0000","last_became_peered":"2023-10-11T06:12:39.554254+0000","last_unstale":"2023-10-11T06:12:40.045980+0000","last_undegraded":"2023-10-11T06:12:40.045980+0000","last_fullsized":"2023-10-11T06:12:40.045980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:27:59.870550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048901+0000","last_change":"2023-10-11T06:12:40.042131+0000","last_active":"2023-10-11T06:12:40.048901+0000","last_peered":"2023-10-11T06:12:40.048901+0000","last_clean":"2023-10-11T06:12:40.048901+0000","last_became_active":"2023-10-11T06:12:40.041881+0000","last_became_peered":"2023-10-11T06:12:40.041881+0000","last_unstale":"2023-10-11T06:12:40.048901+0000","last_undegraded":"2023-10-11T06:12:40.048901+0000","last_fullsized":"2023-10-11T06:12:40.048901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:59.927845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049940+0000","last_change":"2023-10-11T06:12:39.560892+0000","last_active":"2023-10-11T06:12:40.049940+0000","last_peered":"2023-10-11T06:12:40.049940+0000","last_clean":"2023-10-11T06:12:40.049940+0000","last_became_active":"2023-10-11T06:12:39.560546+0000","last_became_peered":"2023-10-11T06:12:39.560546+0000","last_unstale":"2023-10-11T06:12:40.049940+0000","last_undegraded":"2023-10-11T06:12:40.049940+0000","last_fullsized":"2023-10-11T06:12:40.049940+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:16:01.728725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.042347+0000","last_change":"2023-10-11T06:12:40.035526+0000","last_active":"2023-10-11T06:12:40.042347+0000","last_peered":"2023-10-11T06:12:40.042347+0000","last_clean":"2023-10-11T06:12:40.042347+0000","last_became_active":"2023-10-11T06:12:40.035264+0000","last_became_peered":"2023-10-11T06:12:40.035264+0000","last_unstale":"2023-10-11T06:12:40.042347+0000","last_undegraded":"2023-10-11T06:12:40.042347+0000","last_fullsized":"2023-10-11T06:12:40.042347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:38.377463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.047959+0000","last_change":"2023-10-11T06:12:39.547719+0000","last_active":"2023-10-11T06:12:40.047959+0000","last_peered":"2023-10-11T06:12:40.047959+0000","last_clean":"2023-10-11T06:12:40.047959+0000","last_became_active":"2023-10-11T06:12:39.547374+0000","last_became_peered":"2023-10-11T06:12:39.547374+0000","last_unstale":"2023-10-11T06:12:40.047959+0000","last_undegraded":"2023-10-11T06:12:40.047959+0000","last_fullsized":"2023-10-11T06:12:40.047959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:31:56.222312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.032187+0000","last_change":"2023-10-11T06:12:40.027222+0000","last_active":"2023-10-11T06:12:40.032187+0000","last_peered":"2023-10-11T06:12:40.032187+0000","last_clean":"2023-10-11T06:12:40.032187+0000","last_became_active":"2023-10-11T06:12:40.026946+0000","last_became_peered":"2023-10-11T06:12:40.026946+0000","last_unstale":"2023-10-11T06:12:40.032187+0000","last_undegraded":"2023-10-11T06:12:40.032187+0000","last_fullsized":"2023-10-11T06:12:40.032187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:03:49.725336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.016327+0000","last_change":"2023-10-11T06:12:39.554020+0000","last_active":"2023-10-11T06:12:40.016327+0000","last_peered":"2023-10-11T06:12:40.016327+0000","last_clean":"2023-10-11T06:12:40.016327+0000","last_became_active":"2023-10-11T06:12:39.553595+0000","last_became_peered":"2023-10-11T06:12:39.553595+0000","last_unstale":"2023-10-11T06:12:40.016327+0000","last_undegraded":"2023-10-11T06:12:40.016327+0000","last_fullsized":"2023-10-11T06:12:40.016327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:28:42.668410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.048945+0000","last_change":"2023-10-11T06:12:40.043914+0000","last_active":"2023-10-11T06:12:40.048945+0000","last_peered":"2023-10-11T06:12:40.048945+0000","last_clean":"2023-10-11T06:12:40.048945+0000","last_became_active":"2023-10-11T06:12:40.043610+0000","last_became_peered":"2023-10-11T06:12:40.043610+0000","last_unstale":"2023-10-11T06:12:40.048945+0000","last_undegraded":"2023-10-11T06:12:40.048945+0000","last_fullsized":"2023-10-11T06:12:40.048945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:46:39.940351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.039800+0000","last_change":"2023-10-11T06:12:39.541852+0000","last_active":"2023-10-11T06:12:40.039800+0000","last_peered":"2023-10-11T06:12:40.039800+0000","last_clean":"2023-10-11T06:12:40.039800+0000","last_became_active":"2023-10-11T06:12:39.541512+0000","last_became_peered":"2023-10-11T06:12:39.541512+0000","last_unstale":"2023-10-11T06:12:40.039800+0000","last_undegraded":"2023-10-11T06:12:40.039800+0000","last_fullsized":"2023-10-11T06:12:40.039800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:35:59.834474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049937+0000","last_change":"2023-10-11T06:12:40.047718+0000","last_active":"2023-10-11T06:12:40.049937+0000","last_peered":"2023-10-11T06:12:40.049937+0000","last_clean":"2023-10-11T06:12:40.049937+0000","last_became_active":"2023-10-11T06:12:40.047257+0000","last_became_peered":"2023-10-11T06:12:40.047257+0000","last_unstale":"2023-10-11T06:12:40.049937+0000","last_undegraded":"2023-10-11T06:12:40.049937+0000","last_fullsized":"2023-10-11T06:12:40.049937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:37:44.742549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.016233+0000","last_change":"2023-10-11T06:12:39.554743+0000","last_active":"2023-10-11T06:12:40.016233+0000","last_peered":"2023-10-11T06:12:40.016233+0000","last_clean":"2023-10-11T06:12:40.016233+0000","last_became_active":"2023-10-11T06:12:39.554421+0000","last_became_peered":"2023-10-11T06:12:39.554421+0000","last_unstale":"2023-10-11T06:12:40.016233+0000","last_undegraded":"2023-10-11T06:12:40.016233+0000","last_fullsized":"2023-10-11T06:12:40.016233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:08:11.398571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.038869+0000","last_change":"2023-10-11T06:12:40.033658+0000","last_active":"2023-10-11T06:12:40.038869+0000","last_peered":"2023-10-11T06:12:40.038869+0000","last_clean":"2023-10-11T06:12:40.038869+0000","last_became_active":"2023-10-11T06:12:40.033375+0000","last_became_peered":"2023-10-11T06:12:40.033375+0000","last_unstale":"2023-10-11T06:12:40.038869+0000","last_undegraded":"2023-10-11T06:12:40.038869+0000","last_fullsized":"2023-10-11T06:12:40.038869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:16:46.060438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.024742+0000","last_change":"2023-10-11T06:12:39.552614+0000","last_active":"2023-10-11T06:12:40.024742+0000","last_peered":"2023-10-11T06:12:40.024742+0000","last_clean":"2023-10-11T06:12:40.024742+0000","last_became_active":"2023-10-11T06:12:39.552263+0000","last_became_peered":"2023-10-11T06:12:39.552263+0000","last_unstale":"2023-10-11T06:12:40.024742+0000","last_undegraded":"2023-10-11T06:12:40.024742+0000","last_fullsized":"2023-10-11T06:12:40.024742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:48:46.499194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.041760+0000","last_change":"2023-10-11T06:12:40.038045+0000","last_active":"2023-10-11T06:12:40.041760+0000","last_peered":"2023-10-11T06:12:40.041760+0000","last_clean":"2023-10-11T06:12:40.041760+0000","last_became_active":"2023-10-11T06:12:40.037703+0000","last_became_peered":"2023-10-11T06:12:40.037703+0000","last_unstale":"2023-10-11T06:12:40.041760+0000","last_undegraded":"2023-10-11T06:12:40.041760+0000","last_fullsized":"2023-10-11T06:12:40.041760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:16:46.060438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.015493+0000","last_change":"2023-10-11T06:12:39.546913+0000","last_active":"2023-10-11T06:12:40.015493+0000","last_peered":"2023-10-11T06:12:40.015493+0000","last_clean":"2023-10-11T06:12:40.015493+0000","last_became_active":"2023-10-11T06:12:39.546628+0000","last_became_peered":"2023-10-11T06:12:39.546628+0000","last_unstale":"2023-10-11T06:12:40.015493+0000","last_undegraded":"2023-10-11T06:12:40.015493+0000","last_fullsized":"2023-10-11T06:12:40.015493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:03:30.855024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.044454+0000","last_change":"2023-10-11T06:12:40.038791+0000","last_active":"2023-10-11T06:12:40.044454+0000","last_peered":"2023-10-11T06:12:40.044454+0000","last_clean":"2023-10-11T06:12:40.044454+0000","last_became_active":"2023-10-11T06:12:40.038515+0000","last_became_peered":"2023-10-11T06:12:40.038515+0000","last_unstale":"2023-10-11T06:12:40.044454+0000","last_undegraded":"2023-10-11T06:12:40.044454+0000","last_fullsized":"2023-10-11T06:12:40.044454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:50:50.254018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.022718+0000","last_change":"2023-10-11T06:12:39.551032+0000","last_active":"2023-10-11T06:12:40.022718+0000","last_peered":"2023-10-11T06:12:40.022718+0000","last_clean":"2023-10-11T06:12:40.022718+0000","last_became_active":"2023-10-11T06:12:39.550720+0000","last_became_peered":"2023-10-11T06:12:39.550720+0000","last_unstale":"2023-10-11T06:12:40.022718+0000","last_undegraded":"2023-10-11T06:12:40.022718+0000","last_fullsized":"2023-10-11T06:12:40.022718+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:40:53.089945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.541742+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:12:39.541742+0000","last_peered":"2023-10-11T06:12:39.541742+0000","last_clean":"2023-10-11T06:12:39.541742+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:12:39.541742+0000","last_undegraded":"2023-10-11T06:12:39.541742+0000","last_fullsized":"2023-10-11T06:12:39.541742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.530698+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:12:39.530698+0000","last_peered":"2023-10-11T06:12:39.530698+0000","last_clean":"2023-10-11T06:12:39.530698+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:12:39.530698+0000","last_undegraded":"2023-10-11T06:12:39.530698+0000","last_fullsized":"2023-10-11T06:12:39.530698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.043232+0000","last_change":"2023-10-11T06:12:40.040049+0000","last_active":"2023-10-11T06:12:40.043232+0000","last_peered":"2023-10-11T06:12:40.043232+0000","last_clean":"2023-10-11T06:12:40.043232+0000","last_became_active":"2023-10-11T06:12:40.039823+0000","last_became_peered":"2023-10-11T06:12:40.039823+0000","last_unstale":"2023-10-11T06:12:40.043232+0000","last_undegraded":"2023-10-11T06:12:40.043232+0000","last_fullsized":"2023-10-11T06:12:40.043232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:53.963222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045569+0000","last_change":"2023-10-11T06:12:39.557053+0000","last_active":"2023-10-11T06:12:40.045569+0000","last_peered":"2023-10-11T06:12:40.045569+0000","last_clean":"2023-10-11T06:12:40.045569+0000","last_became_active":"2023-10-11T06:12:39.556539+0000","last_became_peered":"2023-10-11T06:12:39.556539+0000","last_unstale":"2023-10-11T06:12:40.045569+0000","last_undegraded":"2023-10-11T06:12:40.045569+0000","last_fullsized":"2023-10-11T06:12:40.045569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:29:02.873373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.541540+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:12:39.541540+0000","last_peered":"2023-10-11T06:12:39.541540+0000","last_clean":"2023-10-11T06:12:39.541540+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:12:39.541540+0000","last_undegraded":"2023-10-11T06:12:39.541540+0000","last_fullsized":"2023-10-11T06:12:39.541540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.049301+0000","last_change":"2023-10-11T06:12:40.046735+0000","last_active":"2023-10-11T06:12:40.049301+0000","last_peered":"2023-10-11T06:12:40.049301+0000","last_clean":"2023-10-11T06:12:40.049301+0000","last_became_active":"2023-10-11T06:12:40.046484+0000","last_became_peered":"2023-10-11T06:12:40.046484+0000","last_unstale":"2023-10-11T06:12:40.049301+0000","last_undegraded":"2023-10-11T06:12:40.049301+0000","last_fullsized":"2023-10-11T06:12:40.049301+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:31:47.911862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.016069+0000","last_change":"2023-10-11T06:12:39.545340+0000","last_active":"2023-10-11T06:12:40.016069+0000","last_peered":"2023-10-11T06:12:40.016069+0000","last_clean":"2023-10-11T06:12:40.016069+0000","last_became_active":"2023-10-11T06:12:39.545003+0000","last_became_peered":"2023-10-11T06:12:39.545003+0000","last_unstale":"2023-10-11T06:12:40.016069+0000","last_undegraded":"2023-10-11T06:12:40.016069+0000","last_fullsized":"2023-10-11T06:12:40.016069+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:35:33.188382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.530797+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:12:39.530797+0000","last_peered":"2023-10-11T06:12:39.530797+0000","last_clean":"2023-10-11T06:12:39.530797+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:12:39.530797+0000","last_undegraded":"2023-10-11T06:12:39.530797+0000","last_fullsized":"2023-10-11T06:12:39.530797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.034916+0000","last_change":"2023-10-11T06:12:40.030168+0000","last_active":"2023-10-11T06:12:40.034916+0000","last_peered":"2023-10-11T06:12:40.034916+0000","last_clean":"2023-10-11T06:12:40.034916+0000","last_became_active":"2023-10-11T06:12:40.029886+0000","last_became_peered":"2023-10-11T06:12:40.029886+0000","last_unstale":"2023-10-11T06:12:40.034916+0000","last_undegraded":"2023-10-11T06:12:40.034916+0000","last_fullsized":"2023-10-11T06:12:40.034916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:00:38.377463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.023534+0000","last_change":"2023-10-11T06:12:39.547036+0000","last_active":"2023-10-11T06:12:40.023534+0000","last_peered":"2023-10-11T06:12:40.023534+0000","last_clean":"2023-10-11T06:12:40.023534+0000","last_became_active":"2023-10-11T06:12:39.546743+0000","last_became_peered":"2023-10-11T06:12:39.546743+0000","last_unstale":"2023-10-11T06:12:40.023534+0000","last_undegraded":"2023-10-11T06:12:40.023534+0000","last_fullsized":"2023-10-11T06:12:40.023534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:21:17.144333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.529092+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:12:39.529092+0000","last_peered":"2023-10-11T06:12:39.529092+0000","last_clean":"2023-10-11T06:12:39.529092+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:12:39.529092+0000","last_undegraded":"2023-10-11T06:12:39.529092+0000","last_fullsized":"2023-10-11T06:12:39.529092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035914+0000","last_change":"2023-10-11T06:12:40.034223+0000","last_active":"2023-10-11T06:12:40.035914+0000","last_peered":"2023-10-11T06:12:40.035914+0000","last_clean":"2023-10-11T06:12:40.035914+0000","last_became_active":"2023-10-11T06:12:40.033974+0000","last_became_peered":"2023-10-11T06:12:40.033974+0000","last_unstale":"2023-10-11T06:12:40.035914+0000","last_undegraded":"2023-10-11T06:12:40.035914+0000","last_fullsized":"2023-10-11T06:12:40.035914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:57:53.399147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.024888+0000","last_change":"2023-10-11T06:12:39.549813+0000","last_active":"2023-10-11T06:12:40.024888+0000","last_peered":"2023-10-11T06:12:40.024888+0000","last_clean":"2023-10-11T06:12:40.024888+0000","last_became_active":"2023-10-11T06:12:39.549504+0000","last_became_peered":"2023-10-11T06:12:39.549504+0000","last_unstale":"2023-10-11T06:12:40.024888+0000","last_undegraded":"2023-10-11T06:12:40.024888+0000","last_fullsized":"2023-10-11T06:12:40.024888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T16:11:40.142203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.535901+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:12:39.535901+0000","last_peered":"2023-10-11T06:12:39.535901+0000","last_clean":"2023-10-11T06:12:39.535901+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:12:39.535901+0000","last_undegraded":"2023-10-11T06:12:39.535901+0000","last_fullsized":"2023-10-11T06:12:39.535901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.035507+0000","last_change":"2023-10-11T06:12:40.032462+0000","last_active":"2023-10-11T06:12:40.035507+0000","last_peered":"2023-10-11T06:12:40.035507+0000","last_clean":"2023-10-11T06:12:40.035507+0000","last_became_active":"2023-10-11T06:12:40.032192+0000","last_became_peered":"2023-10-11T06:12:40.032192+0000","last_unstale":"2023-10-11T06:12:40.035507+0000","last_undegraded":"2023-10-11T06:12:40.035507+0000","last_fullsized":"2023-10-11T06:12:40.035507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:28:00.886095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.017027+0000","last_change":"2023-10-11T06:12:39.545407+0000","last_active":"2023-10-11T06:12:40.017027+0000","last_peered":"2023-10-11T06:12:40.017027+0000","last_clean":"2023-10-11T06:12:40.017027+0000","last_became_active":"2023-10-11T06:12:39.544922+0000","last_became_peered":"2023-10-11T06:12:39.544922+0000","last_unstale":"2023-10-11T06:12:40.017027+0000","last_undegraded":"2023-10-11T06:12:40.017027+0000","last_fullsized":"2023-10-11T06:12:40.017027+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:31:56.222312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.543577+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:12:39.543577+0000","last_peered":"2023-10-11T06:12:39.543577+0000","last_clean":"2023-10-11T06:12:39.543577+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:12:39.543577+0000","last_undegraded":"2023-10-11T06:12:39.543577+0000","last_fullsized":"2023-10-11T06:12:39.543577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.030126+0000","last_change":"2023-10-11T06:12:40.026712+0000","last_active":"2023-10-11T06:12:40.030126+0000","last_peered":"2023-10-11T06:12:40.030126+0000","last_clean":"2023-10-11T06:12:40.030126+0000","last_became_active":"2023-10-11T06:12:40.026379+0000","last_became_peered":"2023-10-11T06:12:40.026379+0000","last_unstale":"2023-10-11T06:12:40.030126+0000","last_undegraded":"2023-10-11T06:12:40.030126+0000","last_fullsized":"2023-10-11T06:12:40.030126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:57:55.747144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.014217+0000","last_change":"2023-10-11T06:12:39.552035+0000","last_active":"2023-10-11T06:12:40.014217+0000","last_peered":"2023-10-11T06:12:40.014217+0000","last_clean":"2023-10-11T06:12:40.014217+0000","last_became_active":"2023-10-11T06:12:39.551708+0000","last_became_peered":"2023-10-11T06:12:39.551708+0000","last_unstale":"2023-10-11T06:12:40.014217+0000","last_undegraded":"2023-10-11T06:12:40.014217+0000","last_fullsized":"2023-10-11T06:12:40.014217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:13:52.947677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.533268+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:12:39.533268+0000","last_peered":"2023-10-11T06:12:39.533268+0000","last_clean":"2023-10-11T06:12:39.533268+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:12:39.533268+0000","last_undegraded":"2023-10-11T06:12:39.533268+0000","last_fullsized":"2023-10-11T06:12:39.533268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.045064+0000","last_change":"2023-10-11T06:12:40.035782+0000","last_active":"2023-10-11T06:12:40.045064+0000","last_peered":"2023-10-11T06:12:40.045064+0000","last_clean":"2023-10-11T06:12:40.045064+0000","last_became_active":"2023-10-11T06:12:40.035462+0000","last_became_peered":"2023-10-11T06:12:40.035462+0000","last_unstale":"2023-10-11T06:12:40.045064+0000","last_undegraded":"2023-10-11T06:12:40.045064+0000","last_fullsized":"2023-10-11T06:12:40.045064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:39.503254+0000","last_clean_scrub_stamp":"2023-10-11T06:12:39.503254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:16:46.060438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-11T06:12:40.011431+0000","last_change":"2023-10-11T06:12:39.542539+0000","last_active":"2023-10-11T06:12:40.011431+0000","last_peered":"2023-10-11T06:12:40.011431+0000","last_clean":"2023-10-11T06:12:40.011431+0000","last_became_active":"2023-10-11T06:12:39.542184+0000","last_became_peered":"2023-10-11T06:12:39.542184+0000","last_unstale":"2023-10-11T06:12:40.011431+0000","last_undegraded":"2023-10-11T06:12:40.011431+0000","last_fullsized":"2023-10-11T06:12:40.011431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:12:38.487709+0000","last_clean_scrub_stamp":"2023-10-11T06:12:38.487709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:25:38.498778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-11T06:12:39.530223+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:12:39.530223+0000","last_peered":"2023-10-11T06:12:39.530223+0000","last_clean":"2023-10-11T06:12:39.530223+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:12:39.530223+0000","last_undegraded":"2023-10-11T06:12:39.530223+0000","last_fullsized":"2023-10-11T06:12:39.530223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":750018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.427}]},{"osd":1,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.94199999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.355}]},{"osd":3,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.135},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.96499999999999997}]},{"osd":4,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.5659999999999998,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.0269999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.2},{"interface":"front","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.161}]},{"osd":6,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.0950000000000002,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.246}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.997},{"interface":"front","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.119},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.194}]},{"osd":3,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":3.6000000000000001,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.274}]},{"osd":4,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.7170000000000001,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.236}]},{"osd":5,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.347},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.1000000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.6669999999999998,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.395}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.81499999999999995}]},{"osd":1,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.87}]},{"osd":2,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":1.069}]},{"osd":3,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":1.27},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.97699999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.371}]},{"osd":6,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.468}]},{"osd":7,"last update":"Wed Oct 11 06:12:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":1.304},{"interface":"front","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3440000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":174010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.401}]},{"osd":1,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.091},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0369999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.2230000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.163},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.524}]},{"osd":5,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.306},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.353}]},{"osd":6,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.575},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.474}]},{"osd":7,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":1.129,"5min":1.129,"15min":1.129},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.647}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.439},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.32300000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.56399999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":4.1470000000000002,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":3.286,"5min":3.286,"15min":3.286},"last":0.753},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.8780000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.77100000000000002}]},{"osd":5,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":3.1280000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":5.1100000000000003,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:12:01 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.88100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.73099999999999998}]},{"osd":1,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.286},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":3.032,"5min":3.032,"15min":3.032},"last":0.998}]},{"osd":4,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.218},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":1.1759999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.139},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.9119999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.2729999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.369},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.2030000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.25}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":159650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.75800000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.79300000000000004}]},{"osd":4,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.268}]},{"osd":5,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.357}]},{"osd":6,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.2389999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.4279999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:12:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.405},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.0940000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":750018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.64800000000000002}]},{"osd":2,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":0.83999999999999997}]},{"osd":3,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0549999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.202}]},{"osd":5,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.671}]},{"osd":6,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.573},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.1339999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.2120000000000002,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.3799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:12:44.913 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T06:12:44.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-11T06:12:45.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.176+0000 7fe3eea1a640 1 Processor -- start 2023-10-11T06:12:45.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.176+0000 7fe3eea1a640 1 -- start start 2023-10-11T06:12:45.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3eea1a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3e80ced30 0x7fe3e80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3eea1a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3e80c8d00 0x7fe3e806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3eea1a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e8071380 con 0x7fe3e815a0b0 2023-10-11T06:12:45.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3eea1a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e806e0f0 con 0x7fe3e80ced30 2023-10-11T06:12:45.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3eea1a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e80cbae0 con 0x7fe3e80c8d00 2023-10-11T06:12:45.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3e80ced30 0x7fe3e80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3dffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3e80ced30 0x7fe3e80cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49674/0 (socket says 172.21.15.84:49674) 2023-10-11T06:12:45.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3dffff640 1 -- 172.21.15.84:0/2175072134 learned_addr learned my addr 172.21.15.84:0/2175072134 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:45.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.178+0000 7fe3e7fff640 1 --1- 172.21.15.84:0/2175072134 >> v1:172.21.15.84:6789/0 conn(0x7fe3e815a0b0 0x7fe3e80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48256/0 (socket says 172.21.15.84:48256) 2023-10-11T06:12:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.179+0000 7fe3ecfb7640 1 --2- 172.21.15.84:0/2175072134 >> v2:172.21.15.99:3300/0 conn(0x7fe3e80c8d00 0x7fe3e806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.179+0000 7fe3dffff640 1 -- 172.21.15.84:0/2175072134 >> v2:172.21.15.99:3300/0 conn(0x7fe3e80c8d00 msgr2=0x7fe3e806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.179+0000 7fe3dffff640 1 --2- 172.21.15.84:0/2175072134 >> v2:172.21.15.99:3300/0 conn(0x7fe3e80c8d00 0x7fe3e806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.179+0000 7fe3dffff640 1 -- 172.21.15.84:0/2175072134 >> v1:172.21.15.84:6789/0 conn(0x7fe3e815a0b0 legacy=0x7fe3e80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.179+0000 7fe3dffff640 1 -- 172.21.15.84:0/2175072134 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e806de10 con 0x7fe3e80ced30 2023-10-11T06:12:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.179+0000 7fe3e77fe640 1 -- 172.21.15.84:0/2175072134 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800932913 0 0) 0x7fe3e8071380 con 0x7fe3e815a0b0 2023-10-11T06:12:45.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.179+0000 7fe3ecfb7640 1 --2- 172.21.15.84:0/2175072134 >> v2:172.21.15.99:3300/0 conn(0x7fe3e80c8d00 0x7fe3e806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:45.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.180+0000 7fe3dffff640 1 --2- 172.21.15.84:0/2175072134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e80cb5a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8070700 tx=0x7fe3d002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268bc8d82c2f4146 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:45.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.180+0000 7fe3e77fe640 1 -- 172.21.15.84:0/2175072134 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00317e0 con 0x7fe3e80ced30 2023-10-11T06:12:45.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.180+0000 7fe3e77fe640 1 -- 172.21.15.84:0/2175072134 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0031940 con 0x7fe3e80ced30 2023-10-11T06:12:45.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.180+0000 7fe3e77fe640 1 -- 172.21.15.84:0/2175072134 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0031cd0 con 0x7fe3e80ced30 2023-10-11T06:12:45.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 -- 172.21.15.84:0/2175072134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 msgr2=0x7fe3e80cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/2175072134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e80cb5a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8070700 tx=0x7fe3d002b880 comp rx=0 tx=0).stop 2023-10-11T06:12:45.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 -- 172.21.15.84:0/2175072134 shutdown_connections 2023-10-11T06:12:45.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/2175072134 >> v2:172.21.15.99:3300/0 conn(0x7fe3e80c8d00 0x7fe3e806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/2175072134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e80cb5a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 -- 172.21.15.84:0/2175072134 >> 172.21.15.84:0/2175072134 conn(0x7fe3e8069c70 msgr2=0x7fe3e806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:45.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 -- 172.21.15.84:0/2175072134 shutdown_connections 2023-10-11T06:12:45.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.181+0000 7fe3eea1a640 1 -- 172.21.15.84:0/2175072134 wait complete. 2023-10-11T06:12:45.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.183+0000 7fe3eea1a640 1 Processor -- start 2023-10-11T06:12:45.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.183+0000 7fe3eea1a640 1 -- start start 2023-10-11T06:12:45.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3eea1a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e80c8d00 0x7fe3e81f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3eea1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e81f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3eea1a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e815a0b0 0x7fe3e81ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3eea1a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80c7720 con 0x7fe3e80ced30 2023-10-11T06:12:45.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3eea1a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e81f1310 con 0x7fe3e815a0b0 2023-10-11T06:12:45.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3eea1a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80c7440 con 0x7fe3e80c8d00 2023-10-11T06:12:45.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e81f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e81f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49678/0 (socket says 172.21.15.84:49678) 2023-10-11T06:12:45.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.184+0000 7fe3dffff640 1 -- 172.21.15.84:0/112646830 learned_addr learned my addr 172.21.15.84:0/112646830 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:45.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3ecfb7640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e815a0b0 0x7fe3e81ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3e7fff640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e80c8d00 0x7fe3e81f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3dffff640 1 -- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e80c8d00 msgr2=0x7fe3e81f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3dffff640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e80c8d00 0x7fe3e81f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3dffff640 1 -- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e815a0b0 msgr2=0x7fe3e81ef9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3dffff640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e815a0b0 0x7fe3e81ef9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3dffff640 1 -- 172.21.15.84:0/112646830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00094d0 con 0x7fe3e80ced30 2023-10-11T06:12:45.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3dffff640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e81f3520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0034040 tx=0x7fe3d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:45.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3e57fa640 1 -- 172.21.15.84:0/112646830 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0004230 con 0x7fe3e80ced30 2023-10-11T06:12:45.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3e57fa640 1 -- 172.21.15.84:0/112646830 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0004390 con 0x7fe3e80ced30 2023-10-11T06:12:45.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3e57fa640 1 -- 172.21.15.84:0/112646830 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0003850 con 0x7fe3e80ced30 2023-10-11T06:12:45.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.185+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e81f0cc0 con 0x7fe3e80ced30 2023-10-11T06:12:45.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.186+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e81eff30 con 0x7fe3e80ced30 2023-10-11T06:12:45.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.187+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e81f0b00 con 0x7fe3e80ced30 2023-10-11T06:12:45.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.187+0000 7fe3e57fa640 1 -- 172.21.15.84:0/112646830 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe3d0043070 con 0x7fe3e80ced30 2023-10-11T06:12:45.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.188+0000 7fe3e57fa640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe3c806a830 0x7fe3c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.188+0000 7fe3e57fa640 1 -- 172.21.15.84:0/112646830 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3d00aaf00 con 0x7fe3e80ced30 2023-10-11T06:12:45.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.189+0000 7fe3e7fff640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe3c806a830 0x7fe3c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.189+0000 7fe3e7fff640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe3c806a830 0x7fe3c806cce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8000f60 tx=0x7fe3d8007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:45.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.193+0000 7fe3e57fa640 1 -- 172.21.15.84:0/112646830 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe3d0076180 con 0x7fe3e80ced30 2023-10-11T06:12:45.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.452+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe3e81f1480 con 0x7fe3e80ced30 2023-10-11T06:12:45.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.460+0000 7fe3e57fa640 1 -- 172.21.15.84:0/112646830 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe3d00797a0 con 0x7fe3e80ced30 2023-10-11T06:12:45.461 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:12:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe3c806a830 msgr2=0x7fe3c806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe3c806a830 0x7fe3c806cce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8000f60 tx=0x7fe3d8007c00 comp rx=0 tx=0).stop 2023-10-11T06:12:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 msgr2=0x7fe3e81f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e81f3520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0034040 tx=0x7fe3d00046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 shutdown_connections 2023-10-11T06:12:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe3c806a830 0x7fe3c806cce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e815a0b0 0x7fe3e81ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3e80ced30 0x7fe3e81f3520 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 --2- 172.21.15.84:0/112646830 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e80c8d00 0x7fe3e81f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 >> 172.21.15.84:0/112646830 conn(0x7fe3e8069c70 msgr2=0x7fe3e80c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 shutdown_connections 2023-10-11T06:12:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.462+0000 7fe3eea1a640 1 -- 172.21.15.84:0/112646830 wait complete. 2023-10-11T06:12:45.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:12:45.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.697+0000 7f4e8fc87640 1 Processor -- start 2023-10-11T06:12:45.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.697+0000 7f4e8fc87640 1 -- start start 2023-10-11T06:12:45.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8fc87640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e8815c290 0x7f4e880af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8fc87640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4e880d25e0 0x7f4e880d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8fc87640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e8806b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8fc87640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e880d2a60 con 0x7f4e8815c290 2023-10-11T06:12:45.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8fc87640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e880d0e90 con 0x7f4e880d25e0 2023-10-11T06:12:45.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8fc87640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8806cd10 con 0x7f4e880d0a50 2023-10-11T06:12:45.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e8815c290 0x7f4e880af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e8815c290 0x7f4e880af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49682/0 (socket says 172.21.15.84:49682) 2023-10-11T06:12:45.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 -- 172.21.15.84:0/2125420384 learned_addr learned my addr 172.21.15.84:0/2125420384 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:45.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 -- 172.21.15.84:0/2125420384 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 msgr2=0x7f4e8806b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:45.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 --2- 172.21.15.84:0/2125420384 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e8806b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 -- 172.21.15.84:0/2125420384 >> v2:172.21.15.99:3300/0 conn(0x7f4e880d25e0 msgr2=0x7f4e880d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:45.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 --2- 172.21.15.84:0/2125420384 >> v2:172.21.15.99:3300/0 conn(0x7f4e880d25e0 0x7f4e880d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.699+0000 7f4e8da23640 1 -- 172.21.15.84:0/2125420384 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880afb60 con 0x7f4e8815c290 2023-10-11T06:12:45.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.700+0000 7f4e8da23640 1 --2- 172.21.15.84:0/2125420384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e8815c290 0x7f4e880af4d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4e880d0380 tx=0x7f4e8402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f997099c7354b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:45.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.701+0000 7f4e8ca21640 1 -- 172.21.15.84:0/2125420384 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e840317e0 con 0x7f4e8815c290 2023-10-11T06:12:45.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.701+0000 7f4e8ca21640 1 -- 172.21.15.84:0/2125420384 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e84031940 con 0x7f4e8815c290 2023-10-11T06:12:45.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.701+0000 7f4e8ca21640 1 -- 172.21.15.84:0/2125420384 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84031cd0 con 0x7f4e8815c290 2023-10-11T06:12:45.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.701+0000 7f4e8fc87640 1 -- 172.21.15.84:0/2125420384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e8815c290 msgr2=0x7f4e880af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.701+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/2125420384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e8815c290 0x7f4e880af4d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4e880d0380 tx=0x7f4e8402b880 comp rx=0 tx=0).stop 2023-10-11T06:12:45.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.702+0000 7f4e8fc87640 1 -- 172.21.15.84:0/2125420384 shutdown_connections 2023-10-11T06:12:45.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.702+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/2125420384 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e8806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.702+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/2125420384 >> v2:172.21.15.99:3300/0 conn(0x7f4e880d25e0 0x7f4e880d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.702+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/2125420384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e8815c290 0x7f4e880af4d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.702+0000 7f4e8fc87640 1 -- 172.21.15.84:0/2125420384 >> 172.21.15.84:0/2125420384 conn(0x7f4e88071cc0 msgr2=0x7f4e8809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:45.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.702+0000 7f4e8fc87640 1 -- 172.21.15.84:0/2125420384 shutdown_connections 2023-10-11T06:12:45.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.702+0000 7f4e8fc87640 1 -- 172.21.15.84:0/2125420384 wait complete. 2023-10-11T06:12:45.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.703+0000 7f4e8fc87640 1 Processor -- start 2023-10-11T06:12:45.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.704+0000 7f4e8fc87640 1 -- start start 2023-10-11T06:12:45.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8fc87640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e880b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8fc87640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e880d25e0 0x7f4e88155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8fc87640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e8815c290 0x7f4e8815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8fc87640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880b6840 con 0x7f4e880d25e0 2023-10-11T06:12:45.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8fc87640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880b5c30 con 0x7f4e8815c290 2023-10-11T06:12:45.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8fc87640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880b5f10 con 0x7f4e880d0a50 2023-10-11T06:12:45.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e880d25e0 0x7f4e88155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e880d25e0 0x7f4e88155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49690/0 (socket says 172.21.15.84:49690) 2023-10-11T06:12:45.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 -- 172.21.15.84:0/3785957827 learned_addr learned my addr 172.21.15.84:0/3785957827 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:45.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8da23640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e880b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8e224640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e8815c290 0x7f4e8815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 -- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 msgr2=0x7f4e880b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e880b8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 -- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e8815c290 msgr2=0x7f4e8815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e8815c290 0x7f4e8815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8d222640 1 -- 172.21.15.84:0/3785957827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e840094d0 con 0x7f4e880d25e0 2023-10-11T06:12:45.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8da23640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e880b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:45.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.705+0000 7f4e8e224640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e8815c290 0x7f4e8815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:45.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.706+0000 7f4e8d222640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e880d25e0 0x7f4e88155450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b4a0 tx=0x7f4e7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:45.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.706+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/3785957827 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78010610 con 0x7f4e880d25e0 2023-10-11T06:12:45.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.706+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/3785957827 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e78010770 con 0x7f4e880d25e0 2023-10-11T06:12:45.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.706+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/3785957827 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78010b20 con 0x7f4e880d25e0 2023-10-11T06:12:45.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.706+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e880b61f0 con 0x7f4e880d25e0 2023-10-11T06:12:45.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.706+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e88155990 con 0x7f4e880d25e0 2023-10-11T06:12:45.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.707+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e880b69f0 con 0x7f4e880d25e0 2023-10-11T06:12:45.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.708+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/3785957827 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4e78010c80 con 0x7f4e880d25e0 2023-10-11T06:12:45.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.709+0000 7f4e7e7fc640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4e6c06a830 0x7f4e6c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:45.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.709+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/3785957827 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e78088d90 con 0x7f4e880d25e0 2023-10-11T06:12:45.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.709+0000 7f4e8da23640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4e6c06a830 0x7f4e6c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:45.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.710+0000 7f4e8da23640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4e6c06a830 0x7f4e6c06cce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e84036000 tx=0x7f4e8403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:45.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.713+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/3785957827 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4e880b69f0 con 0x7f4e880d25e0 2023-10-11T06:12:45.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.942+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e88073ed0 con 0x7f4e880d25e0 2023-10-11T06:12:45.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.943+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/3785957827 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f4e7801e020 con 0x7f4e880d25e0 2023-10-11T06:12:45.945 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:45.945 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:12:40.002715+0000","modified":"2023-10-11T06:12:43.012925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5334":{"gid":5334,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/2172719017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2172719017},{"type":"v1","addr":"172.21.15.99:6835","nonce":2172719017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T06:12:45.945 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 15 2023-10-11T06:12:45.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4e6c06a830 msgr2=0x7f4e6c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4e6c06a830 0x7f4e6c06cce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e84036000 tx=0x7f4e8403d040 comp rx=0 tx=0).stop 2023-10-11T06:12:45.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e880d25e0 msgr2=0x7f4e88155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:45.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e880d25e0 0x7f4e88155450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b4a0 tx=0x7f4e7800b960 comp rx=0 tx=0).stop 2023-10-11T06:12:45.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 shutdown_connections 2023-10-11T06:12:45.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4e6c06a830 0x7f4e6c06cce0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e8815c290 0x7f4e8815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e880d25e0 0x7f4e88155450 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 --2- 172.21.15.84:0/3785957827 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e880d0a50 0x7f4e880b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:45.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 >> 172.21.15.84:0/3785957827 conn(0x7f4e88071cc0 msgr2=0x7f4e880b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:45.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 shutdown_connections 2023-10-11T06:12:45.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:45.946+0000 7f4e8fc87640 1 -- 172.21.15.84:0/3785957827 wait complete. 2023-10-11T06:12:46.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T06:12:40.002715+0000', 'modified': '2023-10-11T06:12:43.012925+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5334}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5334': {'gid': 5334, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.99:6835/2172719017', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 2172719017}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 2172719017}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T06:12:46.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T06:12:46.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-11T06:12:46.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.594+0000 7fbee5ad5640 1 Processor -- start 2023-10-11T06:12:46.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.594+0000 7fbee5ad5640 1 -- start start 2023-10-11T06:12:46.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.596+0000 7fbee5ad5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbee015d4f0 0x7fbee00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.596+0000 7fbee5ad5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.596+0000 7fbee5ad5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee0071a90 con 0x7fbee00c9190 2023-10-11T06:12:46.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.596+0000 7fbee5ad5640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee0072310 con 0x7fbee015d4f0 2023-10-11T06:12:46.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.596+0000 7fbee5ad5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0070280 con 0x7fbee00ca300 2023-10-11T06:12:46.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.596+0000 7fbed7fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.596+0000 7fbedf7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbee015d4f0 0x7fbee00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.597+0000 7fbed7fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:49082/0 (socket says 172.21.15.84:49082) 2023-10-11T06:12:46.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.597+0000 7fbed7fff640 1 -- 172.21.15.84:0/954654180 learned_addr learned my addr 172.21.15.84:0/954654180 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:46.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.597+0000 7fbed7fff640 1 -- 172.21.15.84:0/954654180 >> v2:172.21.15.99:3300/0 conn(0x7fbee015d4f0 msgr2=0x7fbee00cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.597+0000 7fbed7fff640 1 --2- 172.21.15.84:0/954654180 >> v2:172.21.15.99:3300/0 conn(0x7fbee015d4f0 0x7fbee00cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.597+0000 7fbed7fff640 1 -- 172.21.15.84:0/954654180 >> v1:172.21.15.99:6789/0 conn(0x7fbee00c9190 legacy=0x7fbee015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.597+0000 7fbed7fff640 1 -- 172.21.15.84:0/954654180 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0071cc0 con 0x7fbee00ca300 2023-10-11T06:12:46.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.597+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/954654180 >> v2:172.21.15.99:3300/0 conn(0x7fbee015d4f0 0x7fbee00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:46.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.599+0000 7fbed7fff640 1 --2- 172.21.15.84:0/954654180 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c8c50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009870 tx=0x7fbec802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0ad84c73d404c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.599+0000 7fbedeffd640 1 -- 172.21.15.84:0/954654180 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8030840 con 0x7fbee00ca300 2023-10-11T06:12:46.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbedeffd640 1 -- 172.21.15.84:0/954654180 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec80309a0 con 0x7fbee00ca300 2023-10-11T06:12:46.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbedeffd640 1 -- 172.21.15.84:0/954654180 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8030d30 con 0x7fbee00ca300 2023-10-11T06:12:46.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbee5ad5640 1 -- 172.21.15.84:0/954654180 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 msgr2=0x7fbee00c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/954654180 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c8c50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009870 tx=0x7fbec802b980 comp rx=0 tx=0).stop 2023-10-11T06:12:46.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbee5ad5640 1 -- 172.21.15.84:0/954654180 shutdown_connections 2023-10-11T06:12:46.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/954654180 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c8c50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/954654180 >> v2:172.21.15.99:3300/0 conn(0x7fbee015d4f0 0x7fbee00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.600+0000 7fbee5ad5640 1 -- 172.21.15.84:0/954654180 >> 172.21.15.84:0/954654180 conn(0x7fbee0069c70 msgr2=0x7fbee006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:46.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.601+0000 7fbee5ad5640 1 -- 172.21.15.84:0/954654180 shutdown_connections 2023-10-11T06:12:46.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.601+0000 7fbee5ad5640 1 -- 172.21.15.84:0/954654180 wait complete. 2023-10-11T06:12:46.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.603+0000 7fbee5ad5640 1 Processor -- start 2023-10-11T06:12:46.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.603+0000 7fbee5ad5640 1 -- start start 2023-10-11T06:12:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbee5ad5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00c9190 0x7fbee014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbee5ad5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbee5ad5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee015d4f0 0x7fbee00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbee5ad5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee014b010 con 0x7fbee00c9190 2023-10-11T06:12:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbee5ad5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee01591b0 con 0x7fbee015d4f0 2023-10-11T06:12:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbee5ad5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0159490 con 0x7fbee00ca300 2023-10-11T06:12:46.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbedf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00c9190 0x7fbee014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbedf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00c9190 0x7fbee014aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49700/0 (socket says 172.21.15.84:49700) 2023-10-11T06:12:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.604+0000 7fbedf7fe640 1 -- 172.21.15.84:0/994902006 learned_addr learned my addr 172.21.15.84:0/994902006 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbedffff640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee015d4f0 0x7fbee00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbed7fff640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbedf7fe640 1 -- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 msgr2=0x7fbee00c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbedf7fe640 1 -- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee015d4f0 msgr2=0x7fbee00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee015d4f0 0x7fbee00b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbedf7fe640 1 -- 172.21.15.84:0/994902006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00093a0 con 0x7fbee00c9190 2023-10-11T06:12:46.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.605+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00c9190 0x7fbee014aad0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9b0 tx=0x7fbed000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.606+0000 7fbedcff9640 1 -- 172.21.15.84:0/994902006 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0005990 con 0x7fbee00c9190 2023-10-11T06:12:46.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.606+0000 7fbedcff9640 1 -- 172.21.15.84:0/994902006 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0005af0 con 0x7fbee00c9190 2023-10-11T06:12:46.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.606+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec80094d0 con 0x7fbee00c9190 2023-10-11T06:12:46.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.606+0000 7fbedcff9640 1 -- 172.21.15.84:0/994902006 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0005e80 con 0x7fbee00c9190 2023-10-11T06:12:46.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.606+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee0159a50 con 0x7fbee00c9190 2023-10-11T06:12:46.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.608+0000 7fbedcff9640 1 -- 172.21.15.84:0/994902006 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbed000c9a0 con 0x7fbee00c9190 2023-10-11T06:12:46.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.610+0000 7fbedcff9640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbec006a830 0x7fbec006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.610+0000 7fbedcff9640 1 -- 172.21.15.84:0/994902006 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbed0089480 con 0x7fbee00c9190 2023-10-11T06:12:46.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.610+0000 7fbed7fff640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbec006a830 0x7fbec006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.611+0000 7fbed7fff640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbec006a830 0x7fbec006cce0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbec8036000 tx=0x7fbec803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.611+0000 7fbee8641740 1 Processor -- start 2023-10-11T06:12:46.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.611+0000 7fbee8641740 1 -- start start 2023-10-11T06:12:46.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047e8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbee8641740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b0047e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbee8641740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b004794250 con 0x55b0047b8d40 2023-10-11T06:12:46.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b004728f80 con 0x55b0047e14f0 2023-10-11T06:12:46.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b004726710 con 0x55b0047e1120 2023-10-11T06:12:46.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b0047e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b0047e8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49716/0 (socket says 172.21.15.84:49716) 2023-10-11T06:12:46.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 -- 172.21.15.84:0/11308857 learned_addr learned my addr 172.21.15.84:0/11308857 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbed7fff640 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 -- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 msgr2=0x55b0047b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 -- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 msgr2=0x55b0047e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbedffff640 1 -- 172.21.15.84:0/11308857 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0046e9690 con 0x55b0047b8d40 2023-10-11T06:12:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.612+0000 7fbed7fff640 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.613+0000 7fbedffff640 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b0047e8bd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbed8001c30 tx=0x7fbed80020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de96fa4519b95d67 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.613+0000 7fbed57fa640 1 -- 172.21.15.84:0/11308857 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003b80 con 0x55b0047b8d40 2023-10-11T06:12:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.613+0000 7fbed57fa640 1 -- 172.21.15.84:0/11308857 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8003ce0 con 0x55b0047b8d40 2023-10-11T06:12:46.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.613+0000 7fbed57fa640 1 -- 172.21.15.84:0/11308857 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800eaa0 con 0x55b0047b8d40 2023-10-11T06:12:46.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.613+0000 7fbee8641740 1 -- 172.21.15.84:0/11308857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 msgr2=0x55b0047e8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.613+0000 7fbee8641740 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b0047e8bd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbed8001c30 tx=0x7fbed80020f0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 -- 172.21.15.84:0/11308857 shutdown_connections 2023-10-11T06:12:46.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b0047e8bd0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 --2- 172.21.15.84:0/11308857 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 -- 172.21.15.84:0/11308857 >> 172.21.15.84:0/11308857 conn(0x55b0047bab10 msgr2=0x55b004803690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 -- 172.21.15.84:0/11308857 shutdown_connections 2023-10-11T06:12:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 -- 172.21.15.84:0/11308857 wait complete. 2023-10-11T06:12:46.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.614+0000 7fbee8641740 1 -- start start 2023-10-11T06:12:46.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.615+0000 7fbee8641740 1 Processor -- start 2023-10-11T06:12:46.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.615+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbee8641740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e14f0 0x55b0047e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbee8641740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b004722000 con 0x55b0047e14f0 2023-10-11T06:12:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0046c37d0 con 0x55b0047b8d40 2023-10-11T06:12:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0046c3470 con 0x55b0047e1120 2023-10-11T06:12:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbee8641740 10 client.0 _ll_register_callbacks cb 0x55b0047121b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T06:12:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e14f0 0x55b0047e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e14f0 0x55b0047e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49730/0 (socket says 172.21.15.84:49730) 2023-10-11T06:12:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 -- 172.21.15.84:0/2081796493 learned_addr learned my addr 172.21.15.84:0/2081796493 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 -- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 msgr2=0x55b0047e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbed7fff640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047e75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 -- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047b8d40 msgr2=0x55b004806d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedffff640 1 -- 172.21.15.84:0/2081796493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed800f040 con 0x55b0047e14f0 2023-10-11T06:12:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.616+0000 7fbed7fff640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbedffff640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e14f0 0x55b0047e7b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbed80012e0 tx=0x7fbed8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab92ed7847ffc9de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbebe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-11T06:12:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbebe7fc640 1 -- 172.21.15.84:0/2081796493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800ec70 con 0x55b0047e14f0 2023-10-11T06:12:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbebe7fc640 1 -- 172.21.15.84:0/2081796493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbed800edd0 con 0x55b0047e14f0 2023-10-11T06:12:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b00474c550 con 0x55b0047e14f0 2023-10-11T06:12:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbebe7fc640 1 -- 172.21.15.84:0/2081796493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed80050d0 con 0x55b0047e14f0 2023-10-11T06:12:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbee8641740 10 client.5490 fetch_fsmap learned FSMap version 15 2023-10-11T06:12:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55b004720680 con 0x55b0047e14f0 2023-10-11T06:12:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.617+0000 7fbebe7fc640 1 -- 172.21.15.84:0/2081796493 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b00474c550 con 0x55b0047e14f0 2023-10-11T06:12:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbebe7fc640 1 -- 172.21.15.84:0/2081796493 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 15) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbed8005230 con 0x55b0047e14f0 2023-10-11T06:12:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbee8641740 10 client.5490 fetch_fsmap finished waiting for FSMap version 15 2023-10-11T06:12:46.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbee8641740 10 client.5490 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-11T06:12:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbee8641740 20 client.5490 populate_metadata read hostname 'smithi084' 2023-10-11T06:12:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbee8641740 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b0047e1f60 0x55b0047e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbee8641740 4 client.5490 mds_command: new command op to 5334 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T06:12:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 --> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b004714b50 con 0x55b0047e1f60 2023-10-11T06:12:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.618+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b0047e1f60 0x55b0047e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.619+0000 7fbebe7fc640 10 client.5490 ms_handle_connect on v2:172.21.15.99:6834/2172719017 2023-10-11T06:12:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.619+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b0047e1f60 0x55b0047e2330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3cad9aba6086346 server_cookie=ff4c04ccb2722244 in_seq=0 out_seq=0 2023-10-11T06:12:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.621+0000 7fbebe7fc640 1 -- 172.21.15.84:0/2081796493 <== mds.0 v2:172.21.15.99:6834/2172719017 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b004714b50 con 0x55b0047e1f60 2023-10-11T06:12:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.621+0000 7fbebe7fc640 10 client.5490 handle_command_reply: tid=0 2023-10-11T06:12:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.621+0000 7fbee8641740 1 client.5490 shutdown 2023-10-11T06:12:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.621+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e14f0 msgr2=0x55b0047e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.621+0000 7fbee8641740 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e14f0 0x55b0047e7b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbed80012e0 tx=0x7fbed8001250 comp rx=0 tx=0).stop 2023-10-11T06:12:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 shutdown_connections 2023-10-11T06:12:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e14f0 0x55b0047e7b00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b0047e1f60 0x55b0047e2330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e1120 0x55b0047e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 --2- 172.21.15.84:0/2081796493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbebe7fc640 0 client.5490 ms_handle_reset on v2:172.21.15.99:6834/2172719017 2023-10-11T06:12:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 >> 172.21.15.84:0/2081796493 conn(0x55b0047bab10 msgr2=0x55b0047b7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 shutdown_connections 2023-10-11T06:12:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 1 -- 172.21.15.84:0/2081796493 wait complete. 2023-10-11T06:12:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.622+0000 7fbee8641740 20 client.5490 trim_cache size 0 max 16384 2023-10-11T06:12:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.638+0000 7fbee8641740 1 Processor -- start 2023-10-11T06:12:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.638+0000 7fbee8641740 1 -- start start 2023-10-11T06:12:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbee8641740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e1120 0x55b0047b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e14f0 0x55b0047b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbee8641740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b004489f20 con 0x55b0047b8d40 2023-10-11T06:12:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0047ba400 con 0x55b0047e1120 2023-10-11T06:12:46.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0047db880 con 0x55b0047e14f0 2023-10-11T06:12:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbedf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbedf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49738/0 (socket says 172.21.15.84:49738) 2023-10-11T06:12:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbedf7fe640 1 -- 172.21.15.84:0/2848732818 learned_addr learned my addr 172.21.15.84:0/2848732818 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbedf7fe640 1 -- 172.21.15.84:0/2848732818 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e14f0 msgr2=0x55b0047b81f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-11T06:12:46.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/2848732818 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e14f0 0x55b0047b81f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbedf7fe640 1 -- 172.21.15.84:0/2848732818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e1120 msgr2=0x55b0047b7880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-11T06:12:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.639+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/2848732818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e1120 0x55b0047b7880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.640+0000 7fbedf7fe640 1 -- 172.21.15.84:0/2848732818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0046c29e0 con 0x55b0047b8d40 2023-10-11T06:12:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.640+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/2848732818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b004806d50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbed00097e0 tx=0x7fbed0009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d3a92f2ac3a33a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.640+0000 7fbed5ffb640 1 -- 172.21.15.84:0/2848732818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0087900 con 0x55b0047b8d40 2023-10-11T06:12:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.640+0000 7fbed5ffb640 1 -- 172.21.15.84:0/2848732818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0056700 con 0x55b0047b8d40 2023-10-11T06:12:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.640+0000 7fbed5ffb640 1 -- 172.21.15.84:0/2848732818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed005e0b0 con 0x55b0047b8d40 2023-10-11T06:12:46.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 -- 172.21.15.84:0/2848732818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 msgr2=0x55b004806d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 --2- 172.21.15.84:0/2848732818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b004806d50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbed00097e0 tx=0x7fbed0009770 comp rx=0 tx=0).stop 2023-10-11T06:12:46.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 -- 172.21.15.84:0/2848732818 shutdown_connections 2023-10-11T06:12:46.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 --2- 172.21.15.84:0/2848732818 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047e14f0 0x55b0047b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 --2- 172.21.15.84:0/2848732818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e1120 0x55b0047b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 --2- 172.21.15.84:0/2848732818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047b8d40 0x55b004806d50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 -- 172.21.15.84:0/2848732818 >> 172.21.15.84:0/2848732818 conn(0x55b0047bab10 msgr2=0x55b0047e78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:46.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 -- 172.21.15.84:0/2848732818 shutdown_connections 2023-10-11T06:12:46.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 -- 172.21.15.84:0/2848732818 wait complete. 2023-10-11T06:12:46.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.641+0000 7fbee8641740 1 -- start start 2023-10-11T06:12:46.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.642+0000 7fbee8641740 1 Processor -- start 2023-10-11T06:12:46.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.642+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047b8d40 0x55b0047beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbee8641740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e1120 0x55b00486a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbee8641740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047eba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbee8641740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0047da190 con 0x55b0047e1120 2023-10-11T06:12:46.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0047ebf70 con 0x55b0047e14f0 2023-10-11T06:12:46.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbee8641740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0047dc830 con 0x55b0047b8d40 2023-10-11T06:12:46.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbee8641740 10 client.0 _ll_register_callbacks cb 0x7fbed8001640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T06:12:46.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e1120 0x55b00486a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e1120 0x55b00486a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49748/0 (socket says 172.21.15.84:49748) 2023-10-11T06:12:46.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 -- 172.21.15.84:0/1653634707 learned_addr learned my addr 172.21.15.84:0/1653634707 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:46.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047b8d40 0x55b0047beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbedffff640 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 -- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047b8d40 msgr2=0x55b0047beba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047b8d40 0x55b0047beba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 -- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 msgr2=0x55b0047eba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.643+0000 7fbed7fff640 1 -- 172.21.15.84:0/1653634707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee0159a50 con 0x55b0047e1120 2023-10-11T06:12:46.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbed7fff640 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e1120 0x55b00486a6d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009990 tx=0x7fbec802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6e245a7473f24 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbebe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-11T06:12:46.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbebe7fc640 1 -- 172.21.15.84:0/1653634707 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8052070 con 0x55b0047e1120 2023-10-11T06:12:46.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbed8003ce0 con 0x55b0047e1120 2023-10-11T06:12:46.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbebe7fc640 1 -- 172.21.15.84:0/1653634707 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbec803e070 con 0x55b0047e1120 2023-10-11T06:12:46.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbebe7fc640 1 -- 172.21.15.84:0/1653634707 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec8048050 con 0x55b0047e1120 2023-10-11T06:12:46.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbee8641740 10 client.5496 fetch_fsmap learned FSMap version 15 2023-10-11T06:12:46.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55b0046e7ef0 con 0x55b0047e1120 2023-10-11T06:12:46.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.644+0000 7fbebe7fc640 1 -- 172.21.15.84:0/1653634707 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8003ce0 con 0x55b0047e1120 2023-10-11T06:12:46.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbebe7fc640 1 -- 172.21.15.84:0/1653634707 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 15) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbec8043810 con 0x55b0047e1120 2023-10-11T06:12:46.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbee8641740 10 client.5496 fetch_fsmap finished waiting for FSMap version 15 2023-10-11T06:12:46.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbee8641740 10 client.5496 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-11T06:12:46.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbee8641740 20 client.5496 populate_metadata read hostname 'smithi084' 2023-10-11T06:12:46.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbee8641740 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b00486d630 0x55b0047f9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbee8641740 4 client.5496 mds_command: new command op to 5334 tid=0[{"prefix": "status"}] 2023-10-11T06:12:46.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 --> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b00471b080 con 0x55b00486d630 2023-10-11T06:12:46.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.645+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b00486d630 0x55b0047f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.646+0000 7fbedf7fe640 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b00486d630 0x55b0047f9020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e82b439b40b217 server_cookie=9e0e5f768b03eb12 in_seq=0 out_seq=0 2023-10-11T06:12:46.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.646+0000 7fbebe7fc640 10 client.5496 ms_handle_connect on v2:172.21.15.99:6834/2172719017 2023-10-11T06:12:46.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbebe7fc640 1 -- 172.21.15.84:0/1653634707 <== mds.0 v2:172.21.15.99:6834/2172719017 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b00471b080 con 0x55b00486d630 2023-10-11T06:12:46.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbebe7fc640 10 client.5496 handle_command_reply: tid=0 2023-10-11T06:12:46.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 client.5496 shutdown 2023-10-11T06:12:46.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e1120 msgr2=0x55b00486a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e1120 0x55b00486a6d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009990 tx=0x7fbec802e000 comp rx=0 tx=0).stop 2023-10-11T06:12:46.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 shutdown_connections 2023-10-11T06:12:46.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0047e14f0 0x55b0047eba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b0047e1120 0x55b00486a6d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:6834/2172719017,v1:172.21.15.99:6835/2172719017] conn(0x55b00486d630 0x55b0047f9020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 --2- 172.21.15.84:0/1653634707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55b0047b8d40 0x55b0047beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbebe7fc640 0 client.5496 ms_handle_reset on v2:172.21.15.99:6834/2172719017 2023-10-11T06:12:46.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 >> 172.21.15.84:0/1653634707 conn(0x55b0047bab10 msgr2=0x55b0047e5820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:46.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 shutdown_connections 2023-10-11T06:12:46.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 1 -- 172.21.15.84:0/1653634707 wait complete. 2023-10-11T06:12:46.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.647+0000 7fbee8641740 20 client.5496 trim_cache size 0 max 16384 2023-10-11T06:12:46.661 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:12:46.661 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c84dc637-9184-44da-bf67-6cb4224adc5a", 2023-10-11T06:12:46.662 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-11T06:12:46.662 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5334, 2023-10-11T06:12:46.662 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-11T06:12:46.662 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-11T06:12:46.662 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-11T06:12:46.663 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6397322140000004, 2023-10-11T06:12:46.663 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 15, 2023-10-11T06:12:46.663 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 38, 2023-10-11T06:12:46.663 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 38, 2023-10-11T06:12:46.663 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 15.207249299000001 2023-10-11T06:12:46.663 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:12:46.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbec006a830 msgr2=0x7fbec006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbec006a830 0x7fbec006cce0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbec8036000 tx=0x7fbec803b040 comp rx=0 tx=0).stop 2023-10-11T06:12:46.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00c9190 msgr2=0x7fbee014aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00c9190 0x7fbee014aad0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9b0 tx=0x7fbed000ee70 comp rx=0 tx=0).stop 2023-10-11T06:12:46.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 shutdown_connections 2023-10-11T06:12:46.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbec006a830 0x7fbec006cce0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee015d4f0 0x7fbee00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbee00ca300 0x7fbee00c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 --2- 172.21.15.84:0/994902006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee00c9190 0x7fbee014aad0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 >> 172.21.15.84:0/994902006 conn(0x7fbee0069c70 msgr2=0x7fbee00c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:46.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 shutdown_connections 2023-10-11T06:12:46.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.648+0000 7fbee5ad5640 1 -- 172.21.15.84:0/994902006 wait complete. 2023-10-11T06:12:46.667 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:12:46.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.877+0000 7f6efbfff640 1 Processor -- start 2023-10-11T06:12:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.878+0000 7f6efbfff640 1 -- start start 2023-10-11T06:12:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6efc0ceae0 0x7f6efc15a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efbfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc15da50 con 0x7f6efc0b3e70 2023-10-11T06:12:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efbfff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc0cad10 con 0x7f6efc06fd80 2023-10-11T06:12:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efbfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc15dca0 con 0x7f6efc0ceae0 2023-10-11T06:12:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efa7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6efc0ceae0 0x7f6efc15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efaffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6efc0b3e70 0x7f6efc0ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48270/0 (socket says 172.21.15.84:48270) 2023-10-11T06:12:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efa7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6efc0ceae0 0x7f6efc15a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49752/0 (socket says 172.21.15.84:49752) 2023-10-11T06:12:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.879+0000 7f6efaffd640 1 -- 172.21.15.84:0/1565984223 learned_addr learned my addr 172.21.15.84:0/1565984223 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.880+0000 7f6efa7fc640 1 -- 172.21.15.84:0/1565984223 >> v1:172.21.15.99:6789/0 conn(0x7f6efc06fd80 legacy=0x7f6efc15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.880+0000 7f6efa7fc640 1 -- 172.21.15.84:0/1565984223 >> v1:172.21.15.84:6789/0 conn(0x7f6efc0b3e70 legacy=0x7f6efc0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.880+0000 7f6efa7fc640 1 -- 172.21.15.84:0/1565984223 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc15df80 con 0x7f6efc0ceae0 2023-10-11T06:12:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.880+0000 7f6ef9ffb640 1 -- 172.21.15.84:0/1565984223 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078523402 0 0) 0x7f6efc15da50 con 0x7f6efc0b3e70 2023-10-11T06:12:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.881+0000 7f6efa7fc640 1 --2- 172.21.15.84:0/1565984223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc15a470 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00097a0 tx=0x7f6ef002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e82985565b5933a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.881+0000 7f6ef9ffb640 1 -- 172.21.15.84:0/1565984223 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0031830 con 0x7f6efc0ceae0 2023-10-11T06:12:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.881+0000 7f6ef9ffb640 1 -- 172.21.15.84:0/1565984223 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0031990 con 0x7f6efc0ceae0 2023-10-11T06:12:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.881+0000 7f6ef9ffb640 1 -- 172.21.15.84:0/1565984223 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0031d20 con 0x7f6efc0ceae0 2023-10-11T06:12:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.882+0000 7f6efbfff640 1 -- 172.21.15.84:0/1565984223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 msgr2=0x7f6efc15a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.882+0000 7f6efbfff640 1 --2- 172.21.15.84:0/1565984223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc15a470 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00097a0 tx=0x7f6ef002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.882+0000 7f6efbfff640 1 -- 172.21.15.84:0/1565984223 shutdown_connections 2023-10-11T06:12:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.882+0000 7f6efbfff640 1 --2- 172.21.15.84:0/1565984223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc15a470 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.882+0000 7f6efbfff640 1 -- 172.21.15.84:0/1565984223 >> 172.21.15.84:0/1565984223 conn(0x7f6efc069c70 msgr2=0x7f6efc06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.882+0000 7f6efbfff640 1 -- 172.21.15.84:0/1565984223 shutdown_connections 2023-10-11T06:12:46.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.882+0000 7f6efbfff640 1 -- 172.21.15.84:0/1565984223 wait complete. 2023-10-11T06:12:46.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.884+0000 7f6efbfff640 1 Processor -- start 2023-10-11T06:12:46.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.884+0000 7f6efbfff640 1 -- start start 2023-10-11T06:12:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efbfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6efc06fd80 0x7f6efc1f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc0b3e70 0x7f6efc0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efbfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c6430 con 0x7f6efc0ceae0 2023-10-11T06:12:46.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efbfff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c5510 con 0x7f6efc0b3e70 2023-10-11T06:12:46.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efbfff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c57b0 con 0x7f6efc06fd80 2023-10-11T06:12:46.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49762/0 (socket says 172.21.15.84:49762) 2023-10-11T06:12:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.885+0000 7f6efb7fe640 1 -- 172.21.15.84:0/327123653 learned_addr learned my addr 172.21.15.84:0/327123653 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efa7fc640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc0b3e70 0x7f6efc0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efaffd640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6efc06fd80 0x7f6efc1f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efb7fe640 1 -- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6efc06fd80 msgr2=0x7f6efc1f0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efb7fe640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6efc06fd80 0x7f6efc1f0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efb7fe640 1 -- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc0b3e70 msgr2=0x7f6efc0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efb7fe640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc0b3e70 0x7f6efc0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efb7fe640 1 -- 172.21.15.84:0/327123653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef0009400 con 0x7f6efc0ceae0 2023-10-11T06:12:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efa7fc640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc0b3e70 0x7f6efc0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efb7fe640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc158290 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c780 tx=0x7f6ef400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6ee3fff640 1 -- 172.21.15.84:0/327123653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4004b60 con 0x7f6efc0ceae0 2023-10-11T06:12:46.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6ee3fff640 1 -- 172.21.15.84:0/327123653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400ce90 con 0x7f6efc0ceae0 2023-10-11T06:12:46.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6ee3fff640 1 -- 172.21.15.84:0/327123653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400a850 con 0x7f6efc0ceae0 2023-10-11T06:12:46.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.886+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc0c5a90 con 0x7f6efc0ceae0 2023-10-11T06:12:46.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.887+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc0c6050 con 0x7f6efc0ceae0 2023-10-11T06:12:46.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.888+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc04e890 con 0x7f6efc0ceae0 2023-10-11T06:12:46.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.889+0000 7f6ee3fff640 1 -- 172.21.15.84:0/327123653 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6ef400a9b0 con 0x7f6efc0ceae0 2023-10-11T06:12:46.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.890+0000 7f6ee3fff640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ee806a830 0x7f6ee806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:46.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.890+0000 7f6ee3fff640 1 -- 172.21.15.84:0/327123653 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ef40891e0 con 0x7f6efc0ceae0 2023-10-11T06:12:46.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.890+0000 7f6efaffd640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ee806a830 0x7f6ee806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:46.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.890+0000 7f6efaffd640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ee806a830 0x7f6ee806cce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007fd0 tx=0x7f6eec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:46.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:46.894+0000 7f6ee3fff640 1 -- 172.21.15.84:0/327123653 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f6ef4054450 con 0x7f6efc0ceae0 2023-10-11T06:12:47.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.120+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6efc0cb510 con 0x7f6efc0ceae0 2023-10-11T06:12:47.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.121+0000 7f6ee3fff640 1 -- 172.21.15.84:0/327123653 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6ef4057a70 con 0x7f6efc0ceae0 2023-10-11T06:12:47.122 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:12:47.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ee806a830 msgr2=0x7f6ee806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:47.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ee806a830 0x7f6ee806cce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007fd0 tx=0x7f6eec009040 comp rx=0 tx=0).stop 2023-10-11T06:12:47.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 msgr2=0x7f6efc158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:47.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc158290 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c780 tx=0x7f6ef400cc40 comp rx=0 tx=0).stop 2023-10-11T06:12:47.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 shutdown_connections 2023-10-11T06:12:47.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6ee806a830 0x7f6ee806cce0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:47.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0ceae0 0x7f6efc158290 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:47.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc0b3e70 0x7f6efc0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:47.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 --2- 172.21.15.84:0/327123653 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6efc06fd80 0x7f6efc1f0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:47.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.123+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 >> 172.21.15.84:0/327123653 conn(0x7f6efc069c70 msgr2=0x7f6efc0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:47.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.124+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 shutdown_connections 2023-10-11T06:12:47.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:47.124+0000 7f6efbfff640 1 -- 172.21.15.84:0/327123653 wait complete. 2023-10-11T06:12:47.141 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:12:47.141 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:12:47.142 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:12:47.142 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T06:12:47.142 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:47.142 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:12:47.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:12:47.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:12:47.143 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:12:47.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:47.144 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:12:47.197 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:12:47.197 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:12:47.197 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:12:47.197 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:12:47.198 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:12:47.198 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:12:47.198 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:12:47.198 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:12:47.198 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:12:47.199 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:12:47.199 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:12:47.199 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:12:47.199 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:12:47.199 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:12:47.200 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:12:47.200 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:12:47.200 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:12:47.200 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42310sec preferred_lft 42310sec 2023-10-11T06:12:47.201 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:12:47.201 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:12:47.201 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:12:47.201 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff 2023-10-11T06:12:47.201 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:12:47.201 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:12:47.202 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1c33:d5ff:fea1:21f9/64 scope link 2023-10-11T06:12:47.202 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:12:47.202 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:12:47.202 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:12:47.202 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::d4ff:bcff:feac:c798/64 scope link 2023-10-11T06:12:47.203 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:12:47.203 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:12:47.203 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:12:47.203 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4088:59ff:feb7:538e/64 scope link 2023-10-11T06:12:47.203 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:12:47.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:47.205 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:12:47.291 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:12:47.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:47.296 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:12:47.352 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:12:47.352 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:12:47.353 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:12:47.353 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:12:47.353 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:12:47.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:47.354 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:12:47.355 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:12:47.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:12:47.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:12:47.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:12:47.356 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:12:47.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:47.553 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:12:47.603 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T06:12:47.604 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:12:47.604 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:12:47.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:47.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-11T06:12:47.699 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T06:12:47.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:12:47.700 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:47.755 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:12:47.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:12:47.756 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:47.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:12:47.898 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:12:47.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:12:47.994 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:12:47.997 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:12:47.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:47.999 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:12:48.054 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:12:48.055 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T06:12:48.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:12:48.137 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:12:48.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:12:48.207 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:12:48.209 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:12:48.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:48.210 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:12:48.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-10-11T06:12:48.246+0000 7f3b06e3c480 -1 init, newargv = 0x560921a49c70 newargc=15 2023-10-11T06:12:48.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[101943]: starting ceph client 2023-10-11T06:12:48.258 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[101943]: starting fuse 2023-10-11T06:12:49.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:12:49.460 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:12:49.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:12:49.549 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:12:49.552 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:12:49.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:49.554 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:12:49.607 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:12:49.607 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-10-11T06:12:49.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:12:49.608 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:12:49.609 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:12:49.609 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:12:49.609 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:12:49.610 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:12:49.610 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:12:49.611 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T06:12:49.611 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:12:49.611 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T06:12:49.612 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:12:49.613 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:12:49.613 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:12:49.614 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:12:49.615 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:12:49.615 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:12:49.616 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:12:49.616 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:12:49.617 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:12:49.617 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:12:49.618 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:12:49.618 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:12:49.619 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:12:49.619 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:12:49.620 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:12:49.620 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-10-11T06:12:49.620 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:12:49.802 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.101943.asok 2023-10-11T06:12:49.983 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.101943.asok 2023-10-11T06:12:49.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:49.984 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.101943.asok status 2023-10-11T06:12:50.225 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:12:50.225 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:12:50.225 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:12:50.225 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:12:50.226 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-10-11T06:12:50.226 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:12:50.226 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T06:12:50.226 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "101943", 2023-10-11T06:12:50.226 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:12:50.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:50.227 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:12:50.227 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:12:50.227 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5511, 2023-10-11T06:12:50.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:12:50.228 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:12:50.228 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:12:50.228 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 5511 2023-10-11T06:12:50.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:50.228 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:12:50.228 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:12:50.229 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:12:50.229 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2858111332 2023-10-11T06:12:50.229 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:12:50.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:50.229 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:12:50.230 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:12:50.230 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:12:50.230 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2858111332 2023-10-11T06:12:50.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:12:50.231 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.5511 192.168.0.1:0/2858111332", 2023-10-11T06:12:50.231 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/2858111332", 2023-10-11T06:12:50.231 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:12:50.231 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 15, 2023-10-11T06:12:50.231 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 38, 2023-10-11T06:12:50.232 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:12:50.232 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:12:50.232 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:12:50.232 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:12:50.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:50.241 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:12:50.241 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:50.296 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:12:50.296 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:50.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:50.297 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:50.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2023-10-11T06:12:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.597+0000 7f6d0ee42640 1 Processor -- start 2023-10-11T06:12:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.597+0000 7f6d0ee42640 1 -- start start 2023-10-11T06:12:50.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.599+0000 7f6d0ee42640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 0x7f6d080d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.599+0000 7f6d0ee42640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6d0806c4b0 0x7f6d0815e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.599+0000 7f6d0ee42640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0815e0d0 con 0x7f6d0806b990 2023-10-11T06:12:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.599+0000 7f6d0ee42640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080d1da0 con 0x7f6d0806c4b0 2023-10-11T06:12:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.599+0000 7f6d0ee42640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0806ca20 con 0x7f6d080d0390 2023-10-11T06:12:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.599+0000 7f6d0d63f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6d0806b990 0x7f6d0806bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46074/0 (socket says 172.21.15.84:46074) 2023-10-11T06:12:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.599+0000 7f6d0d63f640 1 -- 172.21.15.84:0/3681789395 learned_addr learned my addr 172.21.15.84:0/3681789395 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:50.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0de40640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 0x7f6d080d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0e641640 1 --2- 172.21.15.84:0/3681789395 >> v2:172.21.15.99:3300/0 conn(0x7f6d0806c4b0 0x7f6d0815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136785042 0 0) 0x7f6d0815e0d0 con 0x7f6d0806b990 2023-10-11T06:12:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf4003520 con 0x7f6d0806b990 2023-10-11T06:12:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4023611071 0 0) 0x7f6cf4003520 con 0x7f6d0806b990 2023-10-11T06:12:50.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 msgr2=0x7f6d080d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 0x7f6d080d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 >> v2:172.21.15.99:3300/0 conn(0x7f6d0806c4b0 msgr2=0x7f6d0815e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 --2- 172.21.15.84:0/3681789395 >> v2:172.21.15.99:3300/0 conn(0x7f6d0806c4b0 0x7f6d0815e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080d0ca0 con 0x7f6d0806b990 2023-10-11T06:12:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f6cf8001670 con 0x7f6d0806b990 2023-10-11T06:12:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 >> v1:172.21.15.84:6789/0 conn(0x7f6d0806b990 legacy=0x7f6d0806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 0x7f6cf4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf4003ef0 0x7f6cf40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cf4004d00 0x7f6cf4005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0806ca20 con 0x7f6cf4004d00 2023-10-11T06:12:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf400b980 con 0x7f6cf4003ef0 2023-10-11T06:12:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.600+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf400baf0 con 0x7f6d080d0390 2023-10-11T06:12:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0e641640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cf4004d00 0x7f6cf4005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0de40640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 0x7f6cf4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0d63f640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf4003ef0 0x7f6cf40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0e641640 1 -- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 msgr2=0x7f6cf4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:50.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0e641640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d080d0390 0x7f6cf4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0e641640 1 -- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf4003ef0 msgr2=0x7f6cf40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:50.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0e641640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf4003ef0 0x7f6cf40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.601+0000 7f6d0e641640 1 -- 172.21.15.84:0/3681789395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4003520 con 0x7f6cf4004d00 2023-10-11T06:12:50.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.602+0000 7f6d0e641640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cf4004d00 0x7f6cf4005170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009fc0 tx=0x7f6cf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879d86a64f7234a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:50.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0002cc40 con 0x7f6cf4004d00 2023-10-11T06:12:50.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0000f040 con 0x7f6cf4004d00 2023-10-11T06:12:50.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ce3e640 1 -- 172.21.15.84:0/3681789395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00034440 con 0x7f6cf4004d00 2023-10-11T06:12:50.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ee42640 1 -- 172.21.15.84:0/3681789395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cf4004d00 msgr2=0x7f6cf4005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:50.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ee42640 1 --2- 172.21.15.84:0/3681789395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6cf4004d00 0x7f6cf4005170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009fc0 tx=0x7f6cf40057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0e641640 1 -- 172.21.15.84:0/3681789395 reap_dead start 2023-10-11T06:12:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ee42640 1 -- 172.21.15.84:0/3681789395 shutdown_connections 2023-10-11T06:12:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ee42640 1 -- 172.21.15.84:0/3681789395 >> 172.21.15.84:0/3681789395 conn(0x7f6d0809bb20 msgr2=0x7f6d080d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ee42640 1 -- 172.21.15.84:0/3681789395 shutdown_connections 2023-10-11T06:12:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.603+0000 7f6d0ee42640 1 -- 172.21.15.84:0/3681789395 wait complete. 2023-10-11T06:12:50.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.605+0000 7f6d0ee42640 1 Processor -- start 2023-10-11T06:12:50.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.605+0000 7f6d0ee42640 1 -- start start 2023-10-11T06:12:50.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.606+0000 7f6d0ee42640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cf4004d00 0x7f6d080c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.606+0000 7f6d0ee42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d081566f0 0x7f6d08156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.606+0000 7f6d0ee42640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d081583d0 0x7f6d08158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.606+0000 7f6d0ee42640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b6440 con 0x7f6d081583d0 2023-10-11T06:12:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.606+0000 7f6d0ee42640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b3a80 con 0x7f6d081566f0 2023-10-11T06:12:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.606+0000 7f6d0ee42640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b3d60 con 0x7f6cf4004d00 2023-10-11T06:12:50.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d081583d0 0x7f6d08158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d081583d0 0x7f6d08158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39846/0 (socket says 172.21.15.84:39846) 2023-10-11T06:12:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 -- 172.21.15.84:0/1819202705 learned_addr learned my addr 172.21.15.84:0/1819202705 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0d63f640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d081566f0 0x7f6d08156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0de40640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cf4004d00 0x7f6d080c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 -- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cf4004d00 msgr2=0x7f6d080c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:50.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cf4004d00 0x7f6d080c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 -- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d081566f0 msgr2=0x7f6d08156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:50.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d081566f0 0x7f6d08156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 -- 172.21.15.84:0/1819202705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080b4040 con 0x7f6d081583d0 2023-10-11T06:12:50.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.607+0000 7f6d0e641640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d081583d0 0x7f6d08158840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6d00037040 tx=0x7f6d00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.608+0000 7f6cfe7fc640 1 -- 172.21.15.84:0/1819202705 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00034ad0 con 0x7f6d081583d0 2023-10-11T06:12:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.608+0000 7f6cfe7fc640 1 -- 172.21.15.84:0/1819202705 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d00034c30 con 0x7f6d081583d0 2023-10-11T06:12:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.608+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d080b4320 con 0x7f6d081583d0 2023-10-11T06:12:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.608+0000 7f6cfe7fc640 1 -- 172.21.15.84:0/1819202705 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00003990 con 0x7f6d081583d0 2023-10-11T06:12:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.608+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d080c9240 con 0x7f6d081583d0 2023-10-11T06:12:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.609+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d080b6580 con 0x7f6d081583d0 2023-10-11T06:12:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.610+0000 7f6cfe7fc640 1 -- 172.21.15.84:0/1819202705 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6d00047460 con 0x7f6d081583d0 2023-10-11T06:12:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.611+0000 7f6cfe7fc640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6cec06a860 0x7f6cec06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.611+0000 7f6cfe7fc640 1 -- 172.21.15.84:0/1819202705 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d000aaed0 con 0x7f6d081583d0 2023-10-11T06:12:50.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.611+0000 7f6d0de40640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6cec06a860 0x7f6cec06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:50.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.612+0000 7f6d0de40640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6cec06a860 0x7f6cec06cd10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009a10 tx=0x7f6d04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:50.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.615+0000 7f6cfe7fc640 1 -- 172.21.15.84:0/1819202705 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f6d080b6580 con 0x7f6d081583d0 2023-10-11T06:12:50.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.804+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f6d080b6580 con 0x7f6d081583d0 2023-10-11T06:12:50.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.912+0000 7f6cfe7fc640 1 -- 172.21.15.84:0/1819202705 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v39) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6d000390a0 con 0x7f6d081583d0 2023-10-11T06:12:50.915 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'data_new' created 2023-10-11T06:12:50.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.918+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6cec06a860 msgr2=0x7f6cec06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:50.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.918+0000 7f6d0ee42640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6cec06a860 0x7f6cec06cd10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009a10 tx=0x7f6d04008040 comp rx=0 tx=0).stop 2023-10-11T06:12:50.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.918+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d081583d0 msgr2=0x7f6d08158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:50.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.918+0000 7f6d0ee42640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d081583d0 0x7f6d08158840 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6d00037040 tx=0x7f6d00009750 comp rx=0 tx=0).stop 2023-10-11T06:12:50.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 shutdown_connections 2023-10-11T06:12:50.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6cec06a860 0x7f6cec06cd10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d081583d0 0x7f6d08158840 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d081566f0 0x7f6d08156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 --2- 172.21.15.84:0/1819202705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6cf4004d00 0x7f6d080c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:50.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 >> 172.21.15.84:0/1819202705 conn(0x7f6d0809bb20 msgr2=0x7f6d0815ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:50.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 shutdown_connections 2023-10-11T06:12:50.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:50.919+0000 7f6d0ee42640 1 -- 172.21.15.84:0/1819202705 wait complete. 2023-10-11T06:12:50.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2023-10-11T06:12:51.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.140+0000 7f24873bb640 1 Processor -- start 2023-10-11T06:12:51.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.140+0000 7f24873bb640 1 -- start start 2023-10-11T06:12:51.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.141+0000 7f24873bb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.141+0000 7f24873bb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f24800ced30 0x7f248006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.141+0000 7f24873bb640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2480071380 con 0x7f24800c8d00 2023-10-11T06:12:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.141+0000 7f24873bb640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248006e0f0 con 0x7f24800ced30 2023-10-11T06:12:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.141+0000 7f24873bb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24800cbae0 con 0x7f248015a0b0 2023-10-11T06:12:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.142+0000 7f2485157640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.142+0000 7f2485157640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:49630/0 (socket says 172.21.15.84:49630) 2023-10-11T06:12:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.142+0000 7f2485157640 1 -- 172.21.15.84:0/902190760 learned_addr learned my addr 172.21.15.84:0/902190760 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.142+0000 7f2485958640 1 --2- 172.21.15.84:0/902190760 >> v2:172.21.15.99:3300/0 conn(0x7f24800ced30 0x7f248006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.143+0000 7f2485157640 1 -- 172.21.15.84:0/902190760 >> v2:172.21.15.99:3300/0 conn(0x7f24800ced30 msgr2=0x7f248006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.143+0000 7f2485157640 1 --2- 172.21.15.84:0/902190760 >> v2:172.21.15.99:3300/0 conn(0x7f24800ced30 0x7f248006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.143+0000 7f2485157640 1 -- 172.21.15.84:0/902190760 >> v1:172.21.15.99:6789/0 conn(0x7f24800c8d00 legacy=0x7f24800ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.143+0000 7f2485157640 1 -- 172.21.15.84:0/902190760 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248006de10 con 0x7f248015a0b0 2023-10-11T06:12:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.144+0000 7f2485157640 1 --2- 172.21.15.84:0/902190760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cb5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f247c009870 tx=0x7f247c02b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e60eb3c8064a7dc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.145+0000 7f2477fff640 1 -- 172.21.15.84:0/902190760 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c031740 con 0x7f248015a0b0 2023-10-11T06:12:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.145+0000 7f2477fff640 1 -- 172.21.15.84:0/902190760 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c0318a0 con 0x7f248015a0b0 2023-10-11T06:12:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.145+0000 7f2477fff640 1 -- 172.21.15.84:0/902190760 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c031c30 con 0x7f248015a0b0 2023-10-11T06:12:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.145+0000 7f24873bb640 1 -- 172.21.15.84:0/902190760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 msgr2=0x7f24800cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.146+0000 7f24873bb640 1 --2- 172.21.15.84:0/902190760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cb5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f247c009870 tx=0x7f247c02b7e0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.146+0000 7f24873bb640 1 -- 172.21.15.84:0/902190760 shutdown_connections 2023-10-11T06:12:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.146+0000 7f24873bb640 1 --2- 172.21.15.84:0/902190760 >> v2:172.21.15.99:3300/0 conn(0x7f24800ced30 0x7f248006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.146+0000 7f24873bb640 1 --2- 172.21.15.84:0/902190760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cb5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.146+0000 7f24873bb640 1 -- 172.21.15.84:0/902190760 >> 172.21.15.84:0/902190760 conn(0x7f2480069c70 msgr2=0x7f248006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:51.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.146+0000 7f24873bb640 1 -- 172.21.15.84:0/902190760 shutdown_connections 2023-10-11T06:12:51.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.146+0000 7f24873bb640 1 -- 172.21.15.84:0/902190760 wait complete. 2023-10-11T06:12:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.148+0000 7f24873bb640 1 Processor -- start 2023-10-11T06:12:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.148+0000 7f24873bb640 1 -- start start 2023-10-11T06:12:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.149+0000 7f24873bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24800c8d00 0x7f248014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.149+0000 7f24873bb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24800ced30 0x7f24801591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.149+0000 7f24873bb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.149+0000 7f24873bb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24801f5370 con 0x7f24800ced30 2023-10-11T06:12:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.149+0000 7f24873bb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24801f2330 con 0x7f24800c8d00 2023-10-11T06:12:51.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.149+0000 7f24873bb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24801f2610 con 0x7f248015a0b0 2023-10-11T06:12:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24800ced30 0x7f24801591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24800ced30 0x7f24801591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39860/0 (socket says 172.21.15.84:39860) 2023-10-11T06:12:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 -- 172.21.15.84:0/784355658 learned_addr learned my addr 172.21.15.84:0/784355658 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2485958640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2485157640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24800c8d00 0x7f248014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 -- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 msgr2=0x7f24800cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 -- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24800c8d00 msgr2=0x7f248014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24800c8d00 0x7f248014ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 -- 172.21.15.84:0/784355658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c0094d0 con 0x7f24800ced30 2023-10-11T06:12:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.150+0000 7f2484956640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24800ced30 0x7f24801591d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f247000b3f0 tx=0x7f247000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.151+0000 7f2475ffb640 1 -- 172.21.15.84:0/784355658 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470011830 con 0x7f24800ced30 2023-10-11T06:12:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.151+0000 7f2475ffb640 1 -- 172.21.15.84:0/784355658 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470011990 con 0x7f24800ced30 2023-10-11T06:12:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.151+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24801f28f0 con 0x7f24800ced30 2023-10-11T06:12:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.151+0000 7f2475ffb640 1 -- 172.21.15.84:0/784355658 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470011d40 con 0x7f24800ced30 2023-10-11T06:12:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.151+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248014bf50 con 0x7f24800ced30 2023-10-11T06:12:51.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.152+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2480073ed0 con 0x7f24800ced30 2023-10-11T06:12:51.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.153+0000 7f2475ffb640 1 -- 172.21.15.84:0/784355658 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f247001a400 con 0x7f24800ced30 2023-10-11T06:12:51.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.155+0000 7f2475ffb640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f246406a860 0x7f246406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.155+0000 7f2475ffb640 1 -- 172.21.15.84:0/784355658 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6454+0+0 (secure 0 0 0) 0x7f2470088fe0 con 0x7f24800ced30 2023-10-11T06:12:51.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.155+0000 7f2485157640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f246406a860 0x7f246406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.156+0000 7f2485157640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f246406a860 0x7f246406cd10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f247c009870 tx=0x7f247c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:51.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.160+0000 7f2475ffb640 1 -- 172.21.15.84:0/784355658 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2470054070 con 0x7f24800ced30 2023-10-11T06:12:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.392+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7f248014c840 con 0x7f24800ced30 2023-10-11T06:12:51.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.402+0000 7f2475ffb640 1 -- 172.21.15.84:0/784355658 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 7 to fsmap v16) v1 ==== 131+0+0 (secure 0 0 0) 0x7f24700150b0 con 0x7f24800ced30 2023-10-11T06:12:51.403 INFO:teuthology.orchestra.run.smithi084.stderr:added data pool 7 to fsmap 2023-10-11T06:12:51.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.407+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f246406a860 msgr2=0x7f246406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.407+0000 7f24873bb640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f246406a860 0x7f246406cd10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f247c009870 tx=0x7f247c008030 comp rx=0 tx=0).stop 2023-10-11T06:12:51.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.407+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24800ced30 msgr2=0x7f24801591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.407+0000 7f24873bb640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24800ced30 0x7f24801591d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f247000b3f0 tx=0x7f247000b8b0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.409+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 shutdown_connections 2023-10-11T06:12:51.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.409+0000 7f24873bb640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f246406a860 0x7f246406cd10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.410+0000 7f24873bb640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f248015a0b0 0x7f24800cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.410+0000 7f24873bb640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24800ced30 0x7f24801591d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.410+0000 7f24873bb640 1 --2- 172.21.15.84:0/784355658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24800c8d00 0x7f248014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.410+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 >> 172.21.15.84:0/784355658 conn(0x7f2480069c70 msgr2=0x7f24800c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:51.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.410+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 shutdown_connections 2023-10-11T06:12:51.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.411+0000 7f24873bb640 1 -- 172.21.15.84:0/784355658 wait complete. 2023-10-11T06:12:51.439 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:12:51.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.624+0000 7f5b68a2d640 1 Processor -- start 2023-10-11T06:12:51.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.624+0000 7f5b68a2d640 1 -- start start 2023-10-11T06:12:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.625+0000 7f5b68a2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b640ceae0 0x7f5b6415a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.625+0000 7f5b68a2d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5b6406fd80 0x7f5b6415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.625+0000 7f5b68a2d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6415da50 con 0x7f5b640b3e70 2023-10-11T06:12:51.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.625+0000 7f5b68a2d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b640cad10 con 0x7f5b640ceae0 2023-10-11T06:12:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.625+0000 7f5b68a2d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6415dca0 con 0x7f5b6406fd80 2023-10-11T06:12:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b62ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b640ceae0 0x7f5b6415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b62ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b640ceae0 0x7f5b6415a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39864/0 (socket says 172.21.15.84:39864) 2023-10-11T06:12:51.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b62ffd640 1 -- 172.21.15.84:0/2763107010 learned_addr learned my addr 172.21.15.84:0/2763107010 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:51.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b63fff640 1 --2- 172.21.15.84:0/2763107010 >> v2:172.21.15.99:3300/0 conn(0x7f5b6406fd80 0x7f5b6415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b62ffd640 1 -- 172.21.15.84:0/2763107010 >> v2:172.21.15.99:3300/0 conn(0x7f5b6406fd80 msgr2=0x7f5b6415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b62ffd640 1 --2- 172.21.15.84:0/2763107010 >> v2:172.21.15.99:3300/0 conn(0x7f5b6406fd80 0x7f5b6415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b62ffd640 1 -- 172.21.15.84:0/2763107010 >> v1:172.21.15.99:6789/0 conn(0x7f5b640b3e70 legacy=0x7f5b640ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b62ffd640 1 -- 172.21.15.84:0/2763107010 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6415df80 con 0x7f5b640ceae0 2023-10-11T06:12:51.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.626+0000 7f5b63fff640 1 --2- 172.21.15.84:0/2763107010 >> v2:172.21.15.99:3300/0 conn(0x7f5b6406fd80 0x7f5b6415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:51.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.628+0000 7f5b62ffd640 1 --2- 172.21.15.84:0/2763107010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640ceae0 0x7f5b6415a470 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5b64072210 tx=0x7f5b5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59045231715078b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:51.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.628+0000 7f5b627fc640 1 -- 172.21.15.84:0/2763107010 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58031870 con 0x7f5b640ceae0 2023-10-11T06:12:51.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.628+0000 7f5b627fc640 1 -- 172.21.15.84:0/2763107010 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b580319d0 con 0x7f5b640ceae0 2023-10-11T06:12:51.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.628+0000 7f5b627fc640 1 -- 172.21.15.84:0/2763107010 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58031d60 con 0x7f5b640ceae0 2023-10-11T06:12:51.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.628+0000 7f5b68a2d640 1 -- 172.21.15.84:0/2763107010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640ceae0 msgr2=0x7f5b6415a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.628+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/2763107010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640ceae0 0x7f5b6415a470 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5b64072210 tx=0x7f5b5802b910 comp rx=0 tx=0).stop 2023-10-11T06:12:51.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.629+0000 7f5b68a2d640 1 -- 172.21.15.84:0/2763107010 shutdown_connections 2023-10-11T06:12:51.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.629+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/2763107010 >> v2:172.21.15.99:3300/0 conn(0x7f5b6406fd80 0x7f5b6415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.629+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/2763107010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640ceae0 0x7f5b6415a470 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.629+0000 7f5b68a2d640 1 -- 172.21.15.84:0/2763107010 >> 172.21.15.84:0/2763107010 conn(0x7f5b64069c70 msgr2=0x7f5b6406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:51.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.629+0000 7f5b68a2d640 1 -- 172.21.15.84:0/2763107010 shutdown_connections 2023-10-11T06:12:51.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.629+0000 7f5b68a2d640 1 -- 172.21.15.84:0/2763107010 wait complete. 2023-10-11T06:12:51.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.630+0000 7f5b68a2d640 1 Processor -- start 2023-10-11T06:12:51.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.631+0000 7f5b68a2d640 1 -- start start 2023-10-11T06:12:51.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b68a2d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b6406fd80 0x7f5b6415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b68a2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640b3e70 0x7f5b640c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b68a2d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b640ceae0 0x7f5b641f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b68a2d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640c6590 con 0x7f5b640b3e70 2023-10-11T06:12:51.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b68a2d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640c27f0 con 0x7f5b640ceae0 2023-10-11T06:12:51.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b68a2d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640c5a50 con 0x7f5b6406fd80 2023-10-11T06:12:51.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640b3e70 0x7f5b640c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640b3e70 0x7f5b640c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39880/0 (socket says 172.21.15.84:39880) 2023-10-11T06:12:51.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 -- 172.21.15.84:0/3570252505 learned_addr learned my addr 172.21.15.84:0/3570252505 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:51.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b637fe640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b6406fd80 0x7f5b6415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b63fff640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b640ceae0 0x7f5b641f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 -- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b6406fd80 msgr2=0x7f5b6415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b6406fd80 0x7f5b6415cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 -- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b640ceae0 msgr2=0x7f5b641f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b640ceae0 0x7f5b641f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b62ffd640 1 -- 172.21.15.84:0/3570252505 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580094d0 con 0x7f5b640b3e70 2023-10-11T06:12:51.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b637fe640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b6406fd80 0x7f5b6415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:51.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.632+0000 7f5b63fff640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b640ceae0 0x7f5b641f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:51.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.633+0000 7f5b62ffd640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640b3e70 0x7f5b640c5510 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5b58000c00 tx=0x7f5b580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:51.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.633+0000 7f5b4bfff640 1 -- 172.21.15.84:0/3570252505 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58031840 con 0x7f5b640b3e70 2023-10-11T06:12:51.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.633+0000 7f5b4bfff640 1 -- 172.21.15.84:0/3570252505 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b580443f0 con 0x7f5b640b3e70 2023-10-11T06:12:51.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.633+0000 7f5b4bfff640 1 -- 172.21.15.84:0/3570252505 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580447a0 con 0x7f5b640b3e70 2023-10-11T06:12:51.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.633+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b640c5cf0 con 0x7f5b640b3e70 2023-10-11T06:12:51.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.633+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b640c62b0 con 0x7f5b640b3e70 2023-10-11T06:12:51.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.634+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6404e890 con 0x7f5b640b3e70 2023-10-11T06:12:51.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.636+0000 7f5b4bfff640 1 -- 172.21.15.84:0/3570252505 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5b5804d450 con 0x7f5b640b3e70 2023-10-11T06:12:51.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.637+0000 7f5b4bfff640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b5006a830 0x7f5b5006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:51.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.637+0000 7f5b4bfff640 1 -- 172.21.15.84:0/3570252505 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f5b5802d040 con 0x7f5b640b3e70 2023-10-11T06:12:51.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.637+0000 7f5b637fe640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b5006a830 0x7f5b5006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:51.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.638+0000 7f5b637fe640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b5006a830 0x7f5b5006cce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b54003eb0 tx=0x7f5b54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:51.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.642+0000 7f5b4bfff640 1 -- 172.21.15.84:0/3570252505 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5b58076a70 con 0x7f5b640b3e70 2023-10-11T06:12:51.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.866+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b6415e230 con 0x7f5b640b3e70 2023-10-11T06:12:51.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.867+0000 7f5b4bfff640 1 -- 172.21.15.84:0/3570252505 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4644 (secure 0 0 0) 0x7f5b5807a090 con 0x7f5b640b3e70 2023-10-11T06:12:51.868 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:51.868 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:12:40.002715+0000","modified":"2023-10-11T06:12:51.393935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5334":{"gid":5334,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/2172719017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2172719017},{"type":"v1","addr":"172.21.15.99:6835","nonce":2172719017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T06:12:51.869 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-10-11T06:12:51.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b5006a830 msgr2=0x7f5b5006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b5006a830 0x7f5b5006cce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b54003eb0 tx=0x7f5b54009040 comp rx=0 tx=0).stop 2023-10-11T06:12:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640b3e70 msgr2=0x7f5b640c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640b3e70 0x7f5b640c5510 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5b58000c00 tx=0x7f5b580046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 shutdown_connections 2023-10-11T06:12:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5b5006a830 0x7f5b5006cce0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b640ceae0 0x7f5b641f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b640b3e70 0x7f5b640c5510 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 --2- 172.21.15.84:0/3570252505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b6406fd80 0x7f5b6415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 >> 172.21.15.84:0/3570252505 conn(0x7f5b64069c70 msgr2=0x7f5b640ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 shutdown_connections 2023-10-11T06:12:51.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:51.869+0000 7f5b68a2d640 1 -- 172.21.15.84:0/3570252505 wait complete. 2023-10-11T06:12:51.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:52.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.087+0000 7f92c9bd5640 1 Processor -- start 2023-10-11T06:12:52.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.087+0000 7f92c9bd5640 1 -- start start 2023-10-11T06:12:52.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.088+0000 7f92c9bd5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92c40afe10 0x7f92c40b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.088+0000 7f92c9bd5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c415c700 con 0x7f92c40d0b00 2023-10-11T06:12:52.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.088+0000 7f92c9bd5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c40b02a0 con 0x7f92c415c330 2023-10-11T06:12:52.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.088+0000 7f92c9bd5640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c40b0f20 con 0x7f92c40afe10 2023-10-11T06:12:52.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.089+0000 7f92c93d4640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f92c40d0b00 0x7f92c40cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46090/0 (socket says 172.21.15.84:46090) 2023-10-11T06:12:52.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.089+0000 7f92c93d4640 1 -- 172.21.15.84:0/2072669811 learned_addr learned my addr 172.21.15.84:0/2072669811 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:52.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.089+0000 7f92c3fff640 1 --2- 172.21.15.84:0/2072669811 >> v2:172.21.15.99:3300/0 conn(0x7f92c40afe10 0x7f92c40b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.089+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386837075 0 0) 0x7f92bc001490 con 0x7f92c40d0b00 2023-10-11T06:12:52.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.089+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac003520 con 0x7f92c40d0b00 2023-10-11T06:12:52.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3861500020 0 0) 0x7f92ac003520 con 0x7f92c40d0b00 2023-10-11T06:12:52.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 >> v2:172.21.15.99:3300/0 conn(0x7f92c40afe10 msgr2=0x7f92c40b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:52.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 --2- 172.21.15.84:0/2072669811 >> v2:172.21.15.99:3300/0 conn(0x7f92c40afe10 0x7f92c40b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 >> v1:172.21.15.99:6789/0 conn(0x7f92c415c330 legacy=0x7f92c40d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40b0580 con 0x7f92c40d0b00 2023-10-11T06:12:52.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f92bc001b80 con 0x7f92c40d0b00 2023-10-11T06:12:52.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 >> v1:172.21.15.84:6789/0 conn(0x7f92c40d0b00 legacy=0x7f92c40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ac003970 0x7f92ac003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92ac0047c0 0x7f92ac004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac005290 0x7f92ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40b0f20 con 0x7f92ac003970 2023-10-11T06:12:52.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac00c1c0 con 0x7f92ac005290 2023-10-11T06:12:52.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac00c380 con 0x7f92ac0047c0 2023-10-11T06:12:52.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c8bd3640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ac003970 0x7f92ac003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c93d4640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac005290 0x7f92ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c3fff640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92ac0047c0 0x7f92ac004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.090+0000 7f92c8bd3640 1 -- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92ac0047c0 msgr2=0x7f92ac004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:52.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.091+0000 7f92c8bd3640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92ac0047c0 0x7f92ac004c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.091+0000 7f92c8bd3640 1 -- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac005290 msgr2=0x7f92ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:52.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.091+0000 7f92c8bd3640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac005290 0x7f92ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.091+0000 7f92c8bd3640 1 -- 172.21.15.84:0/2072669811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac003520 con 0x7f92ac003970 2023-10-11T06:12:52.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.091+0000 7f92c93d4640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac005290 0x7f92ac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:52.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.091+0000 7f92c93d4640 1 -- 172.21.15.84:0/2072669811 reap_dead start 2023-10-11T06:12:52.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.092+0000 7f92c8bd3640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ac003970 0x7f92ac003d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f92b4001770 tx=0x7f92ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e63463035b8bb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:52.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.092+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40052d0 con 0x7f92ac003970 2023-10-11T06:12:52.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.092+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b402e050 con 0x7f92ac003970 2023-10-11T06:12:52.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.092+0000 7f92c37fe640 1 -- 172.21.15.84:0/2072669811 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4003640 con 0x7f92ac003970 2023-10-11T06:12:52.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.092+0000 7f92c9bd5640 1 -- 172.21.15.84:0/2072669811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ac003970 msgr2=0x7f92ac003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.092+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ac003970 0x7f92ac003d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f92b4001770 tx=0x7f92ac0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.093+0000 7f92c9bd5640 1 -- 172.21.15.84:0/2072669811 shutdown_connections 2023-10-11T06:12:52.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.093+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/2072669811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ac003970 0x7f92ac003d80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.093+0000 7f92c9bd5640 1 -- 172.21.15.84:0/2072669811 >> 172.21.15.84:0/2072669811 conn(0x7f92c409bb10 msgr2=0x7f92c40d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:52.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.093+0000 7f92c9bd5640 1 -- 172.21.15.84:0/2072669811 shutdown_connections 2023-10-11T06:12:52.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.093+0000 7f92c9bd5640 1 -- 172.21.15.84:0/2072669811 wait complete. 2023-10-11T06:12:52.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.094+0000 7f92c9bd5640 1 Processor -- start 2023-10-11T06:12:52.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.095+0000 7f92c9bd5640 1 -- start start 2023-10-11T06:12:52.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.095+0000 7f92c9bd5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003970 0x7f92c40c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.095+0000 7f92c9bd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c40cbfe0 0x7f92c41f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.095+0000 7f92c9bd5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92c40cc550 0x7f92c41f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.095+0000 7f92c9bd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41f4f70 con 0x7f92c40cbfe0 2023-10-11T06:12:52.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.095+0000 7f92c9bd5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415ef40 con 0x7f92ac003970 2023-10-11T06:12:52.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.095+0000 7f92c9bd5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41f1820 con 0x7f92c40cc550 2023-10-11T06:12:52.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c40cbfe0 0x7f92c41f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c40cbfe0 0x7f92c41f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39886/0 (socket says 172.21.15.84:39886) 2023-10-11T06:12:52.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 -- 172.21.15.84:0/166983304 learned_addr learned my addr 172.21.15.84:0/166983304 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:52.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c93d4640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92c40cc550 0x7f92c41f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 -- 172.21.15.84:0/166983304 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92c40cc550 msgr2=0x7f92c41f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92c40cc550 0x7f92c41f12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 -- 172.21.15.84:0/166983304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003970 msgr2=0x7f92c40c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003970 0x7f92c40c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 -- 172.21.15.84:0/166983304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c41f51c0 con 0x7f92c40cbfe0 2023-10-11T06:12:52.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.096+0000 7f92c3fff640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c40cbfe0 0x7f92c41f0da0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f92b8010790 tx=0x7f92b8010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:52.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.097+0000 7f92c17fa640 1 -- 172.21.15.84:0/166983304 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8014800 con 0x7f92c40cbfe0 2023-10-11T06:12:52.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.097+0000 7f92c17fa640 1 -- 172.21.15.84:0/166983304 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8014960 con 0x7f92c40cbfe0 2023-10-11T06:12:52.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.097+0000 7f92c17fa640 1 -- 172.21.15.84:0/166983304 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8014cf0 con 0x7f92c40cbfe0 2023-10-11T06:12:52.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.097+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c41f54a0 con 0x7f92c40cbfe0 2023-10-11T06:12:52.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.097+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c41f5a60 con 0x7f92c40cbfe0 2023-10-11T06:12:52.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.098+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c404e890 con 0x7f92c40cbfe0 2023-10-11T06:12:52.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.099+0000 7f92c17fa640 1 -- 172.21.15.84:0/166983304 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f92b8012390 con 0x7f92c40cbfe0 2023-10-11T06:12:52.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.100+0000 7f92c17fa640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f92b006a830 0x7f92b006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.100+0000 7f92c17fa640 1 -- 172.21.15.84:0/166983304 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f92b8088f50 con 0x7f92c40cbfe0 2023-10-11T06:12:52.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.101+0000 7f92c8bd3640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f92b006a830 0x7f92b006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.101+0000 7f92c8bd3640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f92b006a830 0x7f92b006cce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f92b4033000 tx=0x7f92b402f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:52.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.105+0000 7f92c17fa640 1 -- 172.21.15.84:0/166983304 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f92b8053f80 con 0x7f92c40cbfe0 2023-10-11T06:12:52.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.297+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92c40ca470 con 0x7f92c40cbfe0 2023-10-11T06:12:52.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.300+0000 7f92c17fa640 1 -- 172.21.15.84:0/166983304 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+19676 (secure 0 0 0) 0x7f92b8022090 con 0x7f92c40cbfe0 2023-10-11T06:12:52.302 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:52.302 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:51.393589+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:52.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f92b006a830 msgr2=0x7f92b006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f92b006a830 0x7f92b006cce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f92b4033000 tx=0x7f92b402f000 comp rx=0 tx=0).stop 2023-10-11T06:12:52.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c40cbfe0 msgr2=0x7f92c41f0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c40cbfe0 0x7f92c41f0da0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f92b8010790 tx=0x7f92b8010880 comp rx=0 tx=0).stop 2023-10-11T06:12:52.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 shutdown_connections 2023-10-11T06:12:52.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f92b006a830 0x7f92b006cce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92c40cc550 0x7f92c41f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92c40cbfe0 0x7f92c41f0da0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 --2- 172.21.15.84:0/166983304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ac003970 0x7f92c40c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 >> 172.21.15.84:0/166983304 conn(0x7f92c409bb10 msgr2=0x7f92c40b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:52.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 shutdown_connections 2023-10-11T06:12:52.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.303+0000 7f92c9bd5640 1 -- 172.21.15.84:0/166983304 wait complete. 2023-10-11T06:12:52.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:52.323 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2023-10-11T06:12:52.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:52.392 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2023-10-11T06:12:52.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-10-11T06:12:52.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.676+0000 7fe2b1194640 1 Processor -- start 2023-10-11T06:12:52.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.677+0000 7fe2b1194640 1 -- start start 2023-10-11T06:12:52.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.678+0000 7fe2b1194640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2ac159e60 0x7fe2ac0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.678+0000 7fe2b1194640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac0af980 0x7fe2ac15e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.678+0000 7fe2b1194640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac0d14c0 con 0x7fe2ac0cf860 2023-10-11T06:12:52.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.678+0000 7fe2b1194640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac0cf6f0 con 0x7fe2ac159e60 2023-10-11T06:12:52.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.678+0000 7fe2b1194640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac15db70 con 0x7fe2ac0af980 2023-10-11T06:12:52.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2abfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2ac159e60 0x7fe2ac0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2b0993640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe2ac0cf860 0x7fe2ac0cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46104/0 (socket says 172.21.15.84:46104) 2023-10-11T06:12:52.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2b0993640 1 -- 172.21.15.84:0/1244661852 learned_addr learned my addr 172.21.15.84:0/1244661852 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:52.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2ab7fe640 1 --2- 172.21.15.84:0/1244661852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac0af980 0x7fe2ac15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2abfff640 1 -- 172.21.15.84:0/1244661852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac0af980 msgr2=0x7fe2ac15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:52.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2abfff640 1 --2- 172.21.15.84:0/1244661852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac0af980 0x7fe2ac15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2abfff640 1 -- 172.21.15.84:0/1244661852 >> v1:172.21.15.84:6789/0 conn(0x7fe2ac0cf860 legacy=0x7fe2ac0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.679+0000 7fe2abfff640 1 -- 172.21.15.84:0/1244661852 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0d11e0 con 0x7fe2ac159e60 2023-10-11T06:12:52.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.680+0000 7fe2abfff640 1 --2- 172.21.15.84:0/1244661852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac159e60 0x7fe2ac0cacf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009d90 tx=0x7fe29c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdbd6bb8c6e62ed server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:52.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2aaffd640 1 -- 172.21.15.84:0/1244661852 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c02cbd0 con 0x7fe2ac159e60 2023-10-11T06:12:52.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2aaffd640 1 -- 172.21.15.84:0/1244661852 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c02cd30 con 0x7fe2ac159e60 2023-10-11T06:12:52.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2aaffd640 1 -- 172.21.15.84:0/1244661852 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c02cea0 con 0x7fe2ac159e60 2023-10-11T06:12:52.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2b1194640 1 -- 172.21.15.84:0/1244661852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac159e60 msgr2=0x7fe2ac0cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2b1194640 1 --2- 172.21.15.84:0/1244661852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac159e60 0x7fe2ac0cacf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009d90 tx=0x7fe29c02be20 comp rx=0 tx=0).stop 2023-10-11T06:12:52.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2b1194640 1 -- 172.21.15.84:0/1244661852 shutdown_connections 2023-10-11T06:12:52.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2b1194640 1 --2- 172.21.15.84:0/1244661852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac0af980 0x7fe2ac15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2b1194640 1 --2- 172.21.15.84:0/1244661852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac159e60 0x7fe2ac0cacf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.681+0000 7fe2b1194640 1 -- 172.21.15.84:0/1244661852 >> 172.21.15.84:0/1244661852 conn(0x7fe2ac069c70 msgr2=0x7fe2ac06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:52.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.682+0000 7fe2b1194640 1 -- 172.21.15.84:0/1244661852 shutdown_connections 2023-10-11T06:12:52.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.682+0000 7fe2b1194640 1 -- 172.21.15.84:0/1244661852 wait complete. 2023-10-11T06:12:52.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.684+0000 7fe2b1194640 1 Processor -- start 2023-10-11T06:12:52.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.684+0000 7fe2b1194640 1 -- start start 2023-10-11T06:12:52.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2b1194640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac0af980 0x7fe2ac0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2b1194640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0cf860 0x7fe2ac15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2b1194640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac159e60 0x7fe2ac0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2b1194640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0c64d0 con 0x7fe2ac0af980 2023-10-11T06:12:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2b1194640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0c5600 con 0x7fe2ac0cf860 2023-10-11T06:12:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2b1194640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0c5850 con 0x7fe2ac159e60 2023-10-11T06:12:52.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac0af980 0x7fe2ac0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac0af980 0x7fe2ac0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39902/0 (socket says 172.21.15.84:39902) 2023-10-11T06:12:52.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.685+0000 7fe2abfff640 1 -- 172.21.15.84:0/3087030931 learned_addr learned my addr 172.21.15.84:0/3087030931 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:52.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2abfff640 1 -- 172.21.15.84:0/3087030931 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac159e60 msgr2=0x7fe2ac0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2abfff640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac159e60 0x7fe2ac0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2abfff640 1 -- 172.21.15.84:0/3087030931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0cf860 msgr2=0x7fe2ac15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2abfff640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0cf860 0x7fe2ac15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2abfff640 1 -- 172.21.15.84:0/3087030931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009a70 con 0x7fe2ac0af980 2023-10-11T06:12:52.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2abfff640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac0af980 0x7fe2ac0b34d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe29c037040 tx=0x7fe29c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:52.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2a8ff9640 1 -- 172.21.15.84:0/3087030931 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c02cea0 con 0x7fe2ac0af980 2023-10-11T06:12:52.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2a8ff9640 1 -- 172.21.15.84:0/3087030931 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c040650 con 0x7fe2ac0af980 2023-10-11T06:12:52.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2a8ff9640 1 -- 172.21.15.84:0/3087030931 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c040a00 con 0x7fe2ac0af980 2023-10-11T06:12:52.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.686+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac0c5b30 con 0x7fe2ac0af980 2023-10-11T06:12:52.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.687+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac0c60f0 con 0x7fe2ac0af980 2023-10-11T06:12:52.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.688+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac0c6610 con 0x7fe2ac0af980 2023-10-11T06:12:52.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.689+0000 7fe2a8ff9640 1 -- 172.21.15.84:0/3087030931 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe29c040b60 con 0x7fe2ac0af980 2023-10-11T06:12:52.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.690+0000 7fe2a8ff9640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe28c06a830 0x7fe28c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:52.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.690+0000 7fe2a8ff9640 1 -- 172.21.15.84:0/3087030931 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fe29c0abe70 con 0x7fe2ac0af980 2023-10-11T06:12:52.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.690+0000 7fe2ab7fe640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe28c06a830 0x7fe28c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:52.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.691+0000 7fe2ab7fe640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe28c06a830 0x7fe28c06cce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0b3a10 tx=0x7fe294008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:52.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.695+0000 7fe2a8ff9640 1 -- 172.21.15.84:0/3087030931 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe2ac0c6610 con 0x7fe2ac0af980 2023-10-11T06:12:52.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.959+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fe2ac0c6610 con 0x7fe2ac0af980 2023-10-11T06:12:52.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.965+0000 7fe2a8ff9640 1 -- 172.21.15.84:0/3087030931 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v10) v1 ==== 210+0+0 (secure 0 0 0) 0x7fe2ac0c6610 con 0x7fe2ac0af980 2023-10-11T06:12:52.967 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:12:52.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.967+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe28c06a830 msgr2=0x7fe28c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.967+0000 7fe2b1194640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe28c06a830 0x7fe28c06cce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0b3a10 tx=0x7fe294008040 comp rx=0 tx=0).stop 2023-10-11T06:12:52.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac0af980 msgr2=0x7fe2ac0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:52.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac0af980 0x7fe2ac0b34d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe29c037040 tx=0x7fe29c004a40 comp rx=0 tx=0).stop 2023-10-11T06:12:52.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 shutdown_connections 2023-10-11T06:12:52.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe28c06a830 0x7fe28c06cce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2ac159e60 0x7fe2ac0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2ac0cf860 0x7fe2ac15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 --2- 172.21.15.84:0/3087030931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2ac0af980 0x7fe2ac0b34d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:52.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 >> 172.21.15.84:0/3087030931 conn(0x7fe2ac069c70 msgr2=0x7fe2ac0c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:52.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 shutdown_connections 2023-10-11T06:12:52.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:52.968+0000 7fe2b1194640 1 -- 172.21.15.84:0/3087030931 wait complete. 2023-10-11T06:12:52.988 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:12:52.988 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:53.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:53.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.301+0000 7f107cdbb640 1 Processor -- start 2023-10-11T06:12:53.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.301+0000 7f107cdbb640 1 -- start start 2023-10-11T06:12:53.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f107cdbb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f107815d4f0 0x7f10780cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f107cdbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 0x7f10780c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f107cdbb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1078071a90 con 0x7f10780c9190 2023-10-11T06:12:53.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f107cdbb640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1078072310 con 0x7f107815d4f0 2023-10-11T06:12:53.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f107cdbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1078070280 con 0x7f10780ca300 2023-10-11T06:12:53.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f1076d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f10780c9190 0x7f107815dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46118/0 (socket says 172.21.15.84:46118) 2023-10-11T06:12:53.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f1076d9d640 1 -- 172.21.15.84:0/3692602152 learned_addr learned my addr 172.21.15.84:0/3692602152 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:53.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.303+0000 7f107659c640 1 --2- 172.21.15.84:0/3692602152 >> v2:172.21.15.99:3300/0 conn(0x7f107815d4f0 0x7f10780cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f106ffff640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 0x7f10780c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922153332 0 0) 0x7f1078071a90 con 0x7f10780c9190 2023-10-11T06:12:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1064003520 con 0x7f10780c9190 2023-10-11T06:12:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267710969 0 0) 0x7f1064003520 con 0x7f10780c9190 2023-10-11T06:12:53.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 msgr2=0x7f10780c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 0x7f10780c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 >> v2:172.21.15.99:3300/0 conn(0x7f107815d4f0 msgr2=0x7f10780cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 --2- 172.21.15.84:0/3692602152 >> v2:172.21.15.99:3300/0 conn(0x7f107815d4f0 0x7f10780cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1078071cc0 con 0x7f10780c9190 2023-10-11T06:12:53.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f10700019f0 con 0x7f10780c9190 2023-10-11T06:12:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 >> v1:172.21.15.84:6789/0 conn(0x7f10780c9190 legacy=0x7f107815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1064003970 0x7f1064003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 0x7f10640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1064004d00 0x7f1064005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078072310 con 0x7f1064003970 2023-10-11T06:12:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078070280 con 0x7f1064004d00 2023-10-11T06:12:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106400b980 con 0x7f10780ca300 2023-10-11T06:12:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.304+0000 7f107659c640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1064003970 0x7f1064003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.305+0000 7f106ffff640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 0x7f10640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.305+0000 7f107659c640 1 -- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 msgr2=0x7f10640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.305+0000 7f107659c640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10780ca300 0x7f10640047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.305+0000 7f107659c640 1 -- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1064004d00 msgr2=0x7f1064005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.305+0000 7f107659c640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1064004d00 0x7f1064005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.305+0000 7f107659c640 1 -- 172.21.15.84:0/3692602152 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064003520 con 0x7f1064003970 2023-10-11T06:12:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.306+0000 7f107659c640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1064003970 0x7f1064003d80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1068009ab0 tx=0x7f10640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178ba23f4263bea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:53.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.306+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068005990 con 0x7f1064003970 2023-10-11T06:12:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.306+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068005af0 con 0x7f1064003970 2023-10-11T06:12:53.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.306+0000 7f1075d9b640 1 -- 172.21.15.84:0/3692602152 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068005ea0 con 0x7f1064003970 2023-10-11T06:12:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.307+0000 7f107cdbb640 1 -- 172.21.15.84:0/3692602152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1064003970 msgr2=0x7f1064003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.307+0000 7f107cdbb640 1 --2- 172.21.15.84:0/3692602152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1064003970 0x7f1064003d80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1068009ab0 tx=0x7f10640057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.307+0000 7f1076d9d640 1 -- 172.21.15.84:0/3692602152 reap_dead start 2023-10-11T06:12:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.307+0000 7f107cdbb640 1 -- 172.21.15.84:0/3692602152 shutdown_connections 2023-10-11T06:12:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.307+0000 7f107cdbb640 1 -- 172.21.15.84:0/3692602152 >> 172.21.15.84:0/3692602152 conn(0x7f1078069c70 msgr2=0x7f107806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:53.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.307+0000 7f107cdbb640 1 -- 172.21.15.84:0/3692602152 shutdown_connections 2023-10-11T06:12:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.307+0000 7f107cdbb640 1 -- 172.21.15.84:0/3692602152 wait complete. 2023-10-11T06:12:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.309+0000 7f107cdbb640 1 Processor -- start 2023-10-11T06:12:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.309+0000 7f107cdbb640 1 -- start start 2023-10-11T06:12:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f107cdbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1064003970 0x7f10780cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f107cdbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10780c57b0 0x7f10781f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f107cdbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10781558e0 0x7f1078154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f107cdbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10780c77b0 con 0x7f10780c57b0 2023-10-11T06:12:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f107cdbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10781f5fd0 con 0x7f10781558e0 2023-10-11T06:12:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f107cdbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10780cc710 con 0x7f1064003970 2023-10-11T06:12:53.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f106ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10780c57b0 0x7f10781f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f106ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10780c57b0 0x7f10781f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39926/0 (socket says 172.21.15.84:39926) 2023-10-11T06:12:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.310+0000 7f106ffff640 1 -- 172.21.15.84:0/2407614961 learned_addr learned my addr 172.21.15.84:0/2407614961 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106ffff640 1 -- 172.21.15.84:0/2407614961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1064003970 msgr2=0x7f10780cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106ffff640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1064003970 0x7f10780cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106ffff640 1 -- 172.21.15.84:0/2407614961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10781558e0 msgr2=0x7f1078154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106ffff640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10781558e0 0x7f1078154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106ffff640 1 -- 172.21.15.84:0/2407614961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10781f5a10 con 0x7f10780c57b0 2023-10-11T06:12:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106ffff640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10780c57b0 0x7f10781f50c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1060010860 tx=0x7f1060010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106effd640 1 -- 172.21.15.84:0/2407614961 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060009610 con 0x7f10780c57b0 2023-10-11T06:12:53.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106effd640 1 -- 172.21.15.84:0/2407614961 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106000eb60 con 0x7f10780c57b0 2023-10-11T06:12:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f106effd640 1 -- 172.21.15.84:0/2407614961 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106001c400 con 0x7f10780c57b0 2023-10-11T06:12:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.311+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10781f5cf0 con 0x7f10780c57b0 2023-10-11T06:12:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.312+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10781f5600 con 0x7f10780c57b0 2023-10-11T06:12:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.313+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10780c7230 con 0x7f10780c57b0 2023-10-11T06:12:53.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.314+0000 7f106effd640 1 -- 172.21.15.84:0/2407614961 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f106001c560 con 0x7f10780c57b0 2023-10-11T06:12:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.315+0000 7f106effd640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105806a830 0x7f105806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.315+0000 7f106effd640 1 -- 172.21.15.84:0/2407614961 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f1060089790 con 0x7f10780c57b0 2023-10-11T06:12:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.315+0000 7f107659c640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105806a830 0x7f105806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.316+0000 7f107659c640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105806a830 0x7f105806cce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1068035000 tx=0x7f106802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:53.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.319+0000 7f106effd640 1 -- 172.21.15.84:0/2407614961 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f10780c7230 con 0x7f10780c57b0 2023-10-11T06:12:53.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.510+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10780c7230 con 0x7f10780c57b0 2023-10-11T06:12:53.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.514+0000 7f106effd640 1 -- 172.21.15.84:0/2407614961 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7f10780c7230 con 0x7f10780c57b0 2023-10-11T06:12:53.516 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:53.516 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:53.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.516+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105806a830 msgr2=0x7f105806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105806a830 0x7f105806cce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1068035000 tx=0x7f106802e000 comp rx=0 tx=0).stop 2023-10-11T06:12:53.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10780c57b0 msgr2=0x7f10781f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10780c57b0 0x7f10781f50c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1060010860 tx=0x7f1060010d20 comp rx=0 tx=0).stop 2023-10-11T06:12:53.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 shutdown_connections 2023-10-11T06:12:53.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f105806a830 0x7f105806cce0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10781558e0 0x7f1078154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10780c57b0 0x7f10781f50c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 --2- 172.21.15.84:0/2407614961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1064003970 0x7f10780cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 >> 172.21.15.84:0/2407614961 conn(0x7f1078069c70 msgr2=0x7f107815da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:53.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 shutdown_connections 2023-10-11T06:12:53.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.517+0000 7f107cdbb640 1 -- 172.21.15.84:0/2407614961 wait complete. 2023-10-11T06:12:53.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:12:38.260176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:12:39.011036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-11T06:12:50.853138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:53.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:53.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.751+0000 7fa5062ca640 1 Processor -- start 2023-10-11T06:12:53.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.751+0000 7fa5062ca640 1 -- start start 2023-10-11T06:12:53.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa5062ca640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa5062ca640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5000cd6c0 0x7fa5000d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa5062ca640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5000d3460 con 0x7fa5000d3000 2023-10-11T06:12:53.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa5062ca640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5000cf220 con 0x7fa5000cd6c0 2023-10-11T06:12:53.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa5062ca640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000cf500 con 0x7fa50015f7e0 2023-10-11T06:12:53.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa4f7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5000cd6c0 0x7fa5000d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa504867640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa5000d3000 0x7fa50015fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46126/0 (socket says 172.21.15.84:46126) 2023-10-11T06:12:53.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa4f7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5000cd6c0 0x7fa5000d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39938/0 (socket says 172.21.15.84:39938) 2023-10-11T06:12:53.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.753+0000 7fa504867640 1 -- 172.21.15.84:0/223386843 learned_addr learned my addr 172.21.15.84:0/223386843 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:53.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.754+0000 7fa4fffff640 1 --2- 172.21.15.84:0/223386843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.754+0000 7fa4f7fff640 1 -- 172.21.15.84:0/223386843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 msgr2=0x7fa5000cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:53.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.754+0000 7fa4f7fff640 1 --2- 172.21.15.84:0/223386843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.754+0000 7fa4f7fff640 1 -- 172.21.15.84:0/223386843 >> v1:172.21.15.84:6789/0 conn(0x7fa5000d3000 legacy=0x7fa50015fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.754+0000 7fa4f7fff640 1 -- 172.21.15.84:0/223386843 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000cf7e0 con 0x7fa5000cd6c0 2023-10-11T06:12:53.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.754+0000 7fa4fffff640 1 --2- 172.21.15.84:0/223386843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:53.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.755+0000 7fa4f7fff640 1 --2- 172.21.15.84:0/223386843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa5000d2ac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80097a0 tx=0x7fa4e802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96390b2e9d386c93 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:53.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.755+0000 7fa4ff7fe640 1 -- 172.21.15.84:0/223386843 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e802d040 con 0x7fa5000cd6c0 2023-10-11T06:12:53.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.755+0000 7fa4ff7fe640 1 -- 172.21.15.84:0/223386843 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8031810 con 0x7fa5000cd6c0 2023-10-11T06:12:53.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.755+0000 7fa4ff7fe640 1 -- 172.21.15.84:0/223386843 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8031ba0 con 0x7fa5000cd6c0 2023-10-11T06:12:53.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 -- 172.21.15.84:0/223386843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 msgr2=0x7fa5000d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 --2- 172.21.15.84:0/223386843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa5000d2ac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80097a0 tx=0x7fa4e802b8b0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 -- 172.21.15.84:0/223386843 shutdown_connections 2023-10-11T06:12:53.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 --2- 172.21.15.84:0/223386843 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa5000d2ac0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 --2- 172.21.15.84:0/223386843 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 -- 172.21.15.84:0/223386843 >> 172.21.15.84:0/223386843 conn(0x7fa5000713a0 msgr2=0x7fa5000717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:53.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 -- 172.21.15.84:0/223386843 shutdown_connections 2023-10-11T06:12:53.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.756+0000 7fa5062ca640 1 -- 172.21.15.84:0/223386843 wait complete. 2023-10-11T06:12:53.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.758+0000 7fa5062ca640 1 Processor -- start 2023-10-11T06:12:53.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.758+0000 7fa5062ca640 1 -- start start 2023-10-11T06:12:53.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa5062ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa500156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa5062ca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5000d3000 0x7fa5001585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa5062ca640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000d0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa5062ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa500157a10 con 0x7fa5000cd6c0 2023-10-11T06:12:53.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa5062ca640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa500156840 con 0x7fa5000d3000 2023-10-11T06:12:53.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa5062ca640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa500156d90 con 0x7fa50015f7e0 2023-10-11T06:12:53.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa4fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa500156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa4fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa500156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39950/0 (socket says 172.21.15.84:39950) 2023-10-11T06:12:53.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.759+0000 7fa4fffff640 1 -- 172.21.15.84:0/3556828024 learned_addr learned my addr 172.21.15.84:0/3556828024 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:53.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa504867640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fffff640 1 -- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 msgr2=0x7fa5000d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fffff640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000d0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fffff640 1 -- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5000d3000 msgr2=0x7fa5001585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4f7fff640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5000d3000 0x7fa5001585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fffff640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5000d3000 0x7fa5001585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fffff640 1 -- 172.21.15.84:0/3556828024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8009400 con 0x7fa5000cd6c0 2023-10-11T06:12:53.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4f7fff640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5000d3000 0x7fa5001585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:53.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fffff640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa500156300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c820 tx=0x7fa4f000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:53.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fd7fa640 1 -- 172.21.15.84:0/3556828024 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0007a30 con 0x7fa5000cd6c0 2023-10-11T06:12:53.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fd7fa640 1 -- 172.21.15.84:0/3556828024 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f0007b90 con 0x7fa5000cd6c0 2023-10-11T06:12:53.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.760+0000 7fa4fd7fa640 1 -- 172.21.15.84:0/3556828024 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000ce10 con 0x7fa5000cd6c0 2023-10-11T06:12:53.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.761+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa500157070 con 0x7fa5000cd6c0 2023-10-11T06:12:53.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.761+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa500157630 con 0x7fa5000cd6c0 2023-10-11T06:12:53.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.762+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa500073ed0 con 0x7fa5000cd6c0 2023-10-11T06:12:53.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.763+0000 7fa4fd7fa640 1 -- 172.21.15.84:0/3556828024 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa4f00113f0 con 0x7fa5000cd6c0 2023-10-11T06:12:53.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.764+0000 7fa4fd7fa640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4e006a830 0x7fa4e006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:53.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.764+0000 7fa4fd7fa640 1 -- 172.21.15.84:0/3556828024 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fa4f0089260 con 0x7fa5000cd6c0 2023-10-11T06:12:53.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.764+0000 7fa4f7fff640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4e006a830 0x7fa4e006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:53.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.765+0000 7fa4f7fff640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4e006a830 0x7fa4e006cce0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8036000 tx=0x7fa4e803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:53.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.769+0000 7fa4fd7fa640 1 -- 172.21.15.84:0/3556828024 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa4f0054340 con 0x7fa5000cd6c0 2023-10-11T06:12:53.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.961+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5000b5030 con 0x7fa5000cd6c0 2023-10-11T06:12:53.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.962+0000 7fa4fd7fa640 1 -- 172.21.15.84:0/3556828024 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4f0057960 con 0x7fa5000cd6c0 2023-10-11T06:12:53.963 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:53.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4e006a830 msgr2=0x7fa4e006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4e006a830 0x7fa4e006cce0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8036000 tx=0x7fa4e803c040 comp rx=0 tx=0).stop 2023-10-11T06:12:53.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 msgr2=0x7fa500156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:53.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa500156300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c820 tx=0x7fa4f000cce0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 shutdown_connections 2023-10-11T06:12:53.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4e006a830 0x7fa4e006cce0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa50015f7e0 0x7fa5000d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5000d3000 0x7fa5001585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 --2- 172.21.15.84:0/3556828024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5000cd6c0 0x7fa500156300 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:53.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 >> 172.21.15.84:0/3556828024 conn(0x7fa5000713a0 msgr2=0x7fa5000b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:53.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 shutdown_connections 2023-10-11T06:12:53.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:53.965+0000 7fa5062ca640 1 -- 172.21.15.84:0/3556828024 wait complete. 2023-10-11T06:12:53.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:54.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.202+0000 7f3dce2b6640 1 Processor -- start 2023-10-11T06:12:54.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.203+0000 7f3dce2b6640 1 -- start start 2023-10-11T06:12:54.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.204+0000 7f3dce2b6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3dc815f7e0 0x7f3dc80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.204+0000 7f3dce2b6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc80cf220 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.204+0000 7f3dce2b6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc80cf500 con 0x7f3dc80d31d0 2023-10-11T06:12:54.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.204+0000 7f3dce2b6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc80cf7e0 con 0x7f3dc815f7e0 2023-10-11T06:12:54.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc77fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3dc80cd6c0 0x7f3dc80d2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46138/0 (socket says 172.21.15.84:46138) 2023-10-11T06:12:54.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc77fe640 1 -- 172.21.15.84:0/1094066242 learned_addr learned my addr 172.21.15.84:0/1094066242 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:54.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc7fff640 1 --2- 172.21.15.84:0/1094066242 >> v2:172.21.15.84:3300/0 conn(0x7f3dc815f7e0 0x7f3dc80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc6ffd640 1 -- 172.21.15.84:0/1094066242 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273786846 0 0) 0x7f3dc80cf220 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc6ffd640 1 -- 172.21.15.84:0/1094066242 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db4003520 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc7fff640 1 -- 172.21.15.84:0/1094066242 >> v1:172.21.15.99:6789/0 conn(0x7f3dc80d31d0 legacy=0x7f3dc80d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:54.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc7fff640 1 -- 172.21.15.84:0/1094066242 >> v1:172.21.15.84:6789/0 conn(0x7f3dc80cd6c0 legacy=0x7f3dc80d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.205+0000 7f3dc7fff640 1 -- 172.21.15.84:0/1094066242 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc80cfac0 con 0x7f3dc815f7e0 2023-10-11T06:12:54.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.206+0000 7f3dc7fff640 1 --2- 172.21.15.84:0/1094066242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc815f7e0 0x7f3dc80cd180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3db80097a0 tx=0x7f3db802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d3ae02d53f5427 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:54.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.206+0000 7f3dc6ffd640 1 -- 172.21.15.84:0/1094066242 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8031830 con 0x7f3dc815f7e0 2023-10-11T06:12:54.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.206+0000 7f3dc6ffd640 1 -- 172.21.15.84:0/1094066242 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8031990 con 0x7f3dc815f7e0 2023-10-11T06:12:54.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dc6ffd640 1 -- 172.21.15.84:0/1094066242 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8031d00 con 0x7f3dc815f7e0 2023-10-11T06:12:54.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dce2b6640 1 -- 172.21.15.84:0/1094066242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc815f7e0 msgr2=0x7f3dc80cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/1094066242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc815f7e0 0x7f3dc80cd180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3db80097a0 tx=0x7f3db802b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dce2b6640 1 -- 172.21.15.84:0/1094066242 shutdown_connections 2023-10-11T06:12:54.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/1094066242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc815f7e0 0x7f3dc80cd180 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dce2b6640 1 -- 172.21.15.84:0/1094066242 >> 172.21.15.84:0/1094066242 conn(0x7f3dc80713a0 msgr2=0x7f3dc80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:54.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dce2b6640 1 -- 172.21.15.84:0/1094066242 shutdown_connections 2023-10-11T06:12:54.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.207+0000 7f3dce2b6640 1 -- 172.21.15.84:0/1094066242 wait complete. 2023-10-11T06:12:54.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.209+0000 7f3dce2b6640 1 Processor -- start 2023-10-11T06:12:54.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.209+0000 7f3dce2b6640 1 -- start start 2023-10-11T06:12:54.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dce2b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc80cd6c0 0x7f3dc81f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dce2b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc80d31d0 0x7f3dc8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dce2b6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dc815f7e0 0x7f3dc80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dce2b6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc81f2d60 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dce2b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80cade0 con 0x7f3dc80d31d0 2023-10-11T06:12:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dce2b6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80ca5b0 con 0x7f3dc815f7e0 2023-10-11T06:12:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc80cd6c0 0x7f3dc81f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.210+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc80cd6c0 0x7f3dc81f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39976/0 (socket says 172.21.15.84:39976) 2023-10-11T06:12:54.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.211+0000 7f3dc7fff640 1 -- 172.21.15.84:0/2099080768 learned_addr learned my addr 172.21.15.84:0/2099080768 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.211+0000 7f3dc7fff640 1 -- 172.21.15.84:0/2099080768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dc815f7e0 msgr2=0x7f3dc80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.211+0000 7f3dc7fff640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dc815f7e0 0x7f3dc80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.211+0000 7f3dc7fff640 1 -- 172.21.15.84:0/2099080768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc80d31d0 msgr2=0x7f3dc8156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.211+0000 7f3dc7fff640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc80d31d0 0x7f3dc8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.211+0000 7f3dc7fff640 1 -- 172.21.15.84:0/2099080768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8009400 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.211+0000 7f3dc7fff640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc80cd6c0 0x7f3dc81f2820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000c00 tx=0x7f3db80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:54.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.212+0000 7f3dc4ff9640 1 -- 172.21.15.84:0/2099080768 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db803a830 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.212+0000 7f3dc4ff9640 1 -- 172.21.15.84:0/2099080768 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db803a990 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.212+0000 7f3dc4ff9640 1 -- 172.21.15.84:0/2099080768 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db803ad40 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.212+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc80ca820 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.212+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc80b5f90 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.213+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc80b5df0 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.214+0000 7f3dc4ff9640 1 -- 172.21.15.84:0/2099080768 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3db804d450 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.215+0000 7f3dc4ff9640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3da806a830 0x7f3da806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.215+0000 7f3dc4ff9640 1 -- 172.21.15.84:0/2099080768 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f3db80ab2b0 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.215+0000 7f3dc77fe640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3da806a830 0x7f3da806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.216+0000 7f3dc77fe640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3da806a830 0x7f3da806cce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3db0000f60 tx=0x7f3db00078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:54.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.219+0000 7f3dc4ff9640 1 -- 172.21.15.84:0/2099080768 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3dc80b5df0 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.398+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3dc80b5df0 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.399+0000 7f3dc4ff9640 1 -- 172.21.15.84:0/2099080768 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3dc80b5df0 con 0x7f3dc80cd6c0 2023-10-11T06:12:54.401 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:54.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3da806a830 msgr2=0x7f3da806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3da806a830 0x7f3da806cce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3db0000f60 tx=0x7f3db00078f0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc80cd6c0 msgr2=0x7f3dc81f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc80cd6c0 0x7f3dc81f2820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3db8000c00 tx=0x7f3db80046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 shutdown_connections 2023-10-11T06:12:54.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3da806a830 0x7f3da806cce0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3dc815f7e0 0x7f3dc80cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc80d31d0 0x7f3dc8156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 --2- 172.21.15.84:0/2099080768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dc80cd6c0 0x7f3dc81f2820 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 >> 172.21.15.84:0/2099080768 conn(0x7f3dc80713a0 msgr2=0x7f3dc806b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:54.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 shutdown_connections 2023-10-11T06:12:54.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.402+0000 7f3dce2b6640 1 -- 172.21.15.84:0/2099080768 wait complete. 2023-10-11T06:12:54.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.630+0000 7f638d315640 1 Processor -- start 2023-10-11T06:12:54.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.630+0000 7f638d315640 1 -- start start 2023-10-11T06:12:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.631+0000 7f638d315640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6388154570 0x7f638815b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.631+0000 7f638d315640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.632+0000 7f638d315640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638815b680 con 0x7f638806fee0 2023-10-11T06:12:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.632+0000 7f638d315640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6388070420 con 0x7f6388154570 2023-10-11T06:12:54.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.632+0000 7f638d315640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6388071e00 con 0x7f6388154940 2023-10-11T06:12:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.632+0000 7f6387fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6388154570 0x7f638815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.632+0000 7f637ffff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.632+0000 7f6387fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6388154570 0x7f638815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:54398/0 (socket says 172.21.15.84:54398) 2023-10-11T06:12:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.632+0000 7f6387fff640 1 -- 172.21.15.84:0/4119801499 learned_addr learned my addr 172.21.15.84:0/4119801499 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.633+0000 7f637ffff640 1 -- 172.21.15.84:0/4119801499 >> v2:172.21.15.99:3300/0 conn(0x7f6388154570 msgr2=0x7f638815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.633+0000 7f637ffff640 1 --2- 172.21.15.84:0/4119801499 >> v2:172.21.15.99:3300/0 conn(0x7f6388154570 0x7f638815b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.633+0000 7f637ffff640 1 -- 172.21.15.84:0/4119801499 >> v1:172.21.15.99:6789/0 conn(0x7f638806fee0 legacy=0x7f63880affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.633+0000 7f637ffff640 1 -- 172.21.15.84:0/4119801499 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880720e0 con 0x7f6388154940 2023-10-11T06:12:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.635+0000 7f637ffff640 1 --2- 172.21.15.84:0/4119801499 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880cc450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63700097a0 tx=0x7f637002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21ef48b5419dd51 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.635+0000 7f63877fe640 1 -- 172.21.15.84:0/4119801499 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370031830 con 0x7f6388154940 2023-10-11T06:12:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f63877fe640 1 -- 172.21.15.84:0/4119801499 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6370031990 con 0x7f6388154940 2023-10-11T06:12:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f63877fe640 1 -- 172.21.15.84:0/4119801499 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370031d20 con 0x7f6388154940 2023-10-11T06:12:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 -- 172.21.15.84:0/4119801499 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 msgr2=0x7f63880cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 --2- 172.21.15.84:0/4119801499 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880cc450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63700097a0 tx=0x7f637002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 -- 172.21.15.84:0/4119801499 shutdown_connections 2023-10-11T06:12:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 --2- 172.21.15.84:0/4119801499 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880cc450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 --2- 172.21.15.84:0/4119801499 >> v2:172.21.15.99:3300/0 conn(0x7f6388154570 0x7f638815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 -- 172.21.15.84:0/4119801499 >> 172.21.15.84:0/4119801499 conn(0x7f638806bfb0 msgr2=0x7f638806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 -- 172.21.15.84:0/4119801499 shutdown_connections 2023-10-11T06:12:54.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.636+0000 7f638d315640 1 -- 172.21.15.84:0/4119801499 wait complete. 2023-10-11T06:12:54.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.638+0000 7f638d315640 1 Processor -- start 2023-10-11T06:12:54.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.638+0000 7f638d315640 1 -- start start 2023-10-11T06:12:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f638d315640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638806fee0 0x7f63880c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f638d315640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388154570 0x7f63880c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f638d315640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f638d315640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63880c6880 con 0x7f6388154570 2023-10-11T06:12:54.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f638d315640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63880c5c00 con 0x7f638806fee0 2023-10-11T06:12:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f638d315640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63880c5ee0 con 0x7f6388154940 2023-10-11T06:12:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f637ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388154570 0x7f63880c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f637ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388154570 0x7f63880c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39992/0 (socket says 172.21.15.84:39992) 2023-10-11T06:12:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.639+0000 7f637ffff640 1 -- 172.21.15.84:0/2118778853 learned_addr learned my addr 172.21.15.84:0/2118778853 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:54.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f6387fff640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638806fee0 0x7f63880c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f638cb14640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f637ffff640 1 -- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 msgr2=0x7f63880c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f637ffff640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f637ffff640 1 -- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638806fee0 msgr2=0x7f63880c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f637ffff640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638806fee0 0x7f63880c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f637ffff640 1 -- 172.21.15.84:0/2118778853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370009400 con 0x7f6388154570 2023-10-11T06:12:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f6387fff640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638806fee0 0x7f63880c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f638cb14640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:54.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.640+0000 7f637ffff640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388154570 0x7f63880c6ca0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f63700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:54.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.641+0000 7f63857fa640 1 -- 172.21.15.84:0/2118778853 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637003a830 con 0x7f6388154570 2023-10-11T06:12:54.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.641+0000 7f63857fa640 1 -- 172.21.15.84:0/2118778853 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637003a9b0 con 0x7f6388154570 2023-10-11T06:12:54.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.641+0000 7f63857fa640 1 -- 172.21.15.84:0/2118778853 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637003ad60 con 0x7f6388154570 2023-10-11T06:12:54.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.641+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63880c61c0 con 0x7f6388154570 2023-10-11T06:12:54.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.641+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63880c4e00 con 0x7f6388154570 2023-10-11T06:12:54.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.642+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638804e890 con 0x7f6388154570 2023-10-11T06:12:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.643+0000 7f63857fa640 1 -- 172.21.15.84:0/2118778853 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f637004d450 con 0x7f6388154570 2023-10-11T06:12:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.644+0000 7f63857fa640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f636806a830 0x7f636806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.644+0000 7f63857fa640 1 -- 172.21.15.84:0/2118778853 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f63700ab200 con 0x7f6388154570 2023-10-11T06:12:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.644+0000 7f6387fff640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f636806a830 0x7f636806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:54.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.645+0000 7f6387fff640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f636806a830 0x7f636806cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6378009ac0 tx=0x7f6378008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:54.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.648+0000 7f63857fa640 1 -- 172.21.15.84:0/2118778853 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f63700762c0 con 0x7f6388154570 2023-10-11T06:12:54.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.837+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63880cf340 con 0x7f6388154570 2023-10-11T06:12:54.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.838+0000 7f63857fa640 1 -- 172.21.15.84:0/2118778853 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6370035020 con 0x7f6388154570 2023-10-11T06:12:54.840 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:54.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f636806a830 msgr2=0x7f636806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f636806a830 0x7f636806cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6378009ac0 tx=0x7f6378008040 comp rx=0 tx=0).stop 2023-10-11T06:12:54.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388154570 msgr2=0x7f63880c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:54.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388154570 0x7f63880c6ca0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6370000c00 tx=0x7f63700046c0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 shutdown_connections 2023-10-11T06:12:54.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f636806a830 0x7f636806cce0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6388154940 0x7f63880c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6388154570 0x7f63880c6ca0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 --2- 172.21.15.84:0/2118778853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f638806fee0 0x7f63880c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:54.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.840+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 >> 172.21.15.84:0/2118778853 conn(0x7f638806bfb0 msgr2=0x7f638806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:54.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.841+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 shutdown_connections 2023-10-11T06:12:54.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:54.841+0000 7f638d315640 1 -- 172.21.15.84:0/2118778853 wait complete. 2023-10-11T06:12:54.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:55.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.080+0000 7f78caeec640 1 Processor -- start 2023-10-11T06:12:55.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.080+0000 7f78caeec640 1 -- start start 2023-10-11T06:12:55.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.081+0000 7f78caeec640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f78c415a0b0 0x7f78c40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78caeec640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c4071380 con 0x7f78c40ced30 2023-10-11T06:12:55.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78caeec640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c406de10 con 0x7f78c40c8d00 2023-10-11T06:12:55.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78caeec640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c406e0f0 con 0x7f78c415a0b0 2023-10-11T06:12:55.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78c9489640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f78c40ced30 0x7f78c406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46144/0 (socket says 172.21.15.84:46144) 2023-10-11T06:12:55.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78c9489640 1 -- 172.21.15.84:0/133824257 learned_addr learned my addr 172.21.15.84:0/133824257 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:55.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379591433 0 0) 0x7f78c4071380 con 0x7f78c40ced30 2023-10-11T06:12:55.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78c8c88640 1 --2- 172.21.15.84:0/133824257 >> v2:172.21.15.99:3300/0 conn(0x7f78c415a0b0 0x7f78c40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.082+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78b4003520 con 0x7f78c40ced30 2023-10-11T06:12:55.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 661080505 0 0) 0x7f78b4003520 con 0x7f78c40ced30 2023-10-11T06:12:55.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 >> v2:172.21.15.99:3300/0 conn(0x7f78c415a0b0 msgr2=0x7f78c40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 --2- 172.21.15.84:0/133824257 >> v2:172.21.15.99:3300/0 conn(0x7f78c415a0b0 0x7f78c40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 >> v1:172.21.15.99:6789/0 conn(0x7f78c40c8d00 legacy=0x7f78c40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c4070dc0 con 0x7f78c40ced30 2023-10-11T06:12:55.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f78bc0019f0 con 0x7f78c40ced30 2023-10-11T06:12:55.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 >> v1:172.21.15.84:6789/0 conn(0x7f78c40ced30 legacy=0x7f78c406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78b4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78b40047c0 0x7f78b4004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b4005370 0x7f78b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c406e0f0 con 0x7f78b4003970 2023-10-11T06:12:55.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b400c1f0 con 0x7f78b4005370 2023-10-11T06:12:55.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b400c3b0 con 0x7f78b40047c0 2023-10-11T06:12:55.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c8c88640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78b4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c8c88640 1 -- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78b40047c0 msgr2=0x7f78b4004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c9489640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b4005370 0x7f78b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.083+0000 7f78c3fff640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78b40047c0 0x7f78b4004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.084+0000 7f78c8c88640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78b40047c0 0x7f78b4004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.084+0000 7f78c8c88640 1 -- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b4005370 msgr2=0x7f78b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.084+0000 7f78c8c88640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78b4005370 0x7f78b400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.084+0000 7f78c8c88640 1 -- 172.21.15.84:0/133824257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b4003520 con 0x7f78b4003970 2023-10-11T06:12:55.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.084+0000 7f78c9489640 1 -- 172.21.15.84:0/133824257 reap_dead start 2023-10-11T06:12:55.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.084+0000 7f78c3fff640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78b40047c0 0x7f78b4004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:55.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78c8c88640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78b4003d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f78b800ba30 tx=0x7f78b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a58089fc9e813e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8032640 con 0x7f78b4003970 2023-10-11T06:12:55.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b80327a0 con 0x7f78b4003970 2023-10-11T06:12:55.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78c37fe640 1 -- 172.21.15.84:0/133824257 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8032b30 con 0x7f78b4003970 2023-10-11T06:12:55.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78caeec640 1 -- 172.21.15.84:0/133824257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 msgr2=0x7f78b4003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78caeec640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78b4003d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f78b800ba30 tx=0x7f78b40057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78caeec640 1 -- 172.21.15.84:0/133824257 shutdown_connections 2023-10-11T06:12:55.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78caeec640 1 --2- 172.21.15.84:0/133824257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78b4003d80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.085+0000 7f78caeec640 1 -- 172.21.15.84:0/133824257 >> 172.21.15.84:0/133824257 conn(0x7f78c4069c70 msgr2=0x7f78c406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:55.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.086+0000 7f78caeec640 1 -- 172.21.15.84:0/133824257 shutdown_connections 2023-10-11T06:12:55.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.086+0000 7f78caeec640 1 -- 172.21.15.84:0/133824257 wait complete. 2023-10-11T06:12:55.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.087+0000 7f78caeec640 1 Processor -- start 2023-10-11T06:12:55.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.087+0000 7f78caeec640 1 -- start start 2023-10-11T06:12:55.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.088+0000 7f78caeec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78c414bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.088+0000 7f78caeec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78c41f2870 0x7f78c40c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.088+0000 7f78caeec640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78c40c3a70 0x7f78c41f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.088+0000 7f78caeec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40c86d0 con 0x7f78b4003970 2023-10-11T06:12:55.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.088+0000 7f78caeec640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40c3ee0 con 0x7f78c41f2870 2023-10-11T06:12:55.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.088+0000 7f78caeec640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41f23e0 con 0x7f78c40c3a70 2023-10-11T06:12:55.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78c414bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78c414bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40004/0 (socket says 172.21.15.84:40004) 2023-10-11T06:12:55.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 -- 172.21.15.84:0/2158493256 learned_addr learned my addr 172.21.15.84:0/2158493256 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:55.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 -- 172.21.15.84:0/2158493256 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78c40c3a70 msgr2=0x7f78c41f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78c40c3a70 0x7f78c41f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 -- 172.21.15.84:0/2158493256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78c41f2870 msgr2=0x7f78c40c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78c41f2870 0x7f78c40c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 -- 172.21.15.84:0/2158493256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40c5ad0 con 0x7f78b4003970 2023-10-11T06:12:55.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c8c88640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78c414bb30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f78b8035040 tx=0x7f78b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.089+0000 7f78c17fa640 1 -- 172.21.15.84:0/2158493256 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b803baf0 con 0x7f78b4003970 2023-10-11T06:12:55.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.090+0000 7f78c17fa640 1 -- 172.21.15.84:0/2158493256 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b803bc50 con 0x7f78b4003970 2023-10-11T06:12:55.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.090+0000 7f78c17fa640 1 -- 172.21.15.84:0/2158493256 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8030410 con 0x7f78b4003970 2023-10-11T06:12:55.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.090+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c41f5e60 con 0x7f78b4003970 2023-10-11T06:12:55.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.090+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c41f1d90 con 0x7f78b4003970 2023-10-11T06:12:55.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.091+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c4073ed0 con 0x7f78b4003970 2023-10-11T06:12:55.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.092+0000 7f78c17fa640 1 -- 172.21.15.84:0/2158493256 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f78b8030570 con 0x7f78b4003970 2023-10-11T06:12:55.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.092+0000 7f78c17fa640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78a806a830 0x7f78a806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.093+0000 7f78c17fa640 1 -- 172.21.15.84:0/2158493256 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f78b80abb10 con 0x7f78b4003970 2023-10-11T06:12:55.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.093+0000 7f78c3fff640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78a806a830 0x7f78a806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.093+0000 7f78c3fff640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78a806a830 0x7f78a806cce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f78c4154ee0 tx=0x7f78b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.097+0000 7f78c17fa640 1 -- 172.21.15.84:0/2158493256 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f78b807a6f0 con 0x7f78b4003970 2023-10-11T06:12:55.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.274+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78c4073ed0 con 0x7f78b4003970 2023-10-11T06:12:55.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.274+0000 7f78c17fa640 1 -- 172.21.15.84:0/2158493256 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78b8036020 con 0x7f78b4003970 2023-10-11T06:12:55.277 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:55.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78a806a830 msgr2=0x7f78a806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78a806a830 0x7f78a806cce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f78c4154ee0 tx=0x7f78b0009040 comp rx=0 tx=0).stop 2023-10-11T06:12:55.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 msgr2=0x7f78c414bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78c414bb30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f78b8035040 tx=0x7f78b8004730 comp rx=0 tx=0).stop 2023-10-11T06:12:55.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 shutdown_connections 2023-10-11T06:12:55.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78a806a830 0x7f78a806cce0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78c40c3a70 0x7f78c41f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78c41f2870 0x7f78c40c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 --2- 172.21.15.84:0/2158493256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b4003970 0x7f78c414bb30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 >> 172.21.15.84:0/2158493256 conn(0x7f78c4069c70 msgr2=0x7f78c40caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:55.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 shutdown_connections 2023-10-11T06:12:55.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.277+0000 7f78caeec640 1 -- 172.21.15.84:0/2158493256 wait complete. 2023-10-11T06:12:55.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-11T06:12:55.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.496+0000 7f93fb70a640 1 Processor -- start 2023-10-11T06:12:55.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.496+0000 7f93fb70a640 1 -- start start 2023-10-11T06:12:55.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93fb70a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f40ced30 0x7f93f40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93fb70a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f93f40c8d00 0x7f93f406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93fb70a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f4071380 con 0x7f93f415a0b0 2023-10-11T06:12:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93fb70a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f406e0f0 con 0x7f93f40c8d00 2023-10-11T06:12:55.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93fb70a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40cbae0 con 0x7f93f40ced30 2023-10-11T06:12:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93f94a6640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f93f415a0b0 0x7f93f40ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46158/0 (socket says 172.21.15.84:46158) 2023-10-11T06:12:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93f94a6640 1 -- 172.21.15.84:0/3881509137 learned_addr learned my addr 172.21.15.84:0/3881509137 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:55.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93f8ca5640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f40ced30 0x7f93f40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93f9ca7640 1 --2- 172.21.15.84:0/3881509137 >> v2:172.21.15.99:3300/0 conn(0x7f93f40c8d00 0x7f93f406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.498+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749428642 0 0) 0x7f93f4071380 con 0x7f93f415a0b0 2023-10-11T06:12:55.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e0003520 con 0x7f93f415a0b0 2023-10-11T06:12:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1651528636 0 0) 0x7f93e0003520 con 0x7f93f415a0b0 2023-10-11T06:12:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f40ced30 msgr2=0x7f93f40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f40ced30 0x7f93f40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 >> v2:172.21.15.99:3300/0 conn(0x7f93f40c8d00 msgr2=0x7f93f406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 --2- 172.21.15.84:0/3881509137 >> v2:172.21.15.99:3300/0 conn(0x7f93f40c8d00 0x7f93f406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f406de10 con 0x7f93f415a0b0 2023-10-11T06:12:55.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f93f0001670 con 0x7f93f415a0b0 2023-10-11T06:12:55.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93f9ca7640 1 --2- 172.21.15.84:0/3881509137 >> v2:172.21.15.99:3300/0 conn(0x7f93f40c8d00 0x7f93f406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:55.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 >> v1:172.21.15.84:6789/0 conn(0x7f93f415a0b0 legacy=0x7f93f40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93e00039a0 0x7f93e0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00047c0 0x7f93e0004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f40ced30 0x7f93e0005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f406e0f0 con 0x7f93e00039a0 2023-10-11T06:12:55.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40cbae0 con 0x7f93e00047c0 2023-10-11T06:12:55.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e000bae0 con 0x7f93f40ced30 2023-10-11T06:12:55.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.499+0000 7f93f94a6640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93e00039a0 0x7f93e0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.500+0000 7f93f94a6640 1 -- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f40ced30 msgr2=0x7f93e0005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.500+0000 7f93f94a6640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f40ced30 0x7f93e0005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.500+0000 7f93f94a6640 1 -- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00047c0 msgr2=0x7f93e0004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.500+0000 7f93f94a6640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00047c0 0x7f93e0004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.500+0000 7f93f94a6640 1 -- 172.21.15.84:0/3881509137 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0003520 con 0x7f93e00039a0 2023-10-11T06:12:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.501+0000 7f93f94a6640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93e00039a0 0x7f93e0003db0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c240 tx=0x7f93e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138fcb103c7e112 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.501+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0005ad0 con 0x7f93e00039a0 2023-10-11T06:12:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.501+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f0005c30 con 0x7f93e00039a0 2023-10-11T06:12:55.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.501+0000 7f93ebfff640 1 -- 172.21.15.84:0/3881509137 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f002c8b0 con 0x7f93e00039a0 2023-10-11T06:12:55.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.501+0000 7f93fb70a640 1 -- 172.21.15.84:0/3881509137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93e00039a0 msgr2=0x7f93e0003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.501+0000 7f93fb70a640 1 --2- 172.21.15.84:0/3881509137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93e00039a0 0x7f93e0003db0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f93f000c240 tx=0x7f93e00057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.501+0000 7f93f9ca7640 1 -- 172.21.15.84:0/3881509137 reap_dead start 2023-10-11T06:12:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.502+0000 7f93fb70a640 1 -- 172.21.15.84:0/3881509137 shutdown_connections 2023-10-11T06:12:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.502+0000 7f93fb70a640 1 -- 172.21.15.84:0/3881509137 >> 172.21.15.84:0/3881509137 conn(0x7f93f4069c70 msgr2=0x7f93f406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.502+0000 7f93fb70a640 1 -- 172.21.15.84:0/3881509137 shutdown_connections 2023-10-11T06:12:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.502+0000 7f93fb70a640 1 -- 172.21.15.84:0/3881509137 wait complete. 2023-10-11T06:12:55.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.503+0000 7f93fb70a640 1 Processor -- start 2023-10-11T06:12:55.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.504+0000 7f93fb70a640 1 -- start start 2023-10-11T06:12:55.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.504+0000 7f93fb70a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00039a0 0x7f93f414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.504+0000 7f93fb70a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f41f2a60 0x7f93f4151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.504+0000 7f93fb70a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f40c7f90 0x7f93f40c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93fb70a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41f2fd0 con 0x7f93f40c7f90 2023-10-11T06:12:55.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93fb70a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41f32b0 con 0x7f93e00039a0 2023-10-11T06:12:55.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93fb70a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f41f3590 con 0x7f93f41f2a60 2023-10-11T06:12:55.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f40c7f90 0x7f93f40c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f40c7f90 0x7f93f40c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40024/0 (socket says 172.21.15.84:40024) 2023-10-11T06:12:55.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 -- 172.21.15.84:0/3863396971 learned_addr learned my addr 172.21.15.84:0/3863396971 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:55.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f94a6640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00039a0 0x7f93f414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f8ca5640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f41f2a60 0x7f93f4151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 -- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f41f2a60 msgr2=0x7f93f4151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f41f2a60 0x7f93f4151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 -- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00039a0 msgr2=0x7f93f414a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00039a0 0x7f93f414a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 -- 172.21.15.84:0/3863396971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f41f3870 con 0x7f93f40c7f90 2023-10-11T06:12:55.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.505+0000 7f93f9ca7640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f40c7f90 0x7f93f40c3b10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f93f40b01a0 tx=0x7f93ec007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.506+0000 7f93e9ffb640 1 -- 172.21.15.84:0/3863396971 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec003b90 con 0x7f93f40c7f90 2023-10-11T06:12:55.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.506+0000 7f93e9ffb640 1 -- 172.21.15.84:0/3863396971 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec00f040 con 0x7f93f40c7f90 2023-10-11T06:12:55.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.506+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f41f3b50 con 0x7f93f40c7f90 2023-10-11T06:12:55.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.506+0000 7f93e9ffb640 1 -- 172.21.15.84:0/3863396971 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec004030 con 0x7f93f40c7f90 2023-10-11T06:12:55.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.506+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f4151dd0 con 0x7f93f40c7f90 2023-10-11T06:12:55.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.507+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f41f3d40 con 0x7f93f40c7f90 2023-10-11T06:12:55.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.508+0000 7f93e9ffb640 1 -- 172.21.15.84:0/3863396971 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f93ec004190 con 0x7f93f40c7f90 2023-10-11T06:12:55.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.509+0000 7f93e9ffb640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f93d806a830 0x7f93d806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.509+0000 7f93e9ffb640 1 -- 172.21.15.84:0/3863396971 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f93ec088310 con 0x7f93f40c7f90 2023-10-11T06:12:55.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.509+0000 7f93f94a6640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f93d806a830 0x7f93d806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.510+0000 7f93f94a6640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f93d806a830 0x7f93d806cce0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f93f0034000 tx=0x7f93f0009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.513+0000 7f93e9ffb640 1 -- 172.21.15.84:0/3863396971 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f93ec053470 con 0x7f93f40c7f90 2023-10-11T06:12:55.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.702+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f93f4073ed0 con 0x7f93f40c7f90 2023-10-11T06:12:55.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.703+0000 7f93e9ffb640 1 -- 172.21.15.84:0/3863396971 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) v1 ==== 97+0+11 (secure 0 0 0) 0x7f93ec056a90 con 0x7f93f40c7f90 2023-10-11T06:12:55.706 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:55.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f93d806a830 msgr2=0x7f93d806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f93d806a830 0x7f93d806cce0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f93f0034000 tx=0x7f93f0009030 comp rx=0 tx=0).stop 2023-10-11T06:12:55.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f40c7f90 msgr2=0x7f93f40c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f40c7f90 0x7f93f40c3b10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f93f40b01a0 tx=0x7f93ec007d00 comp rx=0 tx=0).stop 2023-10-11T06:12:55.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 shutdown_connections 2023-10-11T06:12:55.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f93d806a830 0x7f93d806cce0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f40c7f90 0x7f93f40c3b10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93f41f2a60 0x7f93f4151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 --2- 172.21.15.84:0/3863396971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e00039a0 0x7f93f414a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 >> 172.21.15.84:0/3863396971 conn(0x7f93f4069c70 msgr2=0x7f93f40cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:55.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 shutdown_connections 2023-10-11T06:12:55.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.706+0000 7f93fb70a640 1 -- 172.21.15.84:0/3863396971 wait complete. 2023-10-11T06:12:55.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:55.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.928+0000 7f60428ec640 1 Processor -- start 2023-10-11T06:12:55.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.928+0000 7f60428ec640 1 -- start start 2023-10-11T06:12:55.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f60428ec640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f603c15d4f0 0x7f603c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f60428ec640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f603c0c9360 0x7f603c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f60428ec640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c071a90 con 0x7f603c0ca300 2023-10-11T06:12:55.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f60428ec640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c072310 con 0x7f603c15d4f0 2023-10-11T06:12:55.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f60428ec640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c070280 con 0x7f603c0c9360 2023-10-11T06:12:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f603bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f603c15d4f0 0x7f603c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f6033fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f603c0ca300 0x7f603c0c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46160/0 (socket says 172.21.15.84:46160) 2023-10-11T06:12:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f6033fff640 1 -- 172.21.15.84:0/1605612566 learned_addr learned my addr 172.21.15.84:0/1605612566 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:55.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.930+0000 7f6040e89640 1 --2- 172.21.15.84:0/1605612566 >> v2:172.21.15.99:3300/0 conn(0x7f603c0c9360 0x7f603c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.931+0000 7f603bfff640 1 -- 172.21.15.84:0/1605612566 >> v2:172.21.15.99:3300/0 conn(0x7f603c0c9360 msgr2=0x7f603c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.931+0000 7f603bfff640 1 --2- 172.21.15.84:0/1605612566 >> v2:172.21.15.99:3300/0 conn(0x7f603c0c9360 0x7f603c0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.931+0000 7f603bfff640 1 -- 172.21.15.84:0/1605612566 >> v1:172.21.15.84:6789/0 conn(0x7f603c0ca300 legacy=0x7f603c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.931+0000 7f603bfff640 1 -- 172.21.15.84:0/1605612566 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c071cc0 con 0x7f603c15d4f0 2023-10-11T06:12:55.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.931+0000 7f603b7fe640 1 -- 172.21.15.84:0/1605612566 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911366602 0 0) 0x7f603c071a90 con 0x7f603c0ca300 2023-10-11T06:12:55.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.932+0000 7f603bfff640 1 --2- 172.21.15.84:0/1605612566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c15d4f0 0x7f603c0cb540 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f603c0b4740 tx=0x7f602c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c937a60ca411c400 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.932+0000 7f603b7fe640 1 -- 172.21.15.84:0/1605612566 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c031830 con 0x7f603c15d4f0 2023-10-11T06:12:55.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.932+0000 7f603b7fe640 1 -- 172.21.15.84:0/1605612566 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c031990 con 0x7f603c15d4f0 2023-10-11T06:12:55.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.932+0000 7f603b7fe640 1 -- 172.21.15.84:0/1605612566 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c031d20 con 0x7f603c15d4f0 2023-10-11T06:12:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 -- 172.21.15.84:0/1605612566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c15d4f0 msgr2=0x7f603c0cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 --2- 172.21.15.84:0/1605612566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c15d4f0 0x7f603c0cb540 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f603c0b4740 tx=0x7f602c02b8d0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 -- 172.21.15.84:0/1605612566 shutdown_connections 2023-10-11T06:12:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 --2- 172.21.15.84:0/1605612566 >> v2:172.21.15.99:3300/0 conn(0x7f603c0c9360 0x7f603c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 --2- 172.21.15.84:0/1605612566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c15d4f0 0x7f603c0cb540 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 -- 172.21.15.84:0/1605612566 >> 172.21.15.84:0/1605612566 conn(0x7f603c069c70 msgr2=0x7f603c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 -- 172.21.15.84:0/1605612566 shutdown_connections 2023-10-11T06:12:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.933+0000 7f60428ec640 1 -- 172.21.15.84:0/1605612566 wait complete. 2023-10-11T06:12:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.935+0000 7f60428ec640 1 Processor -- start 2023-10-11T06:12:55.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.935+0000 7f60428ec640 1 -- start start 2023-10-11T06:12:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f60428ec640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f603c0c9360 0x7f603c1efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f60428ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c0ca300 0x7f603c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f60428ec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f603c15d4f0 0x7f603c153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f60428ec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0c6b70 con 0x7f603c0ca300 2023-10-11T06:12:55.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f60428ec640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c15aec0 con 0x7f603c15d4f0 2023-10-11T06:12:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f60428ec640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0c6890 con 0x7f603c0c9360 2023-10-11T06:12:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f6033fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c0ca300 0x7f603c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f6033fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c0ca300 0x7f603c0c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40032/0 (socket says 172.21.15.84:40032) 2023-10-11T06:12:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.936+0000 7f6033fff640 1 -- 172.21.15.84:0/3215252754 learned_addr learned my addr 172.21.15.84:0/3215252754 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f603bfff640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f603c0c9360 0x7f603c1efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f6040e89640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f603c15d4f0 0x7f603c153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f6033fff640 1 -- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f603c0c9360 msgr2=0x7f603c1efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f6033fff640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f603c0c9360 0x7f603c1efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f6033fff640 1 -- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f603c15d4f0 msgr2=0x7f603c153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f6033fff640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f603c15d4f0 0x7f603c153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f6033fff640 1 -- 172.21.15.84:0/3215252754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c0093a0 con 0x7f603c0ca300 2023-10-11T06:12:55.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f6033fff640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c0ca300 0x7f603c0c6060 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f602400c440 tx=0x7f602400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f60397fa640 1 -- 172.21.15.84:0/3215252754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60240074e0 con 0x7f603c0ca300 2023-10-11T06:12:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f60397fa640 1 -- 172.21.15.84:0/3215252754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6024007640 con 0x7f603c0ca300 2023-10-11T06:12:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c15a870 con 0x7f603c0ca300 2023-10-11T06:12:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f60397fa640 1 -- 172.21.15.84:0/3215252754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024004a20 con 0x7f603c0ca300 2023-10-11T06:12:55.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.937+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c0c65a0 con 0x7f603c0ca300 2023-10-11T06:12:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.939+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c15a6d0 con 0x7f603c0ca300 2023-10-11T06:12:55.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.939+0000 7f60397fa640 1 -- 172.21.15.84:0/3215252754 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6024004b80 con 0x7f603c0ca300 2023-10-11T06:12:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.941+0000 7f60397fa640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f601c06a830 0x7f601c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.941+0000 7f60397fa640 1 -- 172.21.15.84:0/3215252754 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f6024088400 con 0x7f603c0ca300 2023-10-11T06:12:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.941+0000 7f603bfff640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f601c06a830 0x7f601c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:55.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.942+0000 7f603bfff640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f601c06a830 0x7f601c06cce0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f602c036000 tx=0x7f602c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:55.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:55.946+0000 7f60397fa640 1 -- 172.21.15.84:0/3215252754 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f603c15a6d0 con 0x7f603c0ca300 2023-10-11T06:12:56.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.154+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f603c15a6d0 con 0x7f603c0ca300 2023-10-11T06:12:56.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.154+0000 7f60397fa640 1 -- 172.21.15.84:0/3215252754 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f603c15a6d0 con 0x7f603c0ca300 2023-10-11T06:12:56.157 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:56.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f601c06a830 msgr2=0x7f601c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f601c06a830 0x7f601c06cce0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f602c036000 tx=0x7f602c008030 comp rx=0 tx=0).stop 2023-10-11T06:12:56.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c0ca300 msgr2=0x7f603c0c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c0ca300 0x7f603c0c6060 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f602400c440 tx=0x7f602400c900 comp rx=0 tx=0).stop 2023-10-11T06:12:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 shutdown_connections 2023-10-11T06:12:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f601c06a830 0x7f601c06cce0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f603c15d4f0 0x7f603c153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f603c0ca300 0x7f603c0c6060 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 --2- 172.21.15.84:0/3215252754 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f603c0c9360 0x7f603c1efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.157+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 >> 172.21.15.84:0/3215252754 conn(0x7f603c069c70 msgr2=0x7f603c157900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.158+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 shutdown_connections 2023-10-11T06:12:56.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.158+0000 7f60428ec640 1 -- 172.21.15.84:0/3215252754 wait complete. 2023-10-11T06:12:56.178 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:12:56.178 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:12:56.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:12:56.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.433+0000 7fd757af7640 1 Processor -- start 2023-10-11T06:12:56.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.433+0000 7fd757af7640 1 -- start start 2023-10-11T06:12:56.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.435+0000 7fd757af7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd7500af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.435+0000 7fd757af7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7500d0c20 0x7fd7500b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.435+0000 7fd757af7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7500d2a30 con 0x7fd7500d25e0 2023-10-11T06:12:56.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.435+0000 7fd757af7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75006cd10 con 0x7fd7500d0c20 2023-10-11T06:12:56.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.435+0000 7fd757af7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7500afb60 con 0x7fd75015c290 2023-10-11T06:12:56.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.436+0000 7fd756094640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7500d0c20 0x7fd7500b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.436+0000 7fd755893640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd7500af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.436+0000 7fd756094640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7500d0c20 0x7fd7500b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:54470/0 (socket says 172.21.15.84:54470) 2023-10-11T06:12:56.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.436+0000 7fd755893640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd7500af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:49838/0 (socket says 172.21.15.84:49838) 2023-10-11T06:12:56.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.436+0000 7fd756094640 1 -- 172.21.15.84:0/533623952 learned_addr learned my addr 172.21.15.84:0/533623952 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:56.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.437+0000 7fd756094640 1 -- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 msgr2=0x7fd7500af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:56.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.437+0000 7fd756094640 1 --2- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd7500af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.437+0000 7fd756094640 1 -- 172.21.15.84:0/533623952 >> v1:172.21.15.99:6789/0 conn(0x7fd7500d25e0 legacy=0x7fd7500d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.437+0000 7fd756094640 1 -- 172.21.15.84:0/533623952 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7500afe40 con 0x7fd7500d0c20 2023-10-11T06:12:56.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.437+0000 7fd755893640 1 --2- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd7500af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:12:56.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.438+0000 7fd756094640 1 --2- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d0c20 0x7fd7500b0c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd75006ceb0 tx=0x7fd74802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e87ad4fe0e8bc27 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:56.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.438+0000 7fd754891640 1 -- 172.21.15.84:0/533623952 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748031640 con 0x7fd7500d0c20 2023-10-11T06:12:56.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.438+0000 7fd754891640 1 -- 172.21.15.84:0/533623952 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7480317a0 con 0x7fd7500d0c20 2023-10-11T06:12:56.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.438+0000 7fd754891640 1 -- 172.21.15.84:0/533623952 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748031b30 con 0x7fd7500d0c20 2023-10-11T06:12:56.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 -- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d0c20 msgr2=0x7fd7500b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 --2- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d0c20 0x7fd7500b0c00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd75006ceb0 tx=0x7fd74802b8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 -- 172.21.15.84:0/533623952 shutdown_connections 2023-10-11T06:12:56.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 --2- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d0c20 0x7fd7500b0c00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 --2- 172.21.15.84:0/533623952 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd7500af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 -- 172.21.15.84:0/533623952 >> 172.21.15.84:0/533623952 conn(0x7fd750071cc0 msgr2=0x7fd75009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:56.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 -- 172.21.15.84:0/533623952 shutdown_connections 2023-10-11T06:12:56.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.439+0000 7fd757af7640 1 -- 172.21.15.84:0/533623952 wait complete. 2023-10-11T06:12:56.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.441+0000 7fd757af7640 1 Processor -- start 2023-10-11T06:12:56.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.441+0000 7fd757af7640 1 -- start start 2023-10-11T06:12:56.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.442+0000 7fd757af7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7500d0c20 0x7fd750158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.442+0000 7fd757af7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d25e0 0x7fd75009fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.442+0000 7fd757af7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd75009d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.442+0000 7fd757af7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75009c680 con 0x7fd7500d0c20 2023-10-11T06:12:56.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.442+0000 7fd757af7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75009c960 con 0x7fd7500d25e0 2023-10-11T06:12:56.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.442+0000 7fd757af7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75009cc40 con 0x7fd75015c290 2023-10-11T06:12:56.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.443+0000 7fd755893640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7500d0c20 0x7fd750158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.443+0000 7fd755893640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7500d0c20 0x7fd750158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40036/0 (socket says 172.21.15.84:40036) 2023-10-11T06:12:56.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.443+0000 7fd755893640 1 -- 172.21.15.84:0/3014895971 learned_addr learned my addr 172.21.15.84:0/3014895971 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:56.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.443+0000 7fd755092640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d25e0 0x7fd75009fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.443+0000 7fd756094640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd75009d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd755893640 1 -- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 msgr2=0x7fd75009d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd755893640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd75009d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd755893640 1 -- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d25e0 msgr2=0x7fd75009fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd755893640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d25e0 0x7fd75009fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd755893640 1 -- 172.21.15.84:0/3014895971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd740009a70 con 0x7fd7500d0c20 2023-10-11T06:12:56.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd755893640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7500d0c20 0x7fd750158c80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd74000efc0 tx=0x7fd74000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:56.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd74e7fc640 1 -- 172.21.15.84:0/3014895971 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740014680 con 0x7fd7500d0c20 2023-10-11T06:12:56.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd74e7fc640 1 -- 172.21.15.84:0/3014895971 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74000f040 con 0x7fd7500d0c20 2023-10-11T06:12:56.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd74e7fc640 1 -- 172.21.15.84:0/3014895971 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740014a00 con 0x7fd7500d0c20 2023-10-11T06:12:56.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.444+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7480094d0 con 0x7fd7500d0c20 2023-10-11T06:12:56.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.445+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75009d200 con 0x7fd7500d0c20 2023-10-11T06:12:56.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.446+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7500b03e0 con 0x7fd7500d0c20 2023-10-11T06:12:56.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.447+0000 7fd74e7fc640 1 -- 172.21.15.84:0/3014895971 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd740014b60 con 0x7fd7500d0c20 2023-10-11T06:12:56.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.448+0000 7fd74e7fc640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd73c06a830 0x7fd73c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.448+0000 7fd74e7fc640 1 -- 172.21.15.84:0/3014895971 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fd74008a0a0 con 0x7fd7500d0c20 2023-10-11T06:12:56.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.448+0000 7fd755092640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd73c06a830 0x7fd73c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.449+0000 7fd755092640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd73c06a830 0x7fd73c06cce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cf850 tx=0x7fd744009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:56.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.452+0000 7fd74e7fc640 1 -- 172.21.15.84:0/3014895971 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd7500b03e0 con 0x7fd7500d0c20 2023-10-11T06:12:56.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.642+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7500b03e0 con 0x7fd7500d0c20 2023-10-11T06:12:56.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.646+0000 7fd74e7fc640 1 -- 172.21.15.84:0/3014895971 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7fd7500b03e0 con 0x7fd7500d0c20 2023-10-11T06:12:56.648 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:12:56.649 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:12:56.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd73c06a830 msgr2=0x7fd73c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd73c06a830 0x7fd73c06cce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7500cf850 tx=0x7fd744009040 comp rx=0 tx=0).stop 2023-10-11T06:12:56.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7500d0c20 msgr2=0x7fd750158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7500d0c20 0x7fd750158c80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd74000efc0 tx=0x7fd74000c490 comp rx=0 tx=0).stop 2023-10-11T06:12:56.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 shutdown_connections 2023-10-11T06:12:56.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd73c06a830 0x7fd73c06cce0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd75015c290 0x7fd75009d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7500d25e0 0x7fd75009fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 --2- 172.21.15.84:0/3014895971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7500d0c20 0x7fd750158c80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 >> 172.21.15.84:0/3014895971 conn(0x7fd750071cc0 msgr2=0x7fd7500b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:56.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.648+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 shutdown_connections 2023-10-11T06:12:56.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.649+0000 7fd757af7640 1 -- 172.21.15.84:0/3014895971 wait complete. 2023-10-11T06:12:56.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:12:38.260176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:12:39.011036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-11T06:12:50.853138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:12:56.669 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:12:56.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:12:56.723 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:12:56.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.872+0000 7f8ae31f0640 1 Processor -- start 2023-10-11T06:12:56.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.873+0000 7f8ae31f0640 1 -- start start 2023-10-11T06:12:56.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.874+0000 7f8ae31f0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8adc0c9410 0x7f8adc16b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8ae31f0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0cc9e0 con 0x7f8adc06dd00 2023-10-11T06:12:56.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8ae31f0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc06d180 con 0x7f8adc160830 2023-10-11T06:12:56.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8ae31f0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc06d720 con 0x7f8adc0c9410 2023-10-11T06:12:56.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8adbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8adc06dd00 0x7f8adc0c8d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46168/0 (socket says 172.21.15.84:46168) 2023-10-11T06:12:56.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8adbfff640 1 -- 172.21.15.84:0/231118850 learned_addr learned my addr 172.21.15.84:0/231118850 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:56.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8ae178d640 1 --2- 172.21.15.84:0/231118850 >> v2:172.21.15.99:3300/0 conn(0x7f8adc0c9410 0x7f8adc16b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329278462 0 0) 0x7f8adc0cc9e0 con 0x7f8adc06dd00 2023-10-11T06:12:56.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.875+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8acc003520 con 0x7f8adc06dd00 2023-10-11T06:12:56.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013057429 0 0) 0x7f8adc06d180 con 0x7f8adc160830 2023-10-11T06:12:56.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8adc0cc9e0 con 0x7f8adc160830 2023-10-11T06:12:56.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1447044320 0 0) 0x7f8acc003520 con 0x7f8adc06dd00 2023-10-11T06:12:56.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 >> v2:172.21.15.99:3300/0 conn(0x7f8adc0c9410 msgr2=0x7f8adc16b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:56.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 --2- 172.21.15.84:0/231118850 >> v2:172.21.15.99:3300/0 conn(0x7f8adc0c9410 0x7f8adc16b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 >> v1:172.21.15.99:6789/0 conn(0x7f8adc160830 legacy=0x7f8adc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc06d440 con 0x7f8adc06dd00 2023-10-11T06:12:56.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f8ac8001670 con 0x7f8adc06dd00 2023-10-11T06:12:56.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 >> v1:172.21.15.84:6789/0 conn(0x7f8adc06dd00 legacy=0x7f8adc0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8acc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8acc0047c0 0x7f8acc004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8acc0051c0 0x7f8acc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc06d720 con 0x7f8acc003b50 2023-10-11T06:12:56.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc00c0d0 con 0x7f8acc0047c0 2023-10-11T06:12:56.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc00c290 con 0x7f8acc0051c0 2023-10-11T06:12:56.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8ae0f8c640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8acc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8ae178d640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8acc0051c0 0x7f8acc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.876+0000 7f8adbfff640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8acc0047c0 0x7f8acc004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.877+0000 7f8ae0f8c640 1 -- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8acc0051c0 msgr2=0x7f8acc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:56.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.877+0000 7f8ae0f8c640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8acc0051c0 0x7f8acc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.877+0000 7f8ae0f8c640 1 -- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8acc0047c0 msgr2=0x7f8acc004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:56.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.877+0000 7f8ae0f8c640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8acc0047c0 0x7f8acc004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.877+0000 7f8ae0f8c640 1 -- 172.21.15.84:0/231118850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc003520 con 0x7f8acc003b50 2023-10-11T06:12:56.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.877+0000 7f8ae178d640 1 -- 172.21.15.84:0/231118850 reap_dead start 2023-10-11T06:12:56.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.878+0000 7f8ae0f8c640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8acc003f40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00018a0 tx=0x7f8acc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9092b68ade01258 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:56.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.878+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad002f050 con 0x7f8acc003b50 2023-10-11T06:12:56.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.878+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00052d0 con 0x7f8acc003b50 2023-10-11T06:12:56.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.878+0000 7f8adb7fe640 1 -- 172.21.15.84:0/231118850 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0003810 con 0x7f8acc003b50 2023-10-11T06:12:56.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.878+0000 7f8ae31f0640 1 -- 172.21.15.84:0/231118850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 msgr2=0x7f8acc003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.878+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8acc003f40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00018a0 tx=0x7f8acc0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.879+0000 7f8ae31f0640 1 -- 172.21.15.84:0/231118850 shutdown_connections 2023-10-11T06:12:56.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.879+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/231118850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8acc003f40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.879+0000 7f8ae31f0640 1 -- 172.21.15.84:0/231118850 >> 172.21.15.84:0/231118850 conn(0x7f8adc069c70 msgr2=0x7f8adc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:56.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.879+0000 7f8ae31f0640 1 -- 172.21.15.84:0/231118850 shutdown_connections 2023-10-11T06:12:56.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.879+0000 7f8ae31f0640 1 -- 172.21.15.84:0/231118850 wait complete. 2023-10-11T06:12:56.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.880+0000 7f8ae31f0640 1 Processor -- start 2023-10-11T06:12:56.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.880+0000 7f8ae31f0640 1 -- start start 2023-10-11T06:12:56.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.881+0000 7f8ae31f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8adc0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.881+0000 7f8ae31f0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8adc161c20 0x7f8adc153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.881+0000 7f8ae31f0640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8adc162710 0x7f8adc0c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.881+0000 7f8ae31f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc165410 con 0x7f8acc003b50 2023-10-11T06:12:56.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.881+0000 7f8ae31f0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0c5d90 con 0x7f8adc161c20 2023-10-11T06:12:56.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.881+0000 7f8ae31f0640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc154110 con 0x7f8adc162710 2023-10-11T06:12:56.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8adc0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8adc0c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40056/0 (socket says 172.21.15.84:40056) 2023-10-11T06:12:56.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 -- 172.21.15.84:0/1210197959 learned_addr learned my addr 172.21.15.84:0/1210197959 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:56.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 -- 172.21.15.84:0/1210197959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8adc162710 msgr2=0x7f8adc0c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8adc162710 0x7f8adc0c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 -- 172.21.15.84:0/1210197959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8adc161c20 msgr2=0x7f8adc153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8adc161c20 0x7f8adc153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 -- 172.21.15.84:0/1210197959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0b15c0 con 0x7f8acc003b50 2023-10-11T06:12:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.882+0000 7f8ae0f8c640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8adc0c5850 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005040 tx=0x7f8ad00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:56.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.883+0000 7f8ad97fa640 1 -- 172.21.15.84:0/1210197959 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0007040 con 0x7f8acc003b50 2023-10-11T06:12:56.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.883+0000 7f8ad97fa640 1 -- 172.21.15.84:0/1210197959 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00071a0 con 0x7f8acc003b50 2023-10-11T06:12:56.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.883+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc0b18a0 con 0x7f8acc003b50 2023-10-11T06:12:56.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.883+0000 7f8ad97fa640 1 -- 172.21.15.84:0/1210197959 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0007510 con 0x7f8acc003b50 2023-10-11T06:12:56.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.883+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc0b1e60 con 0x7f8acc003b50 2023-10-11T06:12:56.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.884+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc073ed0 con 0x7f8acc003b50 2023-10-11T06:12:56.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.885+0000 7f8ad97fa640 1 -- 172.21.15.84:0/1210197959 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8ad00035f0 con 0x7f8acc003b50 2023-10-11T06:12:56.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.885+0000 7f8ad97fa640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ac006a830 0x7f8ac006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:56.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.886+0000 7f8ad97fa640 1 -- 172.21.15.84:0/1210197959 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f8ad00aaac0 con 0x7f8acc003b50 2023-10-11T06:12:56.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.886+0000 7f8adbfff640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ac006a830 0x7f8ac006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:56.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.886+0000 7f8adbfff640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ac006a830 0x7f8ac006cce0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80010e0 tx=0x7f8ac8000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:56.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:56.890+0000 7f8ad97fa640 1 -- 172.21.15.84:0/1210197959 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8ad0075a60 con 0x7f8acc003b50 2023-10-11T06:12:57.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.076+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8adc15ff00 con 0x7f8acc003b50 2023-10-11T06:12:57.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.077+0000 7f8ad97fa640 1 -- 172.21.15.84:0/1210197959 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ad0041020 con 0x7f8acc003b50 2023-10-11T06:12:57.079 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:12:57.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.079+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ac006a830 msgr2=0x7f8ac006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.079+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ac006a830 0x7f8ac006cce0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80010e0 tx=0x7f8ac8000ed0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.079+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 msgr2=0x7f8adc0c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.079+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8adc0c5850 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005040 tx=0x7f8ad00012d0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.079+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 shutdown_connections 2023-10-11T06:12:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.079+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ac006a830 0x7f8ac006cce0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.079+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8adc162710 0x7f8adc0c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.080+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8adc161c20 0x7f8adc153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.080+0000 7f8ae31f0640 1 --2- 172.21.15.84:0/1210197959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8acc003b50 0x7f8adc0c5850 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.080+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 >> 172.21.15.84:0/1210197959 conn(0x7f8adc069c70 msgr2=0x7f8adc0c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.080+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 shutdown_connections 2023-10-11T06:12:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.080+0000 7f8ae31f0640 1 -- 172.21.15.84:0/1210197959 wait complete. 2023-10-11T06:12:57.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:12:57.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.362+0000 7f7764995640 1 Processor -- start 2023-10-11T06:12:57.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.362+0000 7f7764995640 1 -- start start 2023-10-11T06:12:57.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f7764995640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f776006c880 0x7f776006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f7764995640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7760156be0 0x7f776015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f7764995640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77600cde80 con 0x7f7760156810 2023-10-11T06:12:57.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f7764995640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77600ce140 con 0x7f7760156be0 2023-10-11T06:12:57.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f7764995640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776015da30 con 0x7f776006c880 2023-10-11T06:12:57.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f775ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7760156be0 0x7f776015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f775ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7760156be0 0x7f776015d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40072/0 (socket says 172.21.15.84:40072) 2023-10-11T06:12:57.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.364+0000 7f775ed9d640 1 -- 172.21.15.84:0/2948180302 learned_addr learned my addr 172.21.15.84:0/2948180302 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:57.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.365+0000 7f775ed9d640 1 -- 172.21.15.84:0/2948180302 >> v2:172.21.15.99:3300/0 conn(0x7f776006c880 msgr2=0x7f776006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:57.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.365+0000 7f775dd9b640 1 --2- 172.21.15.84:0/2948180302 >> v2:172.21.15.99:3300/0 conn(0x7f776006c880 0x7f776006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.365+0000 7f775ed9d640 1 --2- 172.21.15.84:0/2948180302 >> v2:172.21.15.99:3300/0 conn(0x7f776006c880 0x7f776006d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.365+0000 7f775ed9d640 1 -- 172.21.15.84:0/2948180302 >> v1:172.21.15.99:6789/0 conn(0x7f7760156810 legacy=0x7f776006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:57.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.365+0000 7f775ed9d640 1 -- 172.21.15.84:0/2948180302 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776006df80 con 0x7f7760156be0 2023-10-11T06:12:57.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.365+0000 7f775dd9b640 1 --2- 172.21.15.84:0/2948180302 >> v2:172.21.15.99:3300/0 conn(0x7f776006c880 0x7f776006d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:57.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.366+0000 7f775ed9d640 1 --2- 172.21.15.84:0/2948180302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7760156be0 0x7f776015d4f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f77600ce0b0 tx=0x7f775802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a398f07a0d46a644 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:57.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.366+0000 7f775d59a640 1 -- 172.21.15.84:0/2948180302 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775802c9f0 con 0x7f7760156be0 2023-10-11T06:12:57.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.366+0000 7f775d59a640 1 -- 172.21.15.84:0/2948180302 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775802cb50 con 0x7f7760156be0 2023-10-11T06:12:57.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.366+0000 7f775d59a640 1 -- 172.21.15.84:0/2948180302 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77580343f0 con 0x7f7760156be0 2023-10-11T06:12:57.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 -- 172.21.15.84:0/2948180302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7760156be0 msgr2=0x7f776015d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 --2- 172.21.15.84:0/2948180302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7760156be0 0x7f776015d4f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f77600ce0b0 tx=0x7f775802be50 comp rx=0 tx=0).stop 2023-10-11T06:12:57.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 -- 172.21.15.84:0/2948180302 shutdown_connections 2023-10-11T06:12:57.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 --2- 172.21.15.84:0/2948180302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7760156be0 0x7f776015d4f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 --2- 172.21.15.84:0/2948180302 >> v2:172.21.15.99:3300/0 conn(0x7f776006c880 0x7f776006d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 -- 172.21.15.84:0/2948180302 >> 172.21.15.84:0/2948180302 conn(0x7f7760069c70 msgr2=0x7f776006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:57.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 -- 172.21.15.84:0/2948180302 shutdown_connections 2023-10-11T06:12:57.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.367+0000 7f7764995640 1 -- 172.21.15.84:0/2948180302 wait complete. 2023-10-11T06:12:57.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.369+0000 7f7764995640 1 Processor -- start 2023-10-11T06:12:57.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.369+0000 7f7764995640 1 -- start start 2023-10-11T06:12:57.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.370+0000 7f7764995640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f776006c880 0x7f77601f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.370+0000 7f7764995640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7760156810 0x7f77600af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.370+0000 7f7764995640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7760156be0 0x7f776015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.370+0000 7f7764995640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77601f2a10 con 0x7f776006c880 2023-10-11T06:12:57.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.370+0000 7f7764995640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77600adfd0 con 0x7f7760156be0 2023-10-11T06:12:57.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.370+0000 7f7764995640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77600ae2b0 con 0x7f7760156810 2023-10-11T06:12:57.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.370+0000 7f775e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f776006c880 0x7f77601f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f776006c880 0x7f77601f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40082/0 (socket says 172.21.15.84:40082) 2023-10-11T06:12:57.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 -- 172.21.15.84:0/1010497835 learned_addr learned my addr 172.21.15.84:0/1010497835 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:57.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775dd9b640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7760156810 0x7f77600af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775ed9d640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7760156be0 0x7f776015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 -- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7760156810 msgr2=0x7f77600af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7760156810 0x7f77600af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 -- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7760156be0 msgr2=0x7f776015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7760156be0 0x7f776015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 -- 172.21.15.84:0/1010497835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758009b30 con 0x7f776006c880 2023-10-11T06:12:57.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f775e59c640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f776006c880 0x7f77601f24d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f775000c430 tx=0x7f775000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:57.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.371+0000 7f774effd640 1 -- 172.21.15.84:0/1010497835 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750011790 con 0x7f776006c880 2023-10-11T06:12:57.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.372+0000 7f774effd640 1 -- 172.21.15.84:0/1010497835 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77500118f0 con 0x7f776006c880 2023-10-11T06:12:57.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.372+0000 7f774effd640 1 -- 172.21.15.84:0/1010497835 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750011ce0 con 0x7f776006c880 2023-10-11T06:12:57.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.372+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77600ae590 con 0x7f776006c880 2023-10-11T06:12:57.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.372+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77600aeb50 con 0x7f776006c880 2023-10-11T06:12:57.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.373+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7760073ed0 con 0x7f776006c880 2023-10-11T06:12:57.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.374+0000 7f774effd640 1 -- 172.21.15.84:0/1010497835 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f775001c460 con 0x7f776006c880 2023-10-11T06:12:57.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.375+0000 7f774effd640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f774006a830 0x7f774006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.375+0000 7f774effd640 1 -- 172.21.15.84:0/1010497835 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f7750089100 con 0x7f776006c880 2023-10-11T06:12:57.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.375+0000 7f775dd9b640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f774006a830 0x7f774006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.376+0000 7f775dd9b640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f774006a830 0x7f774006cce0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f77600c5120 tx=0x7f7754008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:57.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.379+0000 7f774effd640 1 -- 172.21.15.84:0/1010497835 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f77500541c0 con 0x7f776006c880 2023-10-11T06:12:57.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.566+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f77600708f0 con 0x7f776006c880 2023-10-11T06:12:57.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.567+0000 7f774effd640 1 -- 172.21.15.84:0/1010497835 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f77500577e0 con 0x7f776006c880 2023-10-11T06:12:57.569 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:12:57.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.569+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f774006a830 msgr2=0x7f774006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.569+0000 7f7764995640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f774006a830 0x7f774006cce0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f77600c5120 tx=0x7f7754008040 comp rx=0 tx=0).stop 2023-10-11T06:12:57.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.569+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f776006c880 msgr2=0x7f77601f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.569+0000 7f7764995640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f776006c880 0x7f77601f24d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f775000c430 tx=0x7f775000c8f0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 shutdown_connections 2023-10-11T06:12:57.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f774006a830 0x7f774006cce0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7760156be0 0x7f776015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7760156810 0x7f77600af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 --2- 172.21.15.84:0/1010497835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f776006c880 0x7f77601f24d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 >> 172.21.15.84:0/1010497835 conn(0x7f7760069c70 msgr2=0x7f77600b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:57.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 shutdown_connections 2023-10-11T06:12:57.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.570+0000 7f7764995640 1 -- 172.21.15.84:0/1010497835 wait complete. 2023-10-11T06:12:57.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:12:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.798+0000 7fb98bc04640 1 Processor -- start 2023-10-11T06:12:57.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.798+0000 7fb98bc04640 1 -- start start 2023-10-11T06:12:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.800+0000 7fb98bc04640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb98414ff30 0x7fb9840c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.800+0000 7fb98bc04640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb984150300 0x7fb9840cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.800+0000 7fb98bc04640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb9840cc7c0 0x7fb9840ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.800+0000 7fb98bc04640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9840c9b90 con 0x7fb98414ff30 2023-10-11T06:12:57.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.800+0000 7fb98bc04640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98415da10 con 0x7fb9840cc7c0 2023-10-11T06:12:57.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.800+0000 7fb98bc04640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98415ce00 con 0x7fb984150300 2023-10-11T06:12:57.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.800+0000 7fb9899a0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb98414ff30 0x7fb9840c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb9899a0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb98414ff30 0x7fb9840c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40094/0 (socket says 172.21.15.84:40094) 2023-10-11T06:12:57.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb9899a0640 1 -- 172.21.15.84:0/2158367763 learned_addr learned my addr 172.21.15.84:0/2158367763 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:57.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb98919f640 1 --2- 172.21.15.84:0/2158367763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb984150300 0x7fb9840cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb9899a0640 1 -- 172.21.15.84:0/2158367763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb984150300 msgr2=0x7fb9840cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:57.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb9899a0640 1 --2- 172.21.15.84:0/2158367763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb984150300 0x7fb9840cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb9899a0640 1 -- 172.21.15.84:0/2158367763 >> v2:172.21.15.99:3300/0 conn(0x7fb9840cc7c0 msgr2=0x7fb9840ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:57.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb98a1a1640 1 --2- 172.21.15.84:0/2158367763 >> v2:172.21.15.99:3300/0 conn(0x7fb9840cc7c0 0x7fb9840ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb9899a0640 1 --2- 172.21.15.84:0/2158367763 >> v2:172.21.15.99:3300/0 conn(0x7fb9840cc7c0 0x7fb9840ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb9899a0640 1 -- 172.21.15.84:0/2158367763 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98415d0e0 con 0x7fb98414ff30 2023-10-11T06:12:57.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb98919f640 1 --2- 172.21.15.84:0/2158367763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb984150300 0x7fb9840cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:57.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.801+0000 7fb98a1a1640 1 --2- 172.21.15.84:0/2158367763 >> v2:172.21.15.99:3300/0 conn(0x7fb9840cc7c0 0x7fb9840ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:57.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.802+0000 7fb9899a0640 1 --2- 172.21.15.84:0/2158367763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb98414ff30 0x7fb9840c9650 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb974009e50 tx=0x7fb97402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e559e72e4d4d1ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:57.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.802+0000 7fb98899e640 1 -- 172.21.15.84:0/2158367763 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97402c9f0 con 0x7fb98414ff30 2023-10-11T06:12:57.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.802+0000 7fb98899e640 1 -- 172.21.15.84:0/2158367763 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97402cb50 con 0x7fb98414ff30 2023-10-11T06:12:57.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.802+0000 7fb98899e640 1 -- 172.21.15.84:0/2158367763 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740343f0 con 0x7fb98414ff30 2023-10-11T06:12:57.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 -- 172.21.15.84:0/2158367763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb98414ff30 msgr2=0x7fb9840c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 --2- 172.21.15.84:0/2158367763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb98414ff30 0x7fb9840c9650 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb974009e50 tx=0x7fb97402be50 comp rx=0 tx=0).stop 2023-10-11T06:12:57.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 -- 172.21.15.84:0/2158367763 shutdown_connections 2023-10-11T06:12:57.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 --2- 172.21.15.84:0/2158367763 >> v2:172.21.15.99:3300/0 conn(0x7fb9840cc7c0 0x7fb9840ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 --2- 172.21.15.84:0/2158367763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb984150300 0x7fb9840cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 --2- 172.21.15.84:0/2158367763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb98414ff30 0x7fb9840c9650 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 -- 172.21.15.84:0/2158367763 >> 172.21.15.84:0/2158367763 conn(0x7fb984069c70 msgr2=0x7fb98406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:57.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 -- 172.21.15.84:0/2158367763 shutdown_connections 2023-10-11T06:12:57.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.803+0000 7fb98bc04640 1 -- 172.21.15.84:0/2158367763 wait complete. 2023-10-11T06:12:57.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.805+0000 7fb98bc04640 1 Processor -- start 2023-10-11T06:12:57.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.805+0000 7fb98bc04640 1 -- start start 2023-10-11T06:12:57.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.806+0000 7fb98bc04640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb98414ff30 0x7fb98415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.806+0000 7fb98bc04640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb984150300 0x7fb98414adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.806+0000 7fb98bc04640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9840cdd70 0x7fb9840ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.806+0000 7fb98bc04640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840ce710 con 0x7fb9840cdd70 2023-10-11T06:12:57.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.806+0000 7fb98bc04640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98414cd60 con 0x7fb984150300 2023-10-11T06:12:57.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.806+0000 7fb98bc04640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98414cfb0 con 0x7fb98414ff30 2023-10-11T06:12:57.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9840cdd70 0x7fb9840ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9840cdd70 0x7fb9840ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40106/0 (socket says 172.21.15.84:40106) 2023-10-11T06:12:57.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 -- 172.21.15.84:0/1525083089 learned_addr learned my addr 172.21.15.84:0/1525083089 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:57.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb9899a0640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb98414ff30 0x7fb98415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98919f640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb984150300 0x7fb98414adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 -- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb98414ff30 msgr2=0x7fb98415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb98414ff30 0x7fb98415a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 -- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb984150300 msgr2=0x7fb98414adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb984150300 0x7fb98414adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:57.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 -- 172.21.15.84:0/1525083089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb974009b30 con 0x7fb9840cdd70 2023-10-11T06:12:57.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98919f640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb984150300 0x7fb98414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:57.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb98a1a1640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9840cdd70 0x7fb9840ce1d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c590 tx=0x7fb97c00ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:57.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.807+0000 7fb9827fc640 1 -- 172.21.15.84:0/1525083089 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c004220 con 0x7fb9840cdd70 2023-10-11T06:12:57.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.808+0000 7fb9827fc640 1 -- 172.21.15.84:0/1525083089 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c0043a0 con 0x7fb9840cdd70 2023-10-11T06:12:57.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.808+0000 7fb9827fc640 1 -- 172.21.15.84:0/1525083089 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c003850 con 0x7fb9840cdd70 2023-10-11T06:12:57.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.808+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98414d290 con 0x7fb9840cdd70 2023-10-11T06:12:57.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.808+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98414d850 con 0x7fb9840cdd70 2023-10-11T06:12:57.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.809+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98406ab90 con 0x7fb9840cdd70 2023-10-11T06:12:57.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.810+0000 7fb9827fc640 1 -- 172.21.15.84:0/1525083089 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb97c0039b0 con 0x7fb9840cdd70 2023-10-11T06:12:57.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.811+0000 7fb9827fc640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb97006a830 0x7fb97006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:57.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.811+0000 7fb9827fc640 1 -- 172.21.15.84:0/1525083089 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fb97c088df0 con 0x7fb9840cdd70 2023-10-11T06:12:57.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.811+0000 7fb9899a0640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb97006a830 0x7fb97006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:57.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.812+0000 7fb9899a0640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb97006a830 0x7fb97006cce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb974035000 tx=0x7fb974030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:57.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.815+0000 7fb9827fc640 1 -- 172.21.15.84:0/1525083089 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb97c053f50 con 0x7fb9840cdd70 2023-10-11T06:12:57.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.994+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb984073ed0 con 0x7fb9840cdd70 2023-10-11T06:12:57.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.994+0000 7fb9827fc640 1 -- 172.21.15.84:0/1525083089 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb97c01e020 con 0x7fb9840cdd70 2023-10-11T06:12:57.997 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:57.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb97006a830 msgr2=0x7fb97006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:57.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb97006a830 0x7fb97006cce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb974035000 tx=0x7fb974030000 comp rx=0 tx=0).stop 2023-10-11T06:12:58.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9840cdd70 msgr2=0x7fb9840ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9840cdd70 0x7fb9840ce1d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c590 tx=0x7fb97c00ca50 comp rx=0 tx=0).stop 2023-10-11T06:12:58.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 shutdown_connections 2023-10-11T06:12:58.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb97006a830 0x7fb97006cce0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9840cdd70 0x7fb9840ce1d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb984150300 0x7fb98414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 --2- 172.21.15.84:0/1525083089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb98414ff30 0x7fb98415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 >> 172.21.15.84:0/1525083089 conn(0x7fb984069c70 msgr2=0x7fb98406d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:58.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 shutdown_connections 2023-10-11T06:12:58.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:57.997+0000 7fb98bc04640 1 -- 172.21.15.84:0/1525083089 wait complete. 2023-10-11T06:12:58.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:12:58.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.227+0000 7fba76a5c640 1 Processor -- start 2023-10-11T06:12:58.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.227+0000 7fba76a5c640 1 -- start start 2023-10-11T06:12:58.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.229+0000 7fba76a5c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba700d2ac0 0x7fba700d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.229+0000 7fba76a5c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba700cf220 con 0x7fba700d0940 2023-10-11T06:12:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.229+0000 7fba76a5c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba700cf500 con 0x7fba7015f7e0 2023-10-11T06:12:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.229+0000 7fba76a5c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba700d3480 con 0x7fba700d2ac0 2023-10-11T06:12:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.229+0000 7fba74ff9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fba700d0940 0x7fba700d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46170/0 (socket says 172.21.15.84:46170) 2023-10-11T06:12:58.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.229+0000 7fba74ff9640 1 -- 172.21.15.84:0/2926052472 learned_addr learned my addr 172.21.15.84:0/2926052472 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:58.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/2926052472 >> v2:172.21.15.99:3300/0 conn(0x7fba700d2ac0 0x7fba700d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974413203 0 0) 0x7fba700cf220 con 0x7fba700d0940 2023-10-11T06:12:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba5c003520 con 0x7fba700d0940 2023-10-11T06:12:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1541301601 0 0) 0x7fba5c003520 con 0x7fba700d0940 2023-10-11T06:12:58.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 >> v2:172.21.15.99:3300/0 conn(0x7fba700d2ac0 msgr2=0x7fba700d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 --2- 172.21.15.84:0/2926052472 >> v2:172.21.15.99:3300/0 conn(0x7fba700d2ac0 0x7fba700d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 >> v1:172.21.15.99:6789/0 conn(0x7fba7015f7e0 legacy=0x7fba700cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700cf7e0 con 0x7fba700d0940 2023-10-11T06:12:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fba68001670 con 0x7fba700d0940 2023-10-11T06:12:58.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 >> v1:172.21.15.84:6789/0 conn(0x7fba700d0940 legacy=0x7fba700d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c003970 0x7fba5c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0047c0 0x7fba5c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.230+0000 7fba6effd640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c005170 0x7fba5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba700d3480 con 0x7fba5c0047c0 2023-10-11T06:12:58.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c00bff0 con 0x7fba5c005170 2023-10-11T06:12:58.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c00c1b0 con 0x7fba5c003970 2023-10-11T06:12:58.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0047c0 0x7fba5c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba74ff9640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c005170 0x7fba5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6f7fe640 1 -- 172.21.15.84:0/2926052472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c003970 msgr2=0x7fba5c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c003970 0x7fba5c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6f7fe640 1 -- 172.21.15.84:0/2926052472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c005170 msgr2=0x7fba5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba5c005170 0x7fba5c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba74ff9640 1 -- 172.21.15.84:0/2926052472 reap_dead start 2023-10-11T06:12:58.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.231+0000 7fba6f7fe640 1 -- 172.21.15.84:0/2926052472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c003520 con 0x7fba5c0047c0 2023-10-11T06:12:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.232+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0047c0 0x7fba5c004c30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fba58009ac0 tx=0x7fba5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a27ad36dd30dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.232+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5802c6b0 con 0x7fba5c0047c0 2023-10-11T06:12:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.232+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5802c810 con 0x7fba5c0047c0 2023-10-11T06:12:58.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba6effd640 1 -- 172.21.15.84:0/2926052472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5802cba0 con 0x7fba5c0047c0 2023-10-11T06:12:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba76a5c640 1 -- 172.21.15.84:0/2926052472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0047c0 msgr2=0x7fba5c004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba76a5c640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0047c0 0x7fba5c004c30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fba58009ac0 tx=0x7fba5c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba76a5c640 1 -- 172.21.15.84:0/2926052472 shutdown_connections 2023-10-11T06:12:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba76a5c640 1 --2- 172.21.15.84:0/2926052472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba5c0047c0 0x7fba5c004c30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba76a5c640 1 -- 172.21.15.84:0/2926052472 >> 172.21.15.84:0/2926052472 conn(0x7fba700713a0 msgr2=0x7fba700717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba76a5c640 1 -- 172.21.15.84:0/2926052472 shutdown_connections 2023-10-11T06:12:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.233+0000 7fba76a5c640 1 -- 172.21.15.84:0/2926052472 wait complete. 2023-10-11T06:12:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.235+0000 7fba76a5c640 1 Processor -- start 2023-10-11T06:12:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.235+0000 7fba76a5c640 1 -- start start 2023-10-11T06:12:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba76a5c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0047c0 0x7fba7014efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba76a5c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba70148640 0x7fba700c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba76a5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7014af00 0x7fba7014b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba76a5c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7014b8b0 con 0x7fba70148640 2023-10-11T06:12:58.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba76a5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba700c80e0 con 0x7fba7014af00 2023-10-11T06:12:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba76a5c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700c7230 con 0x7fba5c0047c0 2023-10-11T06:12:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba6f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba70148640 0x7fba700c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba6f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba70148640 0x7fba700c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40116/0 (socket says 172.21.15.84:40116) 2023-10-11T06:12:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.236+0000 7fba6f7fe640 1 -- 172.21.15.84:0/3432360642 learned_addr learned my addr 172.21.15.84:0/3432360642 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6f7fe640 1 -- 172.21.15.84:0/3432360642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0047c0 msgr2=0x7fba7014efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0047c0 0x7fba7014efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6f7fe640 1 -- 172.21.15.84:0/3432360642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7014af00 msgr2=0x7fba7014b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba74ff9640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7014af00 0x7fba7014b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7014af00 0x7fba7014b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6f7fe640 1 -- 172.21.15.84:0/3432360642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700c74d0 con 0x7fba70148640 2023-10-11T06:12:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6f7fe640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba70148640 0x7fba700c6cf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fba58037040 tx=0x7fba58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6cff9640 1 -- 172.21.15.84:0/3432360642 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58003810 con 0x7fba70148640 2023-10-11T06:12:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6cff9640 1 -- 172.21.15.84:0/3432360642 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba58003970 con 0x7fba70148640 2023-10-11T06:12:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba6cff9640 1 -- 172.21.15.84:0/3432360642 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58003d00 con 0x7fba70148640 2023-10-11T06:12:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.237+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba700c77b0 con 0x7fba70148640 2023-10-11T06:12:58.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.238+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba700c7d70 con 0x7fba70148640 2023-10-11T06:12:58.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.239+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba70073ed0 con 0x7fba70148640 2023-10-11T06:12:58.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.239+0000 7fba6cff9640 1 -- 172.21.15.84:0/3432360642 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fba58040450 con 0x7fba70148640 2023-10-11T06:12:58.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.241+0000 7fba6cff9640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba5006a860 0x7fba5006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.241+0000 7fba6cff9640 1 -- 172.21.15.84:0/3432360642 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fba580ab4b0 con 0x7fba70148640 2023-10-11T06:12:58.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.241+0000 7fba6ffff640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba5006a860 0x7fba5006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.242+0000 7fba6ffff640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba5006a860 0x7fba5006cd10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba70063890 tx=0x7fba60007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:58.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.245+0000 7fba6cff9640 1 -- 172.21.15.84:0/3432360642 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fba580764e0 con 0x7fba70148640 2023-10-11T06:12:58.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.423+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba700b5030 con 0x7fba70148640 2023-10-11T06:12:58.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.424+0000 7fba6cff9640 1 -- 172.21.15.84:0/3432360642 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba58003e60 con 0x7fba70148640 2023-10-11T06:12:58.427 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:12:58.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba5006a860 msgr2=0x7fba5006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba5006a860 0x7fba5006cd10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba70063890 tx=0x7fba60007d50 comp rx=0 tx=0).stop 2023-10-11T06:12:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba70148640 msgr2=0x7fba700c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba70148640 0x7fba700c6cf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fba58037040 tx=0x7fba58004730 comp rx=0 tx=0).stop 2023-10-11T06:12:58.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 shutdown_connections 2023-10-11T06:12:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fba5006a860 0x7fba5006cd10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba7014af00 0x7fba7014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba70148640 0x7fba700c6cf0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 --2- 172.21.15.84:0/3432360642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba5c0047c0 0x7fba7014efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 >> 172.21.15.84:0/3432360642 conn(0x7fba700713a0 msgr2=0x7fba700cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:58.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 shutdown_connections 2023-10-11T06:12:58.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.427+0000 7fba76a5c640 1 -- 172.21.15.84:0/3432360642 wait complete. 2023-10-11T06:12:58.448 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-11T06:12:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.653+0000 7fc11dab6640 1 Processor -- start 2023-10-11T06:12:58.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.653+0000 7fc11dab6640 1 -- start start 2023-10-11T06:12:58.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.655+0000 7fc11dab6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc11815a0b0 0x7fc1180cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.655+0000 7fc11dab6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1180c8d00 0x7fc1180ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.655+0000 7fc11dab6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1180cefc0 con 0x7fc1180ceb60 2023-10-11T06:12:58.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.655+0000 7fc11dab6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11806e0f0 con 0x7fc11815a0b0 2023-10-11T06:12:58.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.655+0000 7fc11dab6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180cbae0 con 0x7fc1180c8d00 2023-10-11T06:12:58.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.655+0000 7fc117fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc1180ceb60 0x7fc11806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46178/0 (socket says 172.21.15.84:46178) 2023-10-11T06:12:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.655+0000 7fc117fff640 1 -- 172.21.15.84:0/1584434859 learned_addr learned my addr 172.21.15.84:0/1584434859 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc10effd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1180c8d00 0x7fc1180ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44233707 0 0) 0x7fc1180cefc0 con 0x7fc1180ceb60 2023-10-11T06:12:58.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc1177fe640 1 --2- 172.21.15.84:0/1584434859 >> v2:172.21.15.99:3300/0 conn(0x7fc11815a0b0 0x7fc1180cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc104003520 con 0x7fc1180ceb60 2023-10-11T06:12:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 664533766 0 0) 0x7fc104003520 con 0x7fc1180ceb60 2023-10-11T06:12:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1180c8d00 msgr2=0x7fc1180ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1180c8d00 0x7fc1180ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 >> v2:172.21.15.99:3300/0 conn(0x7fc11815a0b0 msgr2=0x7fc1180cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 --2- 172.21.15.84:0/1584434859 >> v2:172.21.15.99:3300/0 conn(0x7fc11815a0b0 0x7fc1180cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc118071380 con 0x7fc1180ceb60 2023-10-11T06:12:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fc110001670 con 0x7fc1180ceb60 2023-10-11T06:12:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 >> v1:172.21.15.84:6789/0 conn(0x7fc1180ceb60 legacy=0x7fc11806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040039a0 0x7fc104003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1040047c0 0x7fc104004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1180c8d00 0x7fc104005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11806e0f0 con 0x7fc1040047c0 2023-10-11T06:12:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180cbae0 con 0x7fc1040039a0 2023-10-11T06:12:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10400bae0 con 0x7fc1180c8d00 2023-10-11T06:12:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.656+0000 7fc1177fe640 1 --2- 172.21.15.84:0/1584434859 >> v2:172.21.15.99:3300/0 conn(0x7fc11815a0b0 0x7fc1180cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:12:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.657+0000 7fc10effd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1040047c0 0x7fc104004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.657+0000 7fc10effd640 1 -- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1180c8d00 msgr2=0x7fc104005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.657+0000 7fc10effd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1180c8d00 0x7fc104005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.657+0000 7fc10effd640 1 -- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040039a0 msgr2=0x7fc104003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.657+0000 7fc10effd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040039a0 0x7fc104003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.657+0000 7fc10effd640 1 -- 172.21.15.84:0/1584434859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104003520 con 0x7fc1040047c0 2023-10-11T06:12:58.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.658+0000 7fc10effd640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1040047c0 0x7fc104004c30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc100009ac0 tx=0x7fc1040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1625a133ab534e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:58.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.658+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10002c710 con 0x7fc1040047c0 2023-10-11T06:12:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.658+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10002c870 con 0x7fc1040047c0 2023-10-11T06:12:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.658+0000 7fc116ffd640 1 -- 172.21.15.84:0/1584434859 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10002cc00 con 0x7fc1040047c0 2023-10-11T06:12:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.659+0000 7fc11dab6640 1 -- 172.21.15.84:0/1584434859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1040047c0 msgr2=0x7fc104004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.659+0000 7fc11dab6640 1 --2- 172.21.15.84:0/1584434859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1040047c0 0x7fc104004c30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc100009ac0 tx=0x7fc1040057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.659+0000 7fc117fff640 1 -- 172.21.15.84:0/1584434859 reap_dead start 2023-10-11T06:12:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.659+0000 7fc11dab6640 1 -- 172.21.15.84:0/1584434859 shutdown_connections 2023-10-11T06:12:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.659+0000 7fc11dab6640 1 -- 172.21.15.84:0/1584434859 >> 172.21.15.84:0/1584434859 conn(0x7fc118069c70 msgr2=0x7fc11806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.659+0000 7fc11dab6640 1 -- 172.21.15.84:0/1584434859 shutdown_connections 2023-10-11T06:12:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.659+0000 7fc11dab6640 1 -- 172.21.15.84:0/1584434859 wait complete. 2023-10-11T06:12:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.661+0000 7fc11dab6640 1 Processor -- start 2023-10-11T06:12:58.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.661+0000 7fc11dab6640 1 -- start start 2023-10-11T06:12:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc11dab6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040047c0 0x7fc11815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc11dab6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc11815d0a0 0x7fc118144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc11dab6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc118158780 0x7fc118158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc11dab6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118159160 con 0x7fc118158780 2023-10-11T06:12:58.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc11dab6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180b1760 con 0x7fc1040047c0 2023-10-11T06:12:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc11dab6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180b1a00 con 0x7fc11815d0a0 2023-10-11T06:12:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc117fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc118158780 0x7fc118158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc117fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc118158780 0x7fc118158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40130/0 (socket says 172.21.15.84:40130) 2023-10-11T06:12:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.662+0000 7fc117fff640 1 -- 172.21.15.84:0/429247226 learned_addr learned my addr 172.21.15.84:0/429247226 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc1177fe640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040047c0 0x7fc11815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc10effd640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc11815d0a0 0x7fc118144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc117fff640 1 -- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc11815d0a0 msgr2=0x7fc118144850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc117fff640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc11815d0a0 0x7fc118144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc117fff640 1 -- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040047c0 msgr2=0x7fc11815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc117fff640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040047c0 0x7fc11815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc117fff640 1 -- 172.21.15.84:0/429247226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180b1ce0 con 0x7fc118158780 2023-10-11T06:12:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc1177fe640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040047c0 0x7fc11815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc117fff640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc118158780 0x7fc118158c20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b01a0 tx=0x7fc11000c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc114ff9640 1 -- 172.21.15.84:0/429247226 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110007500 con 0x7fc118158780 2023-10-11T06:12:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc114ff9640 1 -- 172.21.15.84:0/429247226 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc110007660 con 0x7fc118158780 2023-10-11T06:12:58.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1180b1fc0 con 0x7fc118158780 2023-10-11T06:12:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc114ff9640 1 -- 172.21.15.84:0/429247226 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110004a20 con 0x7fc118158780 2023-10-11T06:12:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.663+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11815a9c0 con 0x7fc118158780 2023-10-11T06:12:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.665+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1180b24e0 con 0x7fc118158780 2023-10-11T06:12:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.665+0000 7fc114ff9640 1 -- 172.21.15.84:0/429247226 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc110004b80 con 0x7fc118158780 2023-10-11T06:12:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.666+0000 7fc114ff9640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0f806a830 0x7fc0f806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.666+0000 7fc114ff9640 1 -- 172.21.15.84:0/429247226 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fc110088250 con 0x7fc118158780 2023-10-11T06:12:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.667+0000 7fc1177fe640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0f806a830 0x7fc0f806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.667+0000 7fc1177fe640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0f806a830 0x7fc0f806cce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc1080094c0 tx=0x7fc10800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:58.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.675+0000 7fc114ff9640 1 -- 172.21.15.84:0/429247226 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc1180b24e0 con 0x7fc118158780 2023-10-11T06:12:58.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.865+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fc1180b24e0 con 0x7fc118158780 2023-10-11T06:12:58.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.866+0000 7fc114ff9640 1 -- 172.21.15.84:0/429247226 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) v1 ==== 97+0+11 (secure 0 0 0) 0x7fc1180b24e0 con 0x7fc118158780 2023-10-11T06:12:58.868 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:12:58.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.868+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0f806a830 msgr2=0x7fc0f806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.868+0000 7fc11dab6640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0f806a830 0x7fc0f806cce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc1080094c0 tx=0x7fc10800b040 comp rx=0 tx=0).stop 2023-10-11T06:12:58.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.868+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc118158780 msgr2=0x7fc118158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:58.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.868+0000 7fc11dab6640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc118158780 0x7fc118158c20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b01a0 tx=0x7fc11000c440 comp rx=0 tx=0).stop 2023-10-11T06:12:58.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 shutdown_connections 2023-10-11T06:12:58.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc0f806a830 0x7fc0f806cce0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc118158780 0x7fc118158c20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc11815d0a0 0x7fc118144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 --2- 172.21.15.84:0/429247226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1040047c0 0x7fc11815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:58.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 >> 172.21.15.84:0/429247226 conn(0x7fc118069c70 msgr2=0x7fc1180c35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:58.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 shutdown_connections 2023-10-11T06:12:58.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:58.869+0000 7fc11dab6640 1 -- 172.21.15.84:0/429247226 wait complete. 2023-10-11T06:12:58.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:12:59.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.103+0000 7f74edf27640 1 Processor -- start 2023-10-11T06:12:59.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.103+0000 7f74edf27640 1 -- start start 2023-10-11T06:12:59.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.105+0000 7f74edf27640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f74e80cafb0 0x7f74e80cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.105+0000 7f74edf27640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e80cb970 con 0x7f74e80cc410 2023-10-11T06:12:59.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.105+0000 7f74edf27640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e815fe40 con 0x7f74e815c290 2023-10-11T06:12:59.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.105+0000 7f74edf27640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e80cad70 con 0x7f74e80cafb0 2023-10-11T06:12:59.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.105+0000 7f74e7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f74e80cc410 0x7f74e80cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:38798/0 (socket says 172.21.15.84:38798) 2023-10-11T06:12:59.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.105+0000 7f74e7fff640 1 -- 172.21.15.84:0/726498052 learned_addr learned my addr 172.21.15.84:0/726498052 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:12:59.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74dffff640 1 --2- 172.21.15.84:0/726498052 >> v2:172.21.15.99:3300/0 conn(0x7f74e80cafb0 0x7f74e80cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258967801 0 0) 0x7f74e80cb970 con 0x7f74e80cc410 2023-10-11T06:12:59.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74d4003520 con 0x7f74e80cc410 2023-10-11T06:12:59.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018803264 0 0) 0x7f74e815fe40 con 0x7f74e815c290 2023-10-11T06:12:59.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e80cb970 con 0x7f74e815c290 2023-10-11T06:12:59.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3626246558 0 0) 0x7f74d4003520 con 0x7f74e80cc410 2023-10-11T06:12:59.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 >> v2:172.21.15.99:3300/0 conn(0x7f74e80cafb0 msgr2=0x7f74e80cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:59.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 --2- 172.21.15.84:0/726498052 >> v2:172.21.15.99:3300/0 conn(0x7f74e80cafb0 0x7f74e80cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 >> v1:172.21.15.99:6789/0 conn(0x7f74e815c290 legacy=0x7f74e80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:59.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e806ff30 con 0x7f74e80cc410 2023-10-11T06:12:59.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f74e00019f0 con 0x7f74e80cc410 2023-10-11T06:12:59.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 >> v1:172.21.15.84:6789/0 conn(0x7f74e80cc410 legacy=0x7f74e80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:59.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d4003b00 0x7f74d4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d40047c0 0x7f74d4004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d4005170 0x7f74d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80cad70 con 0x7f74d4003b00 2023-10-11T06:12:59.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d400bfd0 con 0x7f74d40047c0 2023-10-11T06:12:59.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.106+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d400c190 con 0x7f74d4005170 2023-10-11T06:12:59.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e77fe640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d4003b00 0x7f74d4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74dffff640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d40047c0 0x7f74d4004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e7fff640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d4005170 0x7f74d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e77fe640 1 -- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d4005170 msgr2=0x7f74d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:59.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e77fe640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d4005170 0x7f74d400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e77fe640 1 -- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d40047c0 msgr2=0x7f74d4004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:12:59.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e77fe640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d40047c0 0x7f74d4004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e77fe640 1 -- 172.21.15.84:0/726498052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4003520 con 0x7f74d4003b00 2023-10-11T06:12:59.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.107+0000 7f74e7fff640 1 -- 172.21.15.84:0/726498052 reap_dead start 2023-10-11T06:12:59.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.108+0000 7f74e77fe640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d4003b00 0x7f74d4003ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74d8001930 tx=0x7f74d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d0fdc80c47fff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:59.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.108+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8036070 con 0x7f74d4003b00 2023-10-11T06:12:59.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.108+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d8006820 con 0x7f74d4003b00 2023-10-11T06:12:59.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.108+0000 7f74e6ffd640 1 -- 172.21.15.84:0/726498052 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8006c30 con 0x7f74d4003b00 2023-10-11T06:12:59.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.109+0000 7f74edf27640 1 -- 172.21.15.84:0/726498052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d4003b00 msgr2=0x7f74d4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:59.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.109+0000 7f74edf27640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d4003b00 0x7f74d4003ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74d8001930 tx=0x7f74d40057f0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.109+0000 7f74edf27640 1 -- 172.21.15.84:0/726498052 shutdown_connections 2023-10-11T06:12:59.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.109+0000 7f74edf27640 1 --2- 172.21.15.84:0/726498052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d4003b00 0x7f74d4003ed0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.109+0000 7f74edf27640 1 -- 172.21.15.84:0/726498052 >> 172.21.15.84:0/726498052 conn(0x7f74e8069c70 msgr2=0x7f74e806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:59.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.109+0000 7f74edf27640 1 -- 172.21.15.84:0/726498052 shutdown_connections 2023-10-11T06:12:59.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.109+0000 7f74edf27640 1 -- 172.21.15.84:0/726498052 wait complete. 2023-10-11T06:12:59.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.111+0000 7f74edf27640 1 Processor -- start 2023-10-11T06:12:59.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.111+0000 7f74edf27640 1 -- start start 2023-10-11T06:12:59.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74edf27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d4003b00 0x7f74e8150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74edf27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e80b1ee0 0x7f74e80ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74edf27640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74e80b2330 0x7f74e80ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74edf27640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8151220 con 0x7f74e80b1ee0 2023-10-11T06:12:59.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74edf27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80b28c0 con 0x7f74d4003b00 2023-10-11T06:12:59.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74edf27640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80b2ba0 con 0x7f74e80b2330 2023-10-11T06:12:59.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e80b1ee0 0x7f74e80ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74dffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e80b1ee0 0x7f74e80ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47706/0 (socket says 172.21.15.84:47706) 2023-10-11T06:12:59.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.112+0000 7f74dffff640 1 -- 172.21.15.84:0/2908674962 learned_addr learned my addr 172.21.15.84:0/2908674962 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:12:59.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74e7fff640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74e80b2330 0x7f74e80ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74dffff640 1 -- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74e80b2330 msgr2=0x7f74e80ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:59.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74dffff640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74e80b2330 0x7f74e80ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74e77fe640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d4003b00 0x7f74e8150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74dffff640 1 -- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d4003b00 msgr2=0x7f74e8150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:59.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74dffff640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d4003b00 0x7f74e8150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74dffff640 1 -- 172.21.15.84:0/2908674962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80b2e80 con 0x7f74e80b1ee0 2023-10-11T06:12:59.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74e77fe640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d4003b00 0x7f74e8150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:12:59.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74dffff640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e80b1ee0 0x7f74e80ad1a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f74d00114e0 tx=0x7f74d00115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:59.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74e4ff9640 1 -- 172.21.15.84:0/2908674962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0015660 con 0x7f74e80b1ee0 2023-10-11T06:12:59.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74e4ff9640 1 -- 172.21.15.84:0/2908674962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d00157c0 con 0x7f74e80b1ee0 2023-10-11T06:12:59.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.113+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e815dd70 con 0x7f74e80b1ee0 2023-10-11T06:12:59.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.114+0000 7f74e4ff9640 1 -- 172.21.15.84:0/2908674962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0015b30 con 0x7f74e80b1ee0 2023-10-11T06:12:59.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.114+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e80c2040 con 0x7f74e80b1ee0 2023-10-11T06:12:59.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.115+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e815dbd0 con 0x7f74e80b1ee0 2023-10-11T06:12:59.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.116+0000 7f74e4ff9640 1 -- 172.21.15.84:0/2908674962 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f74d0015c90 con 0x7f74e80b1ee0 2023-10-11T06:12:59.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.117+0000 7f74e4ff9640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74c806a830 0x7f74c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:12:59.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.117+0000 7f74e4ff9640 1 -- 172.21.15.84:0/2908674962 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f74d0089610 con 0x7f74e80b1ee0 2023-10-11T06:12:59.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.117+0000 7f74e77fe640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74c806a830 0x7f74c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:12:59.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.118+0000 7f74e77fe640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74c806a830 0x7f74c806cce0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74d8035000 tx=0x7f74d8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:12:59.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.121+0000 7f74e4ff9640 1 -- 172.21.15.84:0/2908674962 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f74e815dbd0 con 0x7f74e80b1ee0 2023-10-11T06:12:59.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.319+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74e815dbd0 con 0x7f74e80b1ee0 2023-10-11T06:12:59.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.320+0000 7f74e4ff9640 1 -- 172.21.15.84:0/2908674962 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f74e815dbd0 con 0x7f74e80b1ee0 2023-10-11T06:12:59.322 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:12:59.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74c806a830 msgr2=0x7f74c806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74c806a830 0x7f74c806cce0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74d8035000 tx=0x7f74d8009030 comp rx=0 tx=0).stop 2023-10-11T06:12:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e80b1ee0 msgr2=0x7f74e80ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:12:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e80b1ee0 0x7f74e80ad1a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f74d00114e0 tx=0x7f74d00115d0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 shutdown_connections 2023-10-11T06:12:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f74c806a830 0x7f74c806cce0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74e80b2330 0x7f74e80ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e80b1ee0 0x7f74e80ad1a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 --2- 172.21.15.84:0/2908674962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d4003b00 0x7f74e8150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:12:59.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 >> 172.21.15.84:0/2908674962 conn(0x7f74e8069c70 msgr2=0x7f74e806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:12:59.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 shutdown_connections 2023-10-11T06:12:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:12:59.322+0000 7f74edf27640 1 -- 172.21.15.84:0/2908674962 wait complete. 2023-10-11T06:12:59.342 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:12:59.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:12:59.344 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T06:12:59.431 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[101943]: fuse finished with error 0 and tester_r 0 2023-10-11T06:12:59.437 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:13:05.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:13:05.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:05.540 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:13:05.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:13:05.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.817+0000 7fa82c39f640 1 Processor -- start 2023-10-11T06:13:05.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.818+0000 7fa82c39f640 1 -- start start 2023-10-11T06:13:05.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.819+0000 7fa82c39f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa82415d4f0 0x7fa8240cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:05.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.819+0000 7fa82c39f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8240c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:05.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.819+0000 7fa82c39f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8240c9190 0x7fa82415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:05.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.819+0000 7fa82c39f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa824072310 con 0x7fa8240c9190 2023-10-11T06:13:05.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.819+0000 7fa82c39f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa824071a90 con 0x7fa82415d4f0 2023-10-11T06:13:05.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.819+0000 7fa82c39f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa824070280 con 0x7fa8240ca300 2023-10-11T06:13:05.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8240c9190 0x7fa82415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:05.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8240c9190 0x7fa82415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47714/0 (socket says 172.21.15.84:47714) 2023-10-11T06:13:05.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 -- 172.21.15.84:0/55945117 learned_addr learned my addr 172.21.15.84:0/55945117 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:05.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82993a640 1 --2- 172.21.15.84:0/55945117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8240c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:05.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 -- 172.21.15.84:0/55945117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 msgr2=0x7fa8240c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:05.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 --2- 172.21.15.84:0/55945117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8240c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 -- 172.21.15.84:0/55945117 >> v2:172.21.15.99:3300/0 conn(0x7fa82415d4f0 msgr2=0x7fa8240cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:05.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 --2- 172.21.15.84:0/55945117 >> v2:172.21.15.99:3300/0 conn(0x7fa82415d4f0 0x7fa8240cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.820+0000 7fa82a93c640 1 -- 172.21.15.84:0/55945117 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824071cc0 con 0x7fa8240c9190 2023-10-11T06:13:05.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.821+0000 7fa82a93c640 1 --2- 172.21.15.84:0/55945117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa82415dbc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa8240cf6a0 tx=0x7fa81c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4458d3a9b5fe3833 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:05.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.822+0000 7fa829139640 1 -- 172.21.15.84:0/55945117 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c02ca70 con 0x7fa8240c9190 2023-10-11T06:13:05.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.822+0000 7fa829139640 1 -- 172.21.15.84:0/55945117 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c02cbd0 con 0x7fa8240c9190 2023-10-11T06:13:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.822+0000 7fa829139640 1 -- 172.21.15.84:0/55945117 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0343f0 con 0x7fa8240c9190 2023-10-11T06:13:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.822+0000 7fa82c39f640 1 -- 172.21.15.84:0/55945117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 msgr2=0x7fa82415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.822+0000 7fa82c39f640 1 --2- 172.21.15.84:0/55945117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa82415dbc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa8240cf6a0 tx=0x7fa81c02bed0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.823+0000 7fa82c39f640 1 -- 172.21.15.84:0/55945117 shutdown_connections 2023-10-11T06:13:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.823+0000 7fa82c39f640 1 --2- 172.21.15.84:0/55945117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa82415dbc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.823+0000 7fa82c39f640 1 --2- 172.21.15.84:0/55945117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8240c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.823+0000 7fa82c39f640 1 --2- 172.21.15.84:0/55945117 >> v2:172.21.15.99:3300/0 conn(0x7fa82415d4f0 0x7fa8240cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.823+0000 7fa82c39f640 1 -- 172.21.15.84:0/55945117 >> 172.21.15.84:0/55945117 conn(0x7fa824069c70 msgr2=0x7fa82406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:05.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.823+0000 7fa82c39f640 1 -- 172.21.15.84:0/55945117 shutdown_connections 2023-10-11T06:13:05.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.823+0000 7fa82c39f640 1 -- 172.21.15.84:0/55945117 wait complete. 2023-10-11T06:13:05.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.824+0000 7fa82c39f640 1 Processor -- start 2023-10-11T06:13:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.825+0000 7fa82c39f640 1 -- start start 2023-10-11T06:13:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.825+0000 7fa82c39f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa824156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.825+0000 7fa82c39f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8241590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.825+0000 7fa82c39f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82415d4f0 0x7fa82415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:05.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.825+0000 7fa82c39f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa824157340 con 0x7fa8240c9190 2023-10-11T06:13:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.825+0000 7fa82c39f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82414e3d0 con 0x7fa82415d4f0 2023-10-11T06:13:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.825+0000 7fa82c39f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa824159a60 con 0x7fa8240ca300 2023-10-11T06:13:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa824156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa824156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47724/0 (socket says 172.21.15.84:47724) 2023-10-11T06:13:05.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 -- 172.21.15.84:0/1288427086 learned_addr learned my addr 172.21.15.84:0/1288427086 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a93c640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82415d4f0 0x7fa82415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82993a640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8241590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 -- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 msgr2=0x7fa8241590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8241590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 -- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82415d4f0 msgr2=0x7fa82415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82415d4f0 0x7fa82415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.826+0000 7fa82a13b640 1 -- 172.21.15.84:0/1288427086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c009b30 con 0x7fa8240c9190 2023-10-11T06:13:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.827+0000 7fa82a13b640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa824156e00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa82000b3f0 tx=0x7fa82000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.827+0000 7fa81affd640 1 -- 172.21.15.84:0/1288427086 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820005970 con 0x7fa8240c9190 2023-10-11T06:13:05.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.827+0000 7fa81affd640 1 -- 172.21.15.84:0/1288427086 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa820005ad0 con 0x7fa8240c9190 2023-10-11T06:13:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.827+0000 7fa81affd640 1 -- 172.21.15.84:0/1288427086 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820005e80 con 0x7fa8240c9190 2023-10-11T06:13:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.827+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa824159d40 con 0x7fa8240c9190 2023-10-11T06:13:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.827+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8240c6930 con 0x7fa8240c9190 2023-10-11T06:13:05.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.828+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa824073ed0 con 0x7fa8240c9190 2023-10-11T06:13:05.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.829+0000 7fa81affd640 1 -- 172.21.15.84:0/1288427086 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa820011630 con 0x7fa8240c9190 2023-10-11T06:13:05.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.830+0000 7fa81affd640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa80806a830 0x7fa80806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:05.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.830+0000 7fa81affd640 1 -- 172.21.15.84:0/1288427086 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fa820088f30 con 0x7fa8240c9190 2023-10-11T06:13:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.830+0000 7fa82993a640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa80806a830 0x7fa80806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.831+0000 7fa82993a640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa80806a830 0x7fa80806cce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa814006fd0 tx=0x7fa814008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:05.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:05.835+0000 7fa81affd640 1 -- 172.21.15.84:0/1288427086 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa820053f60 con 0x7fa8240c9190 2023-10-11T06:13:06.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.064+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa8240b09b0 con 0x7fa8240c9190 2023-10-11T06:13:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.065+0000 7fa81affd640 1 -- 172.21.15.84:0/1288427086 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+82 (secure 0 0 0) 0x7fa820015070 con 0x7fa8240c9190 2023-10-11T06:13:06.068 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-10-11T06:13:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa80806a830 msgr2=0x7fa80806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa80806a830 0x7fa80806cce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa814006fd0 tx=0x7fa814008040 comp rx=0 tx=0).stop 2023-10-11T06:13:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 msgr2=0x7fa824156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa824156e00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa82000b3f0 tx=0x7fa82000b8b0 comp rx=0 tx=0).stop 2023-10-11T06:13:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 shutdown_connections 2023-10-11T06:13:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa80806a830 0x7fa80806cce0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82415d4f0 0x7fa82415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8240ca300 0x7fa8241590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 --2- 172.21.15.84:0/1288427086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8240c9190 0x7fa824156e00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.068+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 >> 172.21.15.84:0/1288427086 conn(0x7fa824069c70 msgr2=0x7fa8240af690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.069+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 shutdown_connections 2023-10-11T06:13:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:06.069+0000 7fa82c39f640 1 -- 172.21.15.84:0/1288427086 wait complete. 2023-10-11T06:13:06.089 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:13:06.089 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:13:06.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:13:06.090 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T06:13:06.091 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:06.091 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:13:06.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:13:06.092 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:13:06.092 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:06.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:06.093 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:13:06.145 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:13:06.145 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:13:06.145 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:13:06.145 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:06.146 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:13:06.146 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:06.146 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:06.146 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:06.146 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:13:06.147 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:06.147 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:06.147 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:13:06.147 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:06.147 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:06.148 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:13:06.148 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:06.148 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:13:06.148 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42291sec preferred_lft 42291sec 2023-10-11T06:13:06.148 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:13:06.149 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:06.149 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:13:06.149 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:06.149 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:13:06.149 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:06.149 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1c33:d5ff:fea1:21f9/64 scope link 2023-10-11T06:13:06.150 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:06.150 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:13:06.150 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:13:06.150 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::d4ff:bcff:feac:c798/64 scope link 2023-10-11T06:13:06.150 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:06.151 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:13:06.151 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:13:06.151 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4088:59ff:feb7:538e/64 scope link 2023-10-11T06:13:06.151 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:06.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:06.152 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:13:06.242 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:13:06.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:06.247 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:13:06.303 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:13:06.303 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:13:06.304 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:13:06.304 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:13:06.304 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:13:06.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:06.305 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:13:06.305 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:13:06.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:13:06.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:13:06.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:13:06.306 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:13:06.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:06.492 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:13:06.543 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T06:13:06.544 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:13:06.544 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:06.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:06.545 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-11T06:13:06.641 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T06:13:06.642 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:13:06.642 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:06.695 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:06.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:13:06.696 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:06.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:06.883 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:06.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:06.971 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:06.974 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:06.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:06.976 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:07.030 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:13:07.031 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T06:13:07.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:07.119 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:07.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:07.191 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:07.193 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:07.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:07.195 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:07.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-10-11T06:13:07.226+0000 7efd8403a480 -1 init, newargv = 0x55c76a49a840 newargc=15 2023-10-11T06:13:07.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[103332]: starting ceph client 2023-10-11T06:13:07.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[103332]: starting fuse 2023-10-11T06:13:08.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:08.442 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:08.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:08.543 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:08.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:08.544 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:08.598 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:13:08.599 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-10-11T06:13:08.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:13:08.600 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:13:08.600 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:13:08.600 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:13:08.600 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:13:08.601 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:08.601 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:13:08.601 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T06:13:08.602 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:13:08.602 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T06:13:08.602 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:08.603 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:13:08.603 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:13:08.603 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:13:08.604 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:13:08.604 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:08.605 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:13:08.605 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:13:08.605 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:13:08.606 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:13:08.606 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:13:08.606 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:13:08.607 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:13:08.607 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:13:08.607 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:08.608 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-10-11T06:13:08.608 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:13:08.792 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.103332.asok 2023-10-11T06:13:08.973 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103332.asok 2023-10-11T06:13:08.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:08.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103332.asok status 2023-10-11T06:13:09.211 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:13:09.211 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:13:09.211 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:13:09.212 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:13:09.212 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-10-11T06:13:09.212 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:13:09.212 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T06:13:09.212 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "103332", 2023-10-11T06:13:09.213 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:13:09.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:09.213 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:13:09.213 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:13:09.213 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5640, 2023-10-11T06:13:09.214 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:13:09.214 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:13:09.214 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:13:09.214 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 5640 2023-10-11T06:13:09.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:09.215 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:13:09.215 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:13:09.215 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:13:09.215 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 17520669 2023-10-11T06:13:09.215 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:09.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:09.216 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:13:09.216 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:13:09.216 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:13:09.216 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 17520669 2023-10-11T06:13:09.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:09.217 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.5640 192.168.0.1:0/17520669", 2023-10-11T06:13:09.217 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/17520669", 2023-10-11T06:13:09.217 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:13:09.217 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 16, 2023-10-11T06:13:09.217 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 41, 2023-10-11T06:13:09.218 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:13:09.218 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:13:09.218 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:13:09.218 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:13:09.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:09.227 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:13:09.227 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:09.282 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:13:09.283 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:09.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:09.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:09.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:09.378 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-10-11T06:13:09.455 INFO:teuthology.orchestra.run.smithi084.stderr:setfattr: layoutfile: Permission denied 2023-10-11T06:13:09.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:13:09.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:09.457 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-10-11T06:13:09.518 INFO:teuthology.orchestra.run.smithi084.stderr:setfattr: layoutdir: Permission denied 2023-10-11T06:13:09.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:13:09.519 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:09.520 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:09.570 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:09.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.779+0000 7f7bfe10a640 1 Processor -- start 2023-10-11T06:13:09.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.780+0000 7f7bfe10a640 1 -- start start 2023-10-11T06:13:09.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bfe10a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7bf815d4f0 0x7f7bf80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:09.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bfe10a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7bf80c9360 0x7f7bf80ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:09.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bfe10a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf8071a90 con 0x7f7bf80ca300 2023-10-11T06:13:09.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bfe10a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf8072310 con 0x7f7bf815d4f0 2023-10-11T06:13:09.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bfe10a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf8070280 con 0x7f7bf80c9360 2023-10-11T06:13:09.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bf77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7bf815d4f0 0x7f7bf80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:09.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bf77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7bf815d4f0 0x7f7bf80cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60224/0 (socket says 172.21.15.84:60224) 2023-10-11T06:13:09.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.781+0000 7f7bf77fe640 1 -- 172.21.15.84:0/59487679 learned_addr learned my addr 172.21.15.84:0/59487679 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:09.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.782+0000 7f7bf7fff640 1 --2- 172.21.15.84:0/59487679 >> v2:172.21.15.99:3300/0 conn(0x7f7bf80c9360 0x7f7bf80ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:09.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.782+0000 7f7bf77fe640 1 -- 172.21.15.84:0/59487679 >> v2:172.21.15.99:3300/0 conn(0x7f7bf80c9360 msgr2=0x7f7bf80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:09.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.782+0000 7f7bf77fe640 1 --2- 172.21.15.84:0/59487679 >> v2:172.21.15.99:3300/0 conn(0x7f7bf80c9360 0x7f7bf80ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.782+0000 7f7bf77fe640 1 -- 172.21.15.84:0/59487679 >> v1:172.21.15.99:6789/0 conn(0x7f7bf80ca300 legacy=0x7f7bf80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:09.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.782+0000 7f7bf77fe640 1 -- 172.21.15.84:0/59487679 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8071cc0 con 0x7f7bf815d4f0 2023-10-11T06:13:09.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.783+0000 7f7bf77fe640 1 --2- 172.21.15.84:0/59487679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf815d4f0 0x7f7bf80cb540 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80b4740 tx=0x7f7be802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5f7b43d9afdaa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:09.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.783+0000 7f7bf6ffd640 1 -- 172.21.15.84:0/59487679 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004230 con 0x7f7bf815d4f0 2023-10-11T06:13:09.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.783+0000 7f7bf6ffd640 1 -- 172.21.15.84:0/59487679 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be8004390 con 0x7f7bf815d4f0 2023-10-11T06:13:09.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.783+0000 7f7bf6ffd640 1 -- 172.21.15.84:0/59487679 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8003810 con 0x7f7bf815d4f0 2023-10-11T06:13:09.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.783+0000 7f7bfe10a640 1 -- 172.21.15.84:0/59487679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf815d4f0 msgr2=0x7f7bf80cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:09.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.783+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/59487679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf815d4f0 0x7f7bf80cb540 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80b4740 tx=0x7f7be802b8f0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.784+0000 7f7bfe10a640 1 -- 172.21.15.84:0/59487679 shutdown_connections 2023-10-11T06:13:09.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.784+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/59487679 >> v2:172.21.15.99:3300/0 conn(0x7f7bf80c9360 0x7f7bf80ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.784+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/59487679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf815d4f0 0x7f7bf80cb540 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.784+0000 7f7bfe10a640 1 -- 172.21.15.84:0/59487679 >> 172.21.15.84:0/59487679 conn(0x7f7bf8069c70 msgr2=0x7f7bf806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:09.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.784+0000 7f7bfe10a640 1 -- 172.21.15.84:0/59487679 shutdown_connections 2023-10-11T06:13:09.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.784+0000 7f7bfe10a640 1 -- 172.21.15.84:0/59487679 wait complete. 2023-10-11T06:13:09.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.785+0000 7f7bfe10a640 1 Processor -- start 2023-10-11T06:13:09.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.786+0000 7f7bfe10a640 1 -- start start 2023-10-11T06:13:09.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.786+0000 7f7bfe10a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf80c9360 0x7f7bf814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:09.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.786+0000 7f7bfe10a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf80ca300 0x7f7bf8154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:09.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.786+0000 7f7bfe10a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bf815d4f0 0x7f7bf8151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:09.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.786+0000 7f7bfe10a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf814dc80 con 0x7f7bf80c9360 2023-10-11T06:13:09.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.786+0000 7f7bfe10a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf814d7b0 con 0x7f7bf80ca300 2023-10-11T06:13:09.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.786+0000 7f7bfe10a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80b3160 con 0x7f7bf815d4f0 2023-10-11T06:13:09.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf80c9360 0x7f7bf814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:09.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf80c9360 0x7f7bf814d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60240/0 (socket says 172.21.15.84:60240) 2023-10-11T06:13:09.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 -- 172.21.15.84:0/2269851172 learned_addr learned my addr 172.21.15.84:0/2269851172 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:09.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 -- 172.21.15.84:0/2269851172 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bf815d4f0 msgr2=0x7f7bf8151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:09.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bf815d4f0 0x7f7bf8151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 -- 172.21.15.84:0/2269851172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf80ca300 msgr2=0x7f7bf8154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:09.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf80ca300 0x7f7bf8154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 -- 172.21.15.84:0/2269851172 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be8009460 con 0x7f7bf80c9360 2023-10-11T06:13:09.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.787+0000 7f7bf77fe640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf80c9360 0x7f7bf814d270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7be80090b0 tx=0x7f7be80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:09.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.788+0000 7f7bf4ff9640 1 -- 172.21.15.84:0/2269851172 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8003c40 con 0x7f7bf80c9360 2023-10-11T06:13:09.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.788+0000 7f7bf4ff9640 1 -- 172.21.15.84:0/2269851172 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be803f070 con 0x7f7bf80c9360 2023-10-11T06:13:09.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.788+0000 7f7bf4ff9640 1 -- 172.21.15.84:0/2269851172 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004500 con 0x7f7bf80c9360 2023-10-11T06:13:09.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.788+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf80b28d0 con 0x7f7bf80c9360 2023-10-11T06:13:09.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.788+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf80b2b10 con 0x7f7bf80c9360 2023-10-11T06:13:09.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.789+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8073ed0 con 0x7f7bf80c9360 2023-10-11T06:13:09.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.790+0000 7f7bf4ff9640 1 -- 172.21.15.84:0/2269851172 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f7be8041030 con 0x7f7bf80c9360 2023-10-11T06:13:09.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.791+0000 7f7bf4ff9640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7bd806a830 0x7f7bd806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:09.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.791+0000 7f7bf4ff9640 1 -- 172.21.15.84:0/2269851172 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f7be80ab420 con 0x7f7bf80c9360 2023-10-11T06:13:09.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.791+0000 7f7beffff640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7bd806a830 0x7f7bd806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:09.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.792+0000 7f7beffff640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7bd806a830 0x7f7bd806cce0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8164f20 tx=0x7f7be0007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:09.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.795+0000 7f7bf4ff9640 1 -- 172.21.15.84:0/2269851172 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f7be80764f0 con 0x7f7bf80c9360 2023-10-11T06:13:09.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.986+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf80b09b0 con 0x7f7bf80c9360 2023-10-11T06:13:09.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.990+0000 7f7bf4ff9640 1 -- 172.21.15.84:0/2269851172 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7f7be8079b10 con 0x7f7bf80c9360 2023-10-11T06:13:09.992 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:09.992 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.992+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7bd806a830 msgr2=0x7f7bd806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.992+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7bd806a830 0x7f7bd806cce0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8164f20 tx=0x7f7be0007990 comp rx=0 tx=0).stop 2023-10-11T06:13:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.992+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf80c9360 msgr2=0x7f7bf814d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:09.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.992+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf80c9360 0x7f7bf814d270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7be80090b0 tx=0x7f7be80046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.992+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 shutdown_connections 2023-10-11T06:13:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.992+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7bd806a830 0x7f7bd806cce0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.992+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bf815d4f0 0x7f7bf8151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.993+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf80ca300 0x7f7bf8154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.993+0000 7f7bfe10a640 1 --2- 172.21.15.84:0/2269851172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bf80c9360 0x7f7bf814d270 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.993+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 >> 172.21.15.84:0/2269851172 conn(0x7f7bf8069c70 msgr2=0x7f7bf80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.993+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 shutdown_connections 2023-10-11T06:13:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:09.993+0000 7f7bfe10a640 1 -- 172.21.15.84:0/2269851172 wait complete. 2023-10-11T06:13:10.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:12:38.260176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:12:39.011036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-11T06:12:50.853138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:13:10.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:13:10.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.233+0000 7fa13b28f640 1 Processor -- start 2023-10-11T06:13:10.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.233+0000 7fa13b28f640 1 -- start start 2023-10-11T06:13:10.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13b28f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa13415f7e0 0x7fa1340cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13b28f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1340cd6c0 0x7fa1340d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13b28f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa1340d3000 0x7fa13415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13b28f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1340d3460 con 0x7fa13415f7e0 2023-10-11T06:13:10.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13b28f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1340cf220 con 0x7fa1340d3000 2023-10-11T06:13:10.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13b28f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340cf500 con 0x7fa1340cd6c0 2023-10-11T06:13:10.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13902b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa13415f7e0 0x7fa1340cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13902b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa13415f7e0 0x7fa1340cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60252/0 (socket says 172.21.15.84:60252) 2023-10-11T06:13:10.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.235+0000 7fa13902b640 1 -- 172.21.15.84:0/3447071376 learned_addr learned my addr 172.21.15.84:0/3447071376 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:10.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13882a640 1 --2- 172.21.15.84:0/3447071376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1340cd6c0 0x7fa1340d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13902b640 1 -- 172.21.15.84:0/3447071376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1340cd6c0 msgr2=0x7fa1340d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:10.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13902b640 1 --2- 172.21.15.84:0/3447071376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1340cd6c0 0x7fa1340d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13902b640 1 -- 172.21.15.84:0/3447071376 >> v2:172.21.15.99:3300/0 conn(0x7fa1340d3000 msgr2=0x7fa13415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:10.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13982c640 1 --2- 172.21.15.84:0/3447071376 >> v2:172.21.15.99:3300/0 conn(0x7fa1340d3000 0x7fa13415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13902b640 1 --2- 172.21.15.84:0/3447071376 >> v2:172.21.15.99:3300/0 conn(0x7fa1340d3000 0x7fa13415fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13902b640 1 -- 172.21.15.84:0/3447071376 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1340cf7e0 con 0x7fa13415f7e0 2023-10-11T06:13:10.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.236+0000 7fa13982c640 1 --2- 172.21.15.84:0/3447071376 >> v2:172.21.15.99:3300/0 conn(0x7fa1340d3000 0x7fa13415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:10.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.237+0000 7fa13902b640 1 --2- 172.21.15.84:0/3447071376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13415f7e0 0x7fa1340cd180 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa13408bef0 tx=0x7fa12402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b58322d696481a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:10.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.238+0000 7fa133fff640 1 -- 172.21.15.84:0/3447071376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124031870 con 0x7fa13415f7e0 2023-10-11T06:13:10.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.238+0000 7fa133fff640 1 -- 172.21.15.84:0/3447071376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1240319d0 con 0x7fa13415f7e0 2023-10-11T06:13:10.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.238+0000 7fa133fff640 1 -- 172.21.15.84:0/3447071376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124031d60 con 0x7fa13415f7e0 2023-10-11T06:13:10.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.238+0000 7fa13b28f640 1 -- 172.21.15.84:0/3447071376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13415f7e0 msgr2=0x7fa1340cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.238+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3447071376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13415f7e0 0x7fa1340cd180 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa13408bef0 tx=0x7fa12402b910 comp rx=0 tx=0).stop 2023-10-11T06:13:10.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.239+0000 7fa13b28f640 1 -- 172.21.15.84:0/3447071376 shutdown_connections 2023-10-11T06:13:10.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.239+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3447071376 >> v2:172.21.15.99:3300/0 conn(0x7fa1340d3000 0x7fa13415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.239+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3447071376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa1340cd6c0 0x7fa1340d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.239+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3447071376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa13415f7e0 0x7fa1340cd180 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.239+0000 7fa13b28f640 1 -- 172.21.15.84:0/3447071376 >> 172.21.15.84:0/3447071376 conn(0x7fa1340713a0 msgr2=0x7fa1340717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:10.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.239+0000 7fa13b28f640 1 -- 172.21.15.84:0/3447071376 shutdown_connections 2023-10-11T06:13:10.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.239+0000 7fa13b28f640 1 -- 172.21.15.84:0/3447071376 wait complete. 2023-10-11T06:13:10.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.240+0000 7fa13b28f640 1 Processor -- start 2023-10-11T06:13:10.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.241+0000 7fa13b28f640 1 -- start start 2023-10-11T06:13:10.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.241+0000 7fa13b28f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1340cd6c0 0x7fa13414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.241+0000 7fa13b28f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1340d3000 0x7fa1340b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13b28f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f7e0 0x7fa1340b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13b28f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c7e90 con 0x7fa1340cd6c0 2023-10-11T06:13:10.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13b28f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c7210 con 0x7fa1340d3000 2023-10-11T06:13:10.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13b28f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340c74f0 con 0x7fa13415f7e0 2023-10-11T06:13:10.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1340cd6c0 0x7fa13414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1340cd6c0 0x7fa13414bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60264/0 (socket says 172.21.15.84:60264) 2023-10-11T06:13:10.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 -- 172.21.15.84:0/3536720780 learned_addr learned my addr 172.21.15.84:0/3536720780 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:10.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13982c640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f7e0 0x7fa1340b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 -- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f7e0 msgr2=0x7fa1340b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f7e0 0x7fa1340b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13882a640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1340d3000 0x7fa1340b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 -- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1340d3000 msgr2=0x7fa1340b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1340d3000 0x7fa1340b7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.242+0000 7fa13902b640 1 -- 172.21.15.84:0/3536720780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1240094d0 con 0x7fa1340cd6c0 2023-10-11T06:13:10.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.243+0000 7fa13902b640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1340cd6c0 0x7fa13414bfd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa12402b950 tx=0x7fa1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:10.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.243+0000 7fa131ffb640 1 -- 172.21.15.84:0/3536720780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124031860 con 0x7fa1340cd6c0 2023-10-11T06:13:10.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.243+0000 7fa131ffb640 1 -- 172.21.15.84:0/3536720780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1240443f0 con 0x7fa1340cd6c0 2023-10-11T06:13:10.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.243+0000 7fa131ffb640 1 -- 172.21.15.84:0/3536720780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1240447a0 con 0x7fa1340cd6c0 2023-10-11T06:13:10.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.243+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1340c77d0 con 0x7fa1340cd6c0 2023-10-11T06:13:10.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.243+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13414c510 con 0x7fa1340cd6c0 2023-10-11T06:13:10.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.244+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134073ed0 con 0x7fa1340cd6c0 2023-10-11T06:13:10.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.245+0000 7fa131ffb640 1 -- 172.21.15.84:0/3536720780 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa12404d450 con 0x7fa1340cd6c0 2023-10-11T06:13:10.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.246+0000 7fa131ffb640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa12006a830 0x7fa12006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.246+0000 7fa131ffb640 1 -- 172.21.15.84:0/3536720780 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fa12402d040 con 0x7fa1340cd6c0 2023-10-11T06:13:10.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.246+0000 7fa13882a640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa12006a830 0x7fa12006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.247+0000 7fa13882a640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa12006a830 0x7fa12006cce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa128006fd0 tx=0x7fa128008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:10.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.250+0000 7fa131ffb640 1 -- 172.21.15.84:0/3536720780 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa124076a40 con 0x7fa1340cd6c0 2023-10-11T06:13:10.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.441+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1340b5030 con 0x7fa1340cd6c0 2023-10-11T06:13:10.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.442+0000 7fa131ffb640 1 -- 172.21.15.84:0/3536720780 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1240350a0 con 0x7fa1340cd6c0 2023-10-11T06:13:10.444 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:13:10.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa12006a830 msgr2=0x7fa12006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa12006a830 0x7fa12006cce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa128006fd0 tx=0x7fa128008040 comp rx=0 tx=0).stop 2023-10-11T06:13:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1340cd6c0 msgr2=0x7fa13414bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1340cd6c0 0x7fa13414bfd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa12402b950 tx=0x7fa1240046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 shutdown_connections 2023-10-11T06:13:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa12006a830 0x7fa12006cce0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f7e0 0x7fa1340b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1340d3000 0x7fa1340b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 --2- 172.21.15.84:0/3536720780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1340cd6c0 0x7fa13414bfd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 >> 172.21.15.84:0/3536720780 conn(0x7fa1340713a0 msgr2=0x7fa134159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:10.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 shutdown_connections 2023-10-11T06:13:10.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.445+0000 7fa13b28f640 1 -- 172.21.15.84:0/3536720780 wait complete. 2023-10-11T06:13:10.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:13:10.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.684+0000 7f905536e640 1 Processor -- start 2023-10-11T06:13:10.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.684+0000 7f905536e640 1 -- start start 2023-10-11T06:13:10.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f905536e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f905536e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90500ce510 0x7f90500ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f905536e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500c9b90 con 0x7f9050150300 2023-10-11T06:13:10.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f905536e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905015da10 con 0x7f90500ce510 2023-10-11T06:13:10.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f905536e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90500ceef0 con 0x7f905014ff30 2023-10-11T06:13:10.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f904f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90500ce510 0x7f90500ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f904e7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9050150300 0x7f90500cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43584/0 (socket says 172.21.15.84:43584) 2023-10-11T06:13:10.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.686+0000 7f904e7fc640 1 -- 172.21.15.84:0/2020540292 learned_addr learned my addr 172.21.15.84:0/2020540292 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:10.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.687+0000 7f904effd640 1 --2- 172.21.15.84:0/2020540292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.687+0000 7f904f7fe640 1 -- 172.21.15.84:0/2020540292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 msgr2=0x7f90500c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:10.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.687+0000 7f904f7fe640 1 --2- 172.21.15.84:0/2020540292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.687+0000 7f904f7fe640 1 -- 172.21.15.84:0/2020540292 >> v1:172.21.15.84:6789/0 conn(0x7f9050150300 legacy=0x7f90500cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.687+0000 7f904f7fe640 1 -- 172.21.15.84:0/2020540292 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90500cc990 con 0x7f90500ce510 2023-10-11T06:13:10.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.687+0000 7f904dffb640 1 -- 172.21.15.84:0/2020540292 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808406821 0 0) 0x7f90500c9b90 con 0x7f9050150300 2023-10-11T06:13:10.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.688+0000 7f904f7fe640 1 --2- 172.21.15.84:0/2020540292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90500ce510 0x7f90500ce9b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9048009d90 tx=0x7f904802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ec5e5f5ea08d7c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:10.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.688+0000 7f904dffb640 1 -- 172.21.15.84:0/2020540292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904802cbd0 con 0x7f90500ce510 2023-10-11T06:13:10.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.688+0000 7f904dffb640 1 -- 172.21.15.84:0/2020540292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904802cd30 con 0x7f90500ce510 2023-10-11T06:13:10.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.688+0000 7f904dffb640 1 -- 172.21.15.84:0/2020540292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904802cea0 con 0x7f90500ce510 2023-10-11T06:13:10.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 -- 172.21.15.84:0/2020540292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90500ce510 msgr2=0x7f90500ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 --2- 172.21.15.84:0/2020540292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90500ce510 0x7f90500ce9b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9048009d90 tx=0x7f904802be20 comp rx=0 tx=0).stop 2023-10-11T06:13:10.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 -- 172.21.15.84:0/2020540292 shutdown_connections 2023-10-11T06:13:10.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 --2- 172.21.15.84:0/2020540292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90500ce510 0x7f90500ce9b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 --2- 172.21.15.84:0/2020540292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 -- 172.21.15.84:0/2020540292 >> 172.21.15.84:0/2020540292 conn(0x7f9050069c70 msgr2=0x7f905006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:10.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 -- 172.21.15.84:0/2020540292 shutdown_connections 2023-10-11T06:13:10.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.689+0000 7f905536e640 1 -- 172.21.15.84:0/2020540292 wait complete. 2023-10-11T06:13:10.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.691+0000 7f905536e640 1 Processor -- start 2023-10-11T06:13:10.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.691+0000 7f905536e640 1 -- start start 2023-10-11T06:13:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f905536e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90500ce510 0x7f90500b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f905536e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f905536e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9050150300 0x7f9050070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f905536e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500b3060 con 0x7f9050150300 2023-10-11T06:13:10.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f905536e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500c8520 con 0x7f90500ce510 2023-10-11T06:13:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f905536e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90500b23e0 con 0x7f905014ff30 2023-10-11T06:13:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f904f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9050150300 0x7f9050070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f904f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9050150300 0x7f9050070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60288/0 (socket says 172.21.15.84:60288) 2023-10-11T06:13:10.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.692+0000 7f904f7fe640 1 -- 172.21.15.84:0/22820642 learned_addr learned my addr 172.21.15.84:0/22820642 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:10.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904f7fe640 1 -- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 msgr2=0x7f90500c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904effd640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90500ce510 0x7f90500b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904e7fc640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904f7fe640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904f7fe640 1 -- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90500ce510 msgr2=0x7f90500b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904f7fe640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90500ce510 0x7f90500b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904f7fe640 1 -- 172.21.15.84:0/22820642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048009a70 con 0x7f9050150300 2023-10-11T06:13:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f904f7fe640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9050150300 0x7f9050070540 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9048009eb0 tx=0x7f9048004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f90377fe640 1 -- 172.21.15.84:0/22820642 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904802cea0 con 0x7f9050150300 2023-10-11T06:13:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f90377fe640 1 -- 172.21.15.84:0/22820642 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904803f420 con 0x7f9050150300 2023-10-11T06:13:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90500b26c0 con 0x7f9050150300 2023-10-11T06:13:10.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.693+0000 7f90377fe640 1 -- 172.21.15.84:0/22820642 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904803f790 con 0x7f9050150300 2023-10-11T06:13:10.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.694+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90500b2c80 con 0x7f9050150300 2023-10-11T06:13:10.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.695+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90500b31a0 con 0x7f9050150300 2023-10-11T06:13:10.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.696+0000 7f90377fe640 1 -- 172.21.15.84:0/22820642 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f904803f8f0 con 0x7f9050150300 2023-10-11T06:13:10.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.697+0000 7f90377fe640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f903c06a830 0x7f903c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.697+0000 7f90377fe640 1 -- 172.21.15.84:0/22820642 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f90480abc10 con 0x7f9050150300 2023-10-11T06:13:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.697+0000 7f904effd640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f903c06a830 0x7f903c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.698+0000 7f904effd640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f903c06a830 0x7f903c06cce0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9040006fd0 tx=0x7f9040008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:10.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.701+0000 7f90377fe640 1 -- 172.21.15.84:0/22820642 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f90500b31a0 con 0x7f9050150300 2023-10-11T06:13:10.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.889+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f90500b31a0 con 0x7f9050150300 2023-10-11T06:13:10.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.890+0000 7f90377fe640 1 -- 172.21.15.84:0/22820642 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90500b31a0 con 0x7f9050150300 2023-10-11T06:13:10.892 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:13:10.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f903c06a830 msgr2=0x7f903c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f903c06a830 0x7f903c06cce0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9040006fd0 tx=0x7f9040008040 comp rx=0 tx=0).stop 2023-10-11T06:13:10.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9050150300 msgr2=0x7f9050070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:10.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9050150300 0x7f9050070540 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9048009eb0 tx=0x7f9048004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:10.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 shutdown_connections 2023-10-11T06:13:10.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f903c06a830 0x7f903c06cce0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9050150300 0x7f9050070540 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f905014ff30 0x7f90500c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 --2- 172.21.15.84:0/22820642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90500ce510 0x7f90500b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:10.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 >> 172.21.15.84:0/22820642 conn(0x7f9050069c70 msgr2=0x7f905006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:10.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.892+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 shutdown_connections 2023-10-11T06:13:10.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:10.893+0000 7f905536e640 1 -- 172.21.15.84:0/22820642 wait complete. 2023-10-11T06:13:10.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:13:11.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.104+0000 7f8f3ea5c640 1 Processor -- start 2023-10-11T06:13:11.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.105+0000 7f8f3ea5c640 1 -- start start 2023-10-11T06:13:11.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.106+0000 7f8f3ea5c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8f3815f7e0 0x7f8f380cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.106+0000 7f8f3ea5c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8f380cd6c0 0x7f8f380d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.106+0000 7f8f3ea5c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f380d3420 con 0x7f8f380d3000 2023-10-11T06:13:11.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.106+0000 7f8f3ea5c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f380cf220 con 0x7f8f380cd6c0 2023-10-11T06:13:11.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.106+0000 7f8f3ea5c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f380cf500 con 0x7f8f3815f7e0 2023-10-11T06:13:11.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f377fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8f380cd6c0 0x7f8f380d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f377fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8f380cd6c0 0x7f8f380d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60298/0 (socket says 172.21.15.84:60298) 2023-10-11T06:13:11.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f377fe640 1 -- 172.21.15.84:0/3587348579 learned_addr learned my addr 172.21.15.84:0/3587348579 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:11.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f37fff640 1 --2- 172.21.15.84:0/3587348579 >> v2:172.21.15.99:3300/0 conn(0x7f8f3815f7e0 0x7f8f380cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f377fe640 1 -- 172.21.15.84:0/3587348579 >> v2:172.21.15.99:3300/0 conn(0x7f8f3815f7e0 msgr2=0x7f8f380cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:11.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f377fe640 1 --2- 172.21.15.84:0/3587348579 >> v2:172.21.15.99:3300/0 conn(0x7f8f3815f7e0 0x7f8f380cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f377fe640 1 -- 172.21.15.84:0/3587348579 >> v1:172.21.15.99:6789/0 conn(0x7f8f380d3000 legacy=0x7f8f3815fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:11.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.107+0000 7f8f377fe640 1 -- 172.21.15.84:0/3587348579 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380cf7e0 con 0x7f8f380cd6c0 2023-10-11T06:13:11.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.108+0000 7f8f377fe640 1 --2- 172.21.15.84:0/3587348579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f380d2ac0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f38072100 tx=0x7f8f2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab976c8a168a1f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:11.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.109+0000 7f8f36ffd640 1 -- 172.21.15.84:0/3587348579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20031870 con 0x7f8f380cd6c0 2023-10-11T06:13:11.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.109+0000 7f8f36ffd640 1 -- 172.21.15.84:0/3587348579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f200319d0 con 0x7f8f380cd6c0 2023-10-11T06:13:11.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.109+0000 7f8f36ffd640 1 -- 172.21.15.84:0/3587348579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20031d60 con 0x7f8f380cd6c0 2023-10-11T06:13:11.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/3587348579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 msgr2=0x7f8f380d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/3587348579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f380d2ac0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f38072100 tx=0x7f8f2002b910 comp rx=0 tx=0).stop 2023-10-11T06:13:11.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/3587348579 shutdown_connections 2023-10-11T06:13:11.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/3587348579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f380d2ac0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/3587348579 >> v2:172.21.15.99:3300/0 conn(0x7f8f3815f7e0 0x7f8f380cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/3587348579 >> 172.21.15.84:0/3587348579 conn(0x7f8f380713a0 msgr2=0x7f8f380717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:11.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/3587348579 shutdown_connections 2023-10-11T06:13:11.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.110+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/3587348579 wait complete. 2023-10-11T06:13:11.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.111+0000 7f8f3ea5c640 1 Processor -- start 2023-10-11T06:13:11.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.112+0000 7f8f3ea5c640 1 -- start start 2023-10-11T06:13:11.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.112+0000 7f8f3ea5c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f381f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.112+0000 7f8f3ea5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f380d3000 0x7f8f380b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.112+0000 7f8f3ea5c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f3815f7e0 0x7f8f3815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f3ea5c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381f1a90 con 0x7f8f380cd6c0 2023-10-11T06:13:11.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f3ea5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f380b88e0 con 0x7f8f380d3000 2023-10-11T06:13:11.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f3ea5c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38155eb0 con 0x7f8f3815f7e0 2023-10-11T06:13:11.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f381f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f381f1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60314/0 (socket says 172.21.15.84:60314) 2023-10-11T06:13:11.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 -- 172.21.15.84:0/1131959637 learned_addr learned my addr 172.21.15.84:0/1131959637 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:11.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 -- 172.21.15.84:0/1131959637 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f3815f7e0 msgr2=0x7f8f3815a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f3815f7e0 0x7f8f3815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 -- 172.21.15.84:0/1131959637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f380d3000 msgr2=0x7f8f380b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f380d3000 0x7f8f380b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 -- 172.21.15.84:0/1131959637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200094d0 con 0x7f8f380cd6c0 2023-10-11T06:13:11.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.113+0000 7f8f37fff640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f381f1550 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f28007e70 tx=0x7f8f2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:11.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.114+0000 7f8f34ff9640 1 -- 172.21.15.84:0/1131959637 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28014620 con 0x7f8f380cd6c0 2023-10-11T06:13:11.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.114+0000 7f8f34ff9640 1 -- 172.21.15.84:0/1131959637 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f28014780 con 0x7f8f380cd6c0 2023-10-11T06:13:11.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.114+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f38156190 con 0x7f8f380cd6c0 2023-10-11T06:13:11.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.114+0000 7f8f34ff9640 1 -- 172.21.15.84:0/1131959637 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28014b10 con 0x7f8f380cd6c0 2023-10-11T06:13:11.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.114+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f38156470 con 0x7f8f380cd6c0 2023-10-11T06:13:11.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.115+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f38073ed0 con 0x7f8f380cd6c0 2023-10-11T06:13:11.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.116+0000 7f8f34ff9640 1 -- 172.21.15.84:0/1131959637 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8f28014c70 con 0x7f8f380cd6c0 2023-10-11T06:13:11.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.117+0000 7f8f34ff9640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f1806a830 0x7f8f1806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.117+0000 7f8f34ff9640 1 -- 172.21.15.84:0/1131959637 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f8f28089b30 con 0x7f8f380cd6c0 2023-10-11T06:13:11.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.117+0000 7f8f377fe640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f1806a830 0x7f8f1806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.118+0000 7f8f377fe640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f1806a830 0x7f8f1806cce0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f20036000 tx=0x7f8f20030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:11.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.121+0000 7f8f34ff9640 1 -- 172.21.15.84:0/1131959637 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8f28054bf0 con 0x7f8f380cd6c0 2023-10-11T06:13:11.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.308+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f380b5030 con 0x7f8f380cd6c0 2023-10-11T06:13:11.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.309+0000 7f8f34ff9640 1 -- 172.21.15.84:0/1131959637 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f28058210 con 0x7f8f380cd6c0 2023-10-11T06:13:11.311 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:11.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f1806a830 msgr2=0x7f8f1806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f1806a830 0x7f8f1806cce0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f20036000 tx=0x7f8f20030000 comp rx=0 tx=0).stop 2023-10-11T06:13:11.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 msgr2=0x7f8f381f1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f381f1550 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f28007e70 tx=0x7f8f2800e4b0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 shutdown_connections 2023-10-11T06:13:11.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8f1806a830 0x7f8f1806cce0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f3815f7e0 0x7f8f3815a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f380d3000 0x7f8f380b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 --2- 172.21.15.84:0/1131959637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f380cd6c0 0x7f8f381f1550 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 >> 172.21.15.84:0/1131959637 conn(0x7f8f380713a0 msgr2=0x7f8f3815ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:11.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.312+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 shutdown_connections 2023-10-11T06:13:11.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.313+0000 7f8f3ea5c640 1 -- 172.21.15.84:0/1131959637 wait complete. 2023-10-11T06:13:11.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:13:11.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.554+0000 7f36918bd640 1 Processor -- start 2023-10-11T06:13:11.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.554+0000 7f36918bd640 1 -- start start 2023-10-11T06:13:11.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f36918bd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f368c15a1f0 0x7f368c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f36918bd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f368c15e740 0x7f368c0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f36918bd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c0b2830 0x7f368c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f36918bd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0cb280 con 0x7f368c15a1f0 2023-10-11T06:13:11.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f36918bd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0cd1a0 con 0x7f368c15e740 2023-10-11T06:13:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f36918bd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b2cb0 con 0x7f368c0b2830 2023-10-11T06:13:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f368affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f368c15a1f0 0x7f368c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f368affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f368c15a1f0 0x7f368c0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60320/0 (socket says 172.21.15.84:60320) 2023-10-11T06:13:11.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.556+0000 7f368affd640 1 -- 172.21.15.84:0/1518745292 learned_addr learned my addr 172.21.15.84:0/1518745292 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:11.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.557+0000 7f368affd640 1 -- 172.21.15.84:0/1518745292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c0b2830 msgr2=0x7f368c06fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:11.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.557+0000 7f368affd640 1 --2- 172.21.15.84:0/1518745292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c0b2830 0x7f368c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.557+0000 7f368a7fc640 1 --2- 172.21.15.84:0/1518745292 >> v2:172.21.15.99:3300/0 conn(0x7f368c15e740 0x7f368c0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.557+0000 7f368affd640 1 -- 172.21.15.84:0/1518745292 >> v2:172.21.15.99:3300/0 conn(0x7f368c15e740 msgr2=0x7f368c0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:11.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.557+0000 7f368affd640 1 --2- 172.21.15.84:0/1518745292 >> v2:172.21.15.99:3300/0 conn(0x7f368c15e740 0x7f368c0b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.557+0000 7f368affd640 1 -- 172.21.15.84:0/1518745292 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0ccec0 con 0x7f368c15a1f0 2023-10-11T06:13:11.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.557+0000 7f368a7fc640 1 --2- 172.21.15.84:0/1518745292 >> v2:172.21.15.99:3300/0 conn(0x7f368c15e740 0x7f368c0b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:11.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.558+0000 7f368affd640 1 --2- 172.21.15.84:0/1518745292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c15a1f0 0x7f368c0cc820 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f368c0cde50 tx=0x7f367c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0654880b18c702 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:11.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.558+0000 7f3689ffb640 1 -- 172.21.15.84:0/1518745292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c031870 con 0x7f368c15a1f0 2023-10-11T06:13:11.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.558+0000 7f3689ffb640 1 -- 172.21.15.84:0/1518745292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c0319d0 con 0x7f368c15a1f0 2023-10-11T06:13:11.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.558+0000 7f3689ffb640 1 -- 172.21.15.84:0/1518745292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c031d60 con 0x7f368c15a1f0 2023-10-11T06:13:11.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 -- 172.21.15.84:0/1518745292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c15a1f0 msgr2=0x7f368c0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 --2- 172.21.15.84:0/1518745292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c15a1f0 0x7f368c0cc820 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f368c0cde50 tx=0x7f367c02b910 comp rx=0 tx=0).stop 2023-10-11T06:13:11.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 -- 172.21.15.84:0/1518745292 shutdown_connections 2023-10-11T06:13:11.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 --2- 172.21.15.84:0/1518745292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c0b2830 0x7f368c06fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 --2- 172.21.15.84:0/1518745292 >> v2:172.21.15.99:3300/0 conn(0x7f368c15e740 0x7f368c0b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 --2- 172.21.15.84:0/1518745292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c15a1f0 0x7f368c0cc820 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 -- 172.21.15.84:0/1518745292 >> 172.21.15.84:0/1518745292 conn(0x7f368c06ef20 msgr2=0x7f368c0cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:11.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 -- 172.21.15.84:0/1518745292 shutdown_connections 2023-10-11T06:13:11.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.559+0000 7f36918bd640 1 -- 172.21.15.84:0/1518745292 wait complete. 2023-10-11T06:13:11.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.561+0000 7f36918bd640 1 Processor -- start 2023-10-11T06:13:11.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.561+0000 7f36918bd640 1 -- start start 2023-10-11T06:13:11.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f36918bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c0b2830 0x7f368c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f36918bd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c15a1f0 0x7f368c15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f36918bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368c15e740 0x7f368c0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f36918bd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c141fd0 con 0x7f368c0b2830 2023-10-11T06:13:11.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f36918bd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15b5d0 con 0x7f368c15e740 2023-10-11T06:13:11.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f36918bd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15b2f0 con 0x7f368c15a1f0 2023-10-11T06:13:11.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f368affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c0b2830 0x7f368c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f368affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c0b2830 0x7f368c141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60326/0 (socket says 172.21.15.84:60326) 2023-10-11T06:13:11.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.562+0000 7f368affd640 1 -- 172.21.15.84:0/2628317893 learned_addr learned my addr 172.21.15.84:0/2628317893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:11.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f368affd640 1 -- 172.21.15.84:0/2628317893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c15a1f0 msgr2=0x7f368c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f368affd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c15a1f0 0x7f368c15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f368affd640 1 -- 172.21.15.84:0/2628317893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368c15e740 msgr2=0x7f368c0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f368affd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368c15e740 0x7f368c0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f368affd640 1 -- 172.21.15.84:0/2628317893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c0094d0 con 0x7f368c0b2830 2023-10-11T06:13:11.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f368affd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c0b2830 0x7f368c141a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f367c000c00 tx=0x7f367c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:11.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f3673fff640 1 -- 172.21.15.84:0/2628317893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c031840 con 0x7f368c0b2830 2023-10-11T06:13:11.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f3673fff640 1 -- 172.21.15.84:0/2628317893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c0443f0 con 0x7f368c0b2830 2023-10-11T06:13:11.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f3673fff640 1 -- 172.21.15.84:0/2628317893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c0447c0 con 0x7f368c0b2830 2023-10-11T06:13:11.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.563+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c15c240 con 0x7f368c0b2830 2023-10-11T06:13:11.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.564+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c15c800 con 0x7f368c0b2830 2023-10-11T06:13:11.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.565+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c067090 con 0x7f368c0b2830 2023-10-11T06:13:11.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.566+0000 7f3673fff640 1 -- 172.21.15.84:0/2628317893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f367c04d450 con 0x7f368c0b2830 2023-10-11T06:13:11.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.566+0000 7f3673fff640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f367806a830 0x7f367806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.567+0000 7f3673fff640 1 -- 172.21.15.84:0/2628317893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f367c02d040 con 0x7f368c0b2830 2023-10-11T06:13:11.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.567+0000 7f368a7fc640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f367806a830 0x7f367806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:11.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.567+0000 7f368a7fc640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f367806a830 0x7f367806cce0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3680006fd0 tx=0x7f3680008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:11.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.571+0000 7f3673fff640 1 -- 172.21.15.84:0/2628317893 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f367c076a70 con 0x7f368c0b2830 2023-10-11T06:13:11.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.760+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f368c15c9f0 con 0x7f368c0b2830 2023-10-11T06:13:11.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.761+0000 7f3673fff640 1 -- 172.21.15.84:0/2628317893 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f367c07a090 con 0x7f368c0b2830 2023-10-11T06:13:11.763 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:13:11.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f367806a830 msgr2=0x7f367806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f367806a830 0x7f367806cce0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3680006fd0 tx=0x7f3680008040 comp rx=0 tx=0).stop 2023-10-11T06:13:11.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c0b2830 msgr2=0x7f368c141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:11.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c0b2830 0x7f368c141a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f367c000c00 tx=0x7f367c0046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 shutdown_connections 2023-10-11T06:13:11.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f367806a830 0x7f367806cce0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368c15e740 0x7f368c0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f368c15a1f0 0x7f368c15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 --2- 172.21.15.84:0/2628317893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f368c0b2830 0x7f368c141a90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:11.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.764+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 >> 172.21.15.84:0/2628317893 conn(0x7f368c06ef20 msgr2=0x7f368c0c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:11.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.765+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 shutdown_connections 2023-10-11T06:13:11.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.765+0000 7f36918bd640 1 -- 172.21.15.84:0/2628317893 wait complete. 2023-10-11T06:13:11.784 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-11T06:13:11.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.995+0000 7feaae3f0640 1 Processor -- start 2023-10-11T06:13:11.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.995+0000 7feaae3f0640 1 -- start start 2023-10-11T06:13:11.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaae3f0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feaa80ced30 0x7feaa80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaae3f0640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 0x7feaa806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:11.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaae3f0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa8071380 con 0x7feaa815a0b0 2023-10-11T06:13:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaae3f0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa806e0f0 con 0x7feaa80ced30 2023-10-11T06:13:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaae3f0640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80cbae0 con 0x7feaa80c8d00 2023-10-11T06:13:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaa7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7feaa815a0b0 0x7feaa80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43598/0 (socket says 172.21.15.84:43598) 2023-10-11T06:13:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaa7fff640 1 -- 172.21.15.84:0/1821401409 learned_addr learned my addr 172.21.15.84:0/1821401409 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaac98d640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 0x7feaa806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7fea9ffff640 1 --2- 172.21.15.84:0/1821401409 >> v2:172.21.15.99:3300/0 conn(0x7feaa80ced30 0x7feaa80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.997+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888576930 0 0) 0x7feaa8071380 con 0x7feaa815a0b0 2023-10-11T06:13:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea94003520 con 0x7feaa815a0b0 2023-10-11T06:13:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3482817456 0 0) 0x7fea94003520 con 0x7feaa815a0b0 2023-10-11T06:13:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 msgr2=0x7feaa806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 0x7feaa806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 >> v2:172.21.15.99:3300/0 conn(0x7feaa80ced30 msgr2=0x7feaa80cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 --2- 172.21.15.84:0/1821401409 >> v2:172.21.15.99:3300/0 conn(0x7feaa80ced30 0x7feaa80cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa806de10 con 0x7feaa815a0b0 2023-10-11T06:13:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fea98001670 con 0x7feaa815a0b0 2023-10-11T06:13:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 >> v1:172.21.15.84:6789/0 conn(0x7feaa815a0b0 legacy=0x7feaa80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea940039a0 0x7fea94003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea940047c0 0x7fea94004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 0x7fea94005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa806e0f0 con 0x7fea940047c0 2023-10-11T06:13:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80cbae0 con 0x7fea940039a0 2023-10-11T06:13:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9400ba60 con 0x7feaa80c8d00 2023-10-11T06:13:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.998+0000 7fea9ffff640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea940047c0 0x7fea94004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7feaa7fff640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea940039a0 0x7fea94003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7feaac98d640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 0x7fea94005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7fea9ffff640 1 -- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 msgr2=0x7fea94005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7fea9ffff640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feaa80c8d00 0x7fea94005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7fea9ffff640 1 -- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea940039a0 msgr2=0x7fea94003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7fea9ffff640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea940039a0 0x7fea94003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7fea9ffff640 1 -- 172.21.15.84:0/1821401409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94003520 con 0x7fea940047c0 2023-10-11T06:13:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:11.999+0000 7feaa7fff640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea940039a0 0x7fea94003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.000+0000 7fea9ffff640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea940047c0 0x7fea94004c30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feaa80cf120 tx=0x7fea940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ef970a5d8f726b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.000+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9002c680 con 0x7fea940047c0 2023-10-11T06:13:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.000+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9002c7e0 con 0x7fea940047c0 2023-10-11T06:13:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.000+0000 7feaa77fe640 1 -- 172.21.15.84:0/1821401409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9002cb90 con 0x7fea940047c0 2023-10-11T06:13:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.001+0000 7feaae3f0640 1 -- 172.21.15.84:0/1821401409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea940047c0 msgr2=0x7fea94004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.001+0000 7feaae3f0640 1 --2- 172.21.15.84:0/1821401409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea940047c0 0x7fea94004c30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feaa80cf120 tx=0x7fea940057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.001+0000 7feaac98d640 1 -- 172.21.15.84:0/1821401409 reap_dead start 2023-10-11T06:13:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.001+0000 7feaae3f0640 1 -- 172.21.15.84:0/1821401409 shutdown_connections 2023-10-11T06:13:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.001+0000 7feaae3f0640 1 -- 172.21.15.84:0/1821401409 >> 172.21.15.84:0/1821401409 conn(0x7feaa8069c70 msgr2=0x7feaa806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.001+0000 7feaae3f0640 1 -- 172.21.15.84:0/1821401409 shutdown_connections 2023-10-11T06:13:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.001+0000 7feaae3f0640 1 -- 172.21.15.84:0/1821401409 wait complete. 2023-10-11T06:13:12.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.003+0000 7feaae3f0640 1 Processor -- start 2023-10-11T06:13:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.003+0000 7feaae3f0640 1 -- start start 2023-10-11T06:13:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7feaae3f0640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fea940047c0 0x7feaa81f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7feaae3f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa81f4b70 0x7feaa81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7feaae3f0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaa81548f0 0x7feaa81f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7feaae3f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80b2790 con 0x7feaa81f4b70 2023-10-11T06:13:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7feaae3f0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81f4fe0 con 0x7feaa81548f0 2023-10-11T06:13:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7feaae3f0640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa8154d60 con 0x7fea940047c0 2023-10-11T06:13:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa81f4b70 0x7feaa81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7fea9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa81f4b70 0x7feaa81543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60350/0 (socket says 172.21.15.84:60350) 2023-10-11T06:13:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.004+0000 7fea9ffff640 1 -- 172.21.15.84:0/2538913978 learned_addr learned my addr 172.21.15.84:0/2538913978 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7fea9ffff640 1 -- 172.21.15.84:0/2538913978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fea940047c0 msgr2=0x7feaa81f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7fea9ffff640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fea940047c0 0x7feaa81f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7fea9ffff640 1 -- 172.21.15.84:0/2538913978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaa81548f0 msgr2=0x7feaa81f1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7fea9ffff640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaa81548f0 0x7feaa81f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7fea9ffff640 1 -- 172.21.15.84:0/2538913978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa80b20f0 con 0x7feaa81f4b70 2023-10-11T06:13:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7fea9ffff640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa81f4b70 0x7feaa81543b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fea90009b80 tx=0x7fea90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7feaa57fa640 1 -- 172.21.15.84:0/2538913978 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90034a10 con 0x7feaa81f4b70 2023-10-11T06:13:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7feaa57fa640 1 -- 172.21.15.84:0/2538913978 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea90034b70 con 0x7feaa81f4b70 2023-10-11T06:13:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7feaa57fa640 1 -- 172.21.15.84:0/2538913978 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9003e400 con 0x7feaa81f4b70 2023-10-11T06:13:12.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa81f2960 con 0x7feaa81f4b70 2023-10-11T06:13:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.005+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa81f2100 con 0x7feaa81f4b70 2023-10-11T06:13:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.006+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa814c060 con 0x7feaa81f4b70 2023-10-11T06:13:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.007+0000 7feaa57fa640 1 -- 172.21.15.84:0/2538913978 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fea9003e560 con 0x7feaa81f4b70 2023-10-11T06:13:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.008+0000 7feaa57fa640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fea8806a860 0x7fea8806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.008+0000 7feaa57fa640 1 -- 172.21.15.84:0/2538913978 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fea900ab0d0 con 0x7feaa81f4b70 2023-10-11T06:13:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.008+0000 7feaa7fff640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fea8806a860 0x7fea8806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.009+0000 7feaa7fff640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fea8806a860 0x7fea8806cd10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fea980018e0 tx=0x7fea98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.012+0000 7feaa57fa640 1 -- 172.21.15.84:0/2538913978 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fea900761b0 con 0x7feaa81f4b70 2023-10-11T06:13:12.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.193+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7feaa8073ed0 con 0x7feaa81f4b70 2023-10-11T06:13:12.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.194+0000 7feaa57fa640 1 -- 172.21.15.84:0/2538913978 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) v1 ==== 97+0+11 (secure 0 0 0) 0x7fea900797d0 con 0x7feaa81f4b70 2023-10-11T06:13:12.196 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:12.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fea8806a860 msgr2=0x7fea8806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fea8806a860 0x7fea8806cd10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fea980018e0 tx=0x7fea98009040 comp rx=0 tx=0).stop 2023-10-11T06:13:12.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa81f4b70 msgr2=0x7feaa81543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa81f4b70 0x7feaa81543b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fea90009b80 tx=0x7fea90004730 comp rx=0 tx=0).stop 2023-10-11T06:13:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 shutdown_connections 2023-10-11T06:13:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fea8806a860 0x7fea8806cd10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaa81548f0 0x7feaa81f1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa81f4b70 0x7feaa81543b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 --2- 172.21.15.84:0/2538913978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fea940047c0 0x7feaa81f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.196+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 >> 172.21.15.84:0/2538913978 conn(0x7feaa8069c70 msgr2=0x7feaa8150430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:12.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.197+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 shutdown_connections 2023-10-11T06:13:12.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.197+0000 7feaae3f0640 1 -- 172.21.15.84:0/2538913978 wait complete. 2023-10-11T06:13:12.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:13:12.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.427+0000 7f29621f6640 1 Processor -- start 2023-10-11T06:13:12.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.427+0000 7f29621f6640 1 -- start start 2023-10-11T06:13:12.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.428+0000 7f29621f6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.428+0000 7f29621f6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f295c0c9360 0x7f295c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.428+0000 7f29621f6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295c071a90 con 0x7f295c0ca300 2023-10-11T06:13:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.428+0000 7f29621f6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295c072310 con 0x7f295c0c9360 2023-10-11T06:13:12.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.428+0000 7f29621f6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c070280 con 0x7f295c15d4f0 2023-10-11T06:13:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.429+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.429+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:60878/0 (socket says 172.21.15.84:60878) 2023-10-11T06:13:12.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.429+0000 7f295b7fe640 1 -- 172.21.15.84:0/1253338103 learned_addr learned my addr 172.21.15.84:0/1253338103 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:12.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.429+0000 7f295bfff640 1 --2- 172.21.15.84:0/1253338103 >> v2:172.21.15.99:3300/0 conn(0x7f295c0c9360 0x7f295c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.430+0000 7f295b7fe640 1 -- 172.21.15.84:0/1253338103 >> v2:172.21.15.99:3300/0 conn(0x7f295c0c9360 msgr2=0x7f295c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:12.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.430+0000 7f295b7fe640 1 --2- 172.21.15.84:0/1253338103 >> v2:172.21.15.99:3300/0 conn(0x7f295c0c9360 0x7f295c0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.430+0000 7f295b7fe640 1 -- 172.21.15.84:0/1253338103 >> v1:172.21.15.99:6789/0 conn(0x7f295c0ca300 legacy=0x7f295c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:12.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.430+0000 7f295b7fe640 1 -- 172.21.15.84:0/1253338103 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c071cc0 con 0x7f295c15d4f0 2023-10-11T06:13:12.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.430+0000 7f295bfff640 1 --2- 172.21.15.84:0/1253338103 >> v2:172.21.15.99:3300/0 conn(0x7f295c0c9360 0x7f295c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:13:12.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.431+0000 7f295b7fe640 1 --2- 172.21.15.84:0/1253338103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0cb540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f294c009ea0 tx=0x7f294c02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05a609e9993bafd server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.431+0000 7f295a7fc640 1 -- 172.21.15.84:0/1253338103 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c02cac0 con 0x7f295c15d4f0 2023-10-11T06:13:12.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.431+0000 7f295a7fc640 1 -- 172.21.15.84:0/1253338103 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c02cc20 con 0x7f295c15d4f0 2023-10-11T06:13:12.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.431+0000 7f295a7fc640 1 -- 172.21.15.84:0/1253338103 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0343f0 con 0x7f295c15d4f0 2023-10-11T06:13:12.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 -- 172.21.15.84:0/1253338103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 msgr2=0x7f295c0cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 --2- 172.21.15.84:0/1253338103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0cb540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f294c009ea0 tx=0x7f294c02bf20 comp rx=0 tx=0).stop 2023-10-11T06:13:12.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 -- 172.21.15.84:0/1253338103 shutdown_connections 2023-10-11T06:13:12.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 --2- 172.21.15.84:0/1253338103 >> v2:172.21.15.99:3300/0 conn(0x7f295c0c9360 0x7f295c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 --2- 172.21.15.84:0/1253338103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0cb540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 -- 172.21.15.84:0/1253338103 >> 172.21.15.84:0/1253338103 conn(0x7f295c069c70 msgr2=0x7f295c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:12.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 -- 172.21.15.84:0/1253338103 shutdown_connections 2023-10-11T06:13:12.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.432+0000 7f29621f6640 1 -- 172.21.15.84:0/1253338103 wait complete. 2023-10-11T06:13:12.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.435+0000 7f29621f6640 1 Processor -- start 2023-10-11T06:13:12.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.435+0000 7f29621f6640 1 -- start start 2023-10-11T06:13:12.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f29621f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0c9360 0x7f295c1efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f29621f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f295c0ca300 0x7f295c0a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f29621f6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f29621f6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0a8f40 con 0x7f295c0c9360 2023-10-11T06:13:12.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f29621f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1f01a0 con 0x7f295c0ca300 2023-10-11T06:13:12.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f29621f6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c14f580 con 0x7f295c15d4f0 2023-10-11T06:13:12.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0c9360 0x7f295c1efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0c9360 0x7f295c1efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60352/0 (socket says 172.21.15.84:60352) 2023-10-11T06:13:12.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.436+0000 7f295b7fe640 1 -- 172.21.15.84:0/3394255791 learned_addr learned my addr 172.21.15.84:0/3394255791 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:12.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f295b7fe640 1 -- 172.21.15.84:0/3394255791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 msgr2=0x7f295c0a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f295b7fe640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f295b7fe640 1 -- 172.21.15.84:0/3394255791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f295c0ca300 msgr2=0x7f295c0a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f295b7fe640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f295c0ca300 0x7f295c0a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f295b7fe640 1 -- 172.21.15.84:0/3394255791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954009400 con 0x7f295c0c9360 2023-10-11T06:13:12.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f295b7fe640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0c9360 0x7f295c1efc60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f294c009ea0 tx=0x7f294c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f2943fff640 1 -- 172.21.15.84:0/3394255791 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c040670 con 0x7f295c0c9360 2023-10-11T06:13:12.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f2943fff640 1 -- 172.21.15.84:0/3394255791 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c0407d0 con 0x7f295c0c9360 2023-10-11T06:13:12.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c009b30 con 0x7f295c0c9360 2023-10-11T06:13:12.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.437+0000 7f2943fff640 1 -- 172.21.15.84:0/3394255791 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c040b80 con 0x7f295c0c9360 2023-10-11T06:13:12.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.438+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c0c6cc0 con 0x7f295c0c9360 2023-10-11T06:13:12.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.439+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c073ed0 con 0x7f295c0c9360 2023-10-11T06:13:12.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.440+0000 7f2943fff640 1 -- 172.21.15.84:0/3394255791 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f294c040ce0 con 0x7f295c0c9360 2023-10-11T06:13:12.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.441+0000 7f2943fff640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f294806a830 0x7f294806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.441+0000 7f2943fff640 1 -- 172.21.15.84:0/3394255791 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f294c0ac2e0 con 0x7f295c0c9360 2023-10-11T06:13:12.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.441+0000 7f295affd640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f294806a830 0x7f294806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.442+0000 7f295affd640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f294806a830 0x7f294806cce0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2950006fd0 tx=0x7f2950006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.445+0000 7f2943fff640 1 -- 172.21.15.84:0/3394255791 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f294c077440 con 0x7f295c0c9360 2023-10-11T06:13:12.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.650+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f295c0b09b0 con 0x7f295c0c9360 2023-10-11T06:13:12.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.651+0000 7f2943fff640 1 -- 172.21.15.84:0/3394255791 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f294c009510 con 0x7f295c0c9360 2023-10-11T06:13:12.653 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:13:12.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.653+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f294806a830 msgr2=0x7f294806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.653+0000 7f29621f6640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f294806a830 0x7f294806cce0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2950006fd0 tx=0x7f2950006bc0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.653+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0c9360 msgr2=0x7f295c1efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.653+0000 7f29621f6640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0c9360 0x7f295c1efc60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f294c009ea0 tx=0x7f294c004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:12.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 shutdown_connections 2023-10-11T06:13:12.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f294806a830 0x7f294806cce0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f295c15d4f0 0x7f295c0a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f295c0ca300 0x7f295c0a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 --2- 172.21.15.84:0/3394255791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c0c9360 0x7f295c1efc60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 >> 172.21.15.84:0/3394255791 conn(0x7f295c069c70 msgr2=0x7f295c0c9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:12.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 shutdown_connections 2023-10-11T06:13:12.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.654+0000 7f29621f6640 1 -- 172.21.15.84:0/3394255791 wait complete. 2023-10-11T06:13:12.673 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:12.673 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:12.725 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:12.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.983+0000 7f6a17d9c640 1 Processor -- start 2023-10-11T06:13:12.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.983+0000 7f6a17d9c640 1 -- start start 2023-10-11T06:13:12.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.984+0000 7f6a17d9c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6a100ce0d0 0x7f6a100c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a17d9c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a100adb90 0x7f6a100caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a17d9c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a101593f0 con 0x7f6a100cd300 2023-10-11T06:13:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a17d9c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1015a340 con 0x7f6a100adb90 2023-10-11T06:13:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a17d9c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a100ce4a0 con 0x7f6a100ce0d0 2023-10-11T06:13:12.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a15337640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a100adb90 0x7f6a100caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a16339640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6a100cd300 0x7f6a100b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43610/0 (socket says 172.21.15.84:43610) 2023-10-11T06:13:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a16339640 1 -- 172.21.15.84:0/2285319789 learned_addr learned my addr 172.21.15.84:0/2285319789 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a15b38640 1 --2- 172.21.15.84:0/2285319789 >> v2:172.21.15.99:3300/0 conn(0x7f6a100ce0d0 0x7f6a100c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a14b36640 1 -- 172.21.15.84:0/2285319789 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122853194 0 0) 0x7f6a101593f0 con 0x7f6a100cd300 2023-10-11T06:13:12.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a14b36640 1 -- 172.21.15.84:0/2285319789 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69fc003520 con 0x7f6a100cd300 2023-10-11T06:13:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a15337640 1 -- 172.21.15.84:0/2285319789 >> v2:172.21.15.99:3300/0 conn(0x7f6a100ce0d0 msgr2=0x7f6a100c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a15337640 1 --2- 172.21.15.84:0/2285319789 >> v2:172.21.15.99:3300/0 conn(0x7f6a100ce0d0 0x7f6a100c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a15337640 1 -- 172.21.15.84:0/2285319789 >> v1:172.21.15.84:6789/0 conn(0x7f6a100cd300 legacy=0x7f6a100b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.985+0000 7f6a15337640 1 -- 172.21.15.84:0/2285319789 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a101596c0 con 0x7f6a100adb90 2023-10-11T06:13:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.987+0000 7f6a15337640 1 --2- 172.21.15.84:0/2285319789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a100caee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a100cc8c0 tx=0x7f6a0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acba89566235b4df server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.987+0000 7f6a14b36640 1 -- 172.21.15.84:0/2285319789 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00031830 con 0x7f6a100adb90 2023-10-11T06:13:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.987+0000 7f6a14b36640 1 -- 172.21.15.84:0/2285319789 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00031990 con 0x7f6a100adb90 2023-10-11T06:13:12.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.987+0000 7f6a14b36640 1 -- 172.21.15.84:0/2285319789 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00031d20 con 0x7f6a100adb90 2023-10-11T06:13:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.987+0000 7f6a17d9c640 1 -- 172.21.15.84:0/2285319789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 msgr2=0x7f6a100caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.987+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/2285319789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a100caee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a100cc8c0 tx=0x7f6a0002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.988+0000 7f6a17d9c640 1 -- 172.21.15.84:0/2285319789 shutdown_connections 2023-10-11T06:13:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.988+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/2285319789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a100caee0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.988+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/2285319789 >> v2:172.21.15.99:3300/0 conn(0x7f6a100ce0d0 0x7f6a100c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.988+0000 7f6a17d9c640 1 -- 172.21.15.84:0/2285319789 >> 172.21.15.84:0/2285319789 conn(0x7f6a10069c70 msgr2=0x7f6a1006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.988+0000 7f6a17d9c640 1 -- 172.21.15.84:0/2285319789 shutdown_connections 2023-10-11T06:13:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.988+0000 7f6a17d9c640 1 -- 172.21.15.84:0/2285319789 wait complete. 2023-10-11T06:13:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.989+0000 7f6a17d9c640 1 Processor -- start 2023-10-11T06:13:12.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.989+0000 7f6a17d9c640 1 -- start start 2023-10-11T06:13:12.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.990+0000 7f6a17d9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a101571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.990+0000 7f6a17d9c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a100cd300 0x7f6a101f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.990+0000 7f6a17d9c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a100ce0d0 0x7f6a10150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.990+0000 7f6a17d9c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100c34b0 con 0x7f6a100adb90 2023-10-11T06:13:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.990+0000 7f6a17d9c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100c4d90 con 0x7f6a100ce0d0 2023-10-11T06:13:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.990+0000 7f6a17d9c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100af440 con 0x7f6a100cd300 2023-10-11T06:13:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a101571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:12.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a101571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60378/0 (socket says 172.21.15.84:60378) 2023-10-11T06:13:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 -- 172.21.15.84:0/422609970 learned_addr learned my addr 172.21.15.84:0/422609970 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 -- 172.21.15.84:0/422609970 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a100cd300 msgr2=0x7f6a101f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a100cd300 0x7f6a101f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 -- 172.21.15.84:0/422609970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a100ce0d0 msgr2=0x7f6a10150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a100ce0d0 0x7f6a10150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:12.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 -- 172.21.15.84:0/422609970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00009400 con 0x7f6a100adb90 2023-10-11T06:13:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a15b38640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a101571a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00d800 tx=0x7f6a0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.991+0000 7f6a067fc640 1 -- 172.21.15.84:0/422609970 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00b6c0 con 0x7f6a100adb90 2023-10-11T06:13:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.992+0000 7f6a067fc640 1 -- 172.21.15.84:0/422609970 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c00b820 con 0x7f6a100adb90 2023-10-11T06:13:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.992+0000 7f6a067fc640 1 -- 172.21.15.84:0/422609970 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00bbd0 con 0x7f6a100adb90 2023-10-11T06:13:12.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.992+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a100af720 con 0x7f6a100adb90 2023-10-11T06:13:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.992+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a100afce0 con 0x7f6a100adb90 2023-10-11T06:13:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.993+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a10159c60 con 0x7f6a100adb90 2023-10-11T06:13:12.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.994+0000 7f6a067fc640 1 -- 172.21.15.84:0/422609970 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6a0c00bd30 con 0x7f6a100adb90 2023-10-11T06:13:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.994+0000 7f6a067fc640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69f406a830 0x7f69f406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.995+0000 7f6a067fc640 1 -- 172.21.15.84:0/422609970 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f6a0c0896e0 con 0x7f6a100adb90 2023-10-11T06:13:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.995+0000 7f6a15337640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69f406a830 0x7f69f406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.995+0000 7f6a15337640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69f406a830 0x7f69f406cce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a00036000 tx=0x7f6a00030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:13.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:12.999+0000 7f6a067fc640 1 -- 172.21.15.84:0/422609970 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f6a10159c60 con 0x7f6a100adb90 2023-10-11T06:13:13.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.189+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a10159c30 con 0x7f6a100adb90 2023-10-11T06:13:13.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.190+0000 7f6a067fc640 1 -- 172.21.15.84:0/422609970 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7f6a0c0547a0 con 0x7f6a100adb90 2023-10-11T06:13:13.192 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:13.193 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69f406a830 msgr2=0x7f69f406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69f406a830 0x7f69f406cce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a00036000 tx=0x7f6a00030000 comp rx=0 tx=0).stop 2023-10-11T06:13:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 msgr2=0x7f6a101571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a101571a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00d800 tx=0x7f6a0c00dcc0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 shutdown_connections 2023-10-11T06:13:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f69f406a830 0x7f69f406cce0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a100ce0d0 0x7f6a10150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a100cd300 0x7f6a101f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 --2- 172.21.15.84:0/422609970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a100adb90 0x7f6a101571a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 >> 172.21.15.84:0/422609970 conn(0x7f6a10069c70 msgr2=0x7f6a1006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:13.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 shutdown_connections 2023-10-11T06:13:13.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.193+0000 7f6a17d9c640 1 -- 172.21.15.84:0/422609970 wait complete. 2023-10-11T06:13:13.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:12:38.260176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:12:39.011036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-11T06:12:50.853138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:13:13.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:13:13.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.431+0000 7f21dab01640 1 Processor -- start 2023-10-11T06:13:13.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.431+0000 7f21dab01640 1 -- start start 2023-10-11T06:13:13.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21dab01640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d415a0b0 0x7f21d40cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21dab01640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21d40ced30 0x7f21d406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21dab01640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d4071380 con 0x7f21d40c8d00 2023-10-11T06:13:13.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21dab01640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d406e0f0 con 0x7f21d40ced30 2023-10-11T06:13:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21dab01640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40cbae0 con 0x7f21d415a0b0 2023-10-11T06:13:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21d909e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f21d40ced30 0x7f21d406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21d3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f21d40c8d00 0x7f21d40ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43616/0 (socket says 172.21.15.84:43616) 2023-10-11T06:13:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.433+0000 7f21d3fff640 1 -- 172.21.15.84:0/1084476208 learned_addr learned my addr 172.21.15.84:0/1084476208 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:13.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.434+0000 7f21d889d640 1 --2- 172.21.15.84:0/1084476208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d415a0b0 0x7f21d40cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.434+0000 7f21d909e640 1 -- 172.21.15.84:0/1084476208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d415a0b0 msgr2=0x7f21d40cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.434+0000 7f21d909e640 1 --2- 172.21.15.84:0/1084476208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d415a0b0 0x7f21d40cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.434+0000 7f21d909e640 1 -- 172.21.15.84:0/1084476208 >> v1:172.21.15.84:6789/0 conn(0x7f21d40c8d00 legacy=0x7f21d40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.434+0000 7f21d909e640 1 -- 172.21.15.84:0/1084476208 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d406de10 con 0x7f21d40ced30 2023-10-11T06:13:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.435+0000 7f21d909e640 1 --2- 172.21.15.84:0/1084476208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d40ced30 0x7f21d406c170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009dc0 tx=0x7f21cc02bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48272ffd68e3e863 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.435+0000 7f21d37fe640 1 -- 172.21.15.84:0/1084476208 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc02cb10 con 0x7f21d40ced30 2023-10-11T06:13:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.435+0000 7f21d37fe640 1 -- 172.21.15.84:0/1084476208 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc02cc70 con 0x7f21d40ced30 2023-10-11T06:13:13.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.435+0000 7f21d37fe640 1 -- 172.21.15.84:0/1084476208 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc034440 con 0x7f21d40ced30 2023-10-11T06:13:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 -- 172.21.15.84:0/1084476208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d40ced30 msgr2=0x7f21d406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 --2- 172.21.15.84:0/1084476208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d40ced30 0x7f21d406c170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009dc0 tx=0x7f21cc02bd60 comp rx=0 tx=0).stop 2023-10-11T06:13:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 -- 172.21.15.84:0/1084476208 shutdown_connections 2023-10-11T06:13:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 --2- 172.21.15.84:0/1084476208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d40ced30 0x7f21d406c170 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 --2- 172.21.15.84:0/1084476208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d415a0b0 0x7f21d40cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 -- 172.21.15.84:0/1084476208 >> 172.21.15.84:0/1084476208 conn(0x7f21d4069c70 msgr2=0x7f21d406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 -- 172.21.15.84:0/1084476208 shutdown_connections 2023-10-11T06:13:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.436+0000 7f21dab01640 1 -- 172.21.15.84:0/1084476208 wait complete. 2023-10-11T06:13:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.438+0000 7f21dab01640 1 Processor -- start 2023-10-11T06:13:13.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.438+0000 7f21dab01640 1 -- start start 2023-10-11T06:13:13.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21dab01640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21d40c8d00 0x7f21d415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21dab01640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d40ced30 0x7f21d40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21dab01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d415a0b0 0x7f21d40c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21dab01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d41566e0 con 0x7f21d415a0b0 2023-10-11T06:13:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21dab01640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d415c420 con 0x7f21d40c8d00 2023-10-11T06:13:13.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21dab01640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40b4480 con 0x7f21d40ced30 2023-10-11T06:13:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21d909e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d415a0b0 0x7f21d40c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21d909e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d415a0b0 0x7f21d40c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60384/0 (socket says 172.21.15.84:60384) 2023-10-11T06:13:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.439+0000 7f21d909e640 1 -- 172.21.15.84:0/5657867 learned_addr learned my addr 172.21.15.84:0/5657867 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:13.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d3fff640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d40ced30 0x7f21d40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d889d640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21d40c8d00 0x7f21d415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d909e640 1 -- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d40ced30 msgr2=0x7f21d40b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d909e640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d40ced30 0x7f21d40b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d909e640 1 -- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21d40c8d00 msgr2=0x7f21d415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d909e640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21d40c8d00 0x7f21d415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d909e640 1 -- 172.21.15.84:0/5657867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009a70 con 0x7f21d415a0b0 2023-10-11T06:13:13.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d3fff640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d40ced30 0x7f21d40b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:13.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d909e640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d415a0b0 0x7f21d40c4490 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009ee0 tx=0x7f21cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:13.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d17fa640 1 -- 172.21.15.84:0/5657867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc040650 con 0x7f21d415a0b0 2023-10-11T06:13:13.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d17fa640 1 -- 172.21.15.84:0/5657867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc0407b0 con 0x7f21d415a0b0 2023-10-11T06:13:13.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21d17fa640 1 -- 172.21.15.84:0/5657867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc040b60 con 0x7f21d415a0b0 2023-10-11T06:13:13.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.440+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d4155ad0 con 0x7f21d415a0b0 2023-10-11T06:13:13.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.441+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d4156090 con 0x7f21d415a0b0 2023-10-11T06:13:13.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.442+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4073ed0 con 0x7f21d415a0b0 2023-10-11T06:13:13.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.443+0000 7f21d17fa640 1 -- 172.21.15.84:0/5657867 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f21cc040cc0 con 0x7f21d415a0b0 2023-10-11T06:13:13.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.443+0000 7f21d17fa640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21b806a830 0x7f21b806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.444+0000 7f21d17fa640 1 -- 172.21.15.84:0/5657867 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f21cc0ac350 con 0x7f21d415a0b0 2023-10-11T06:13:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.444+0000 7f21d889d640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21b806a830 0x7f21b806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.444+0000 7f21d889d640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21b806a830 0x7f21b806cce0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f21c8006fd0 tx=0x7f21c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:13.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.448+0000 7f21d17fa640 1 -- 172.21.15.84:0/5657867 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f21cc077420 con 0x7f21d415a0b0 2023-10-11T06:13:13.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.635+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21d40c5540 con 0x7f21d415a0b0 2023-10-11T06:13:13.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.636+0000 7f21d17fa640 1 -- 172.21.15.84:0/5657867 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21cc07aa40 con 0x7f21d415a0b0 2023-10-11T06:13:13.637 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:13:13.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21b806a830 msgr2=0x7f21b806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21b806a830 0x7f21b806cce0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f21c8006fd0 tx=0x7f21c8008040 comp rx=0 tx=0).stop 2023-10-11T06:13:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d415a0b0 msgr2=0x7f21d40c4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d415a0b0 0x7f21d40c4490 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009ee0 tx=0x7f21cc004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 shutdown_connections 2023-10-11T06:13:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f21b806a830 0x7f21b806cce0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21d415a0b0 0x7f21d40c4490 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21d40ced30 0x7f21d40b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 --2- 172.21.15.84:0/5657867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21d40c8d00 0x7f21d415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 >> 172.21.15.84:0/5657867 conn(0x7f21d4069c70 msgr2=0x7f21d40c3600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:13.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 shutdown_connections 2023-10-11T06:13:13.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.639+0000 7f21dab01640 1 -- 172.21.15.84:0/5657867 wait complete. 2023-10-11T06:13:13.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:13:13.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.876+0000 7fbe40ce2640 1 Processor -- start 2023-10-11T06:13:13.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.876+0000 7fbe40ce2640 1 -- start start 2023-10-11T06:13:13.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe40ce2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe3c160830 0x7fbe3c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe40ce2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 0x7fbe3c0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe40ce2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0cc9e0 con 0x7fbe3c06dd00 2023-10-11T06:13:13.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe40ce2640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c06d720 con 0x7fbe3c160830 2023-10-11T06:13:13.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe40ce2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c06d180 con 0x7fbe3c0cf270 2023-10-11T06:13:13.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe3ad9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbe3c06dd00 0x7fbe3c164400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43618/0 (socket says 172.21.15.84:43618) 2023-10-11T06:13:13.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe3ad9d640 1 -- 172.21.15.84:0/757213809 learned_addr learned my addr 172.21.15.84:0/757213809 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe33fff640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 0x7fbe3c0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.878+0000 7fbe3a59c640 1 --2- 172.21.15.84:0/757213809 >> v2:172.21.15.99:3300/0 conn(0x7fbe3c160830 0x7fbe3c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744360638 0 0) 0x7fbe3c0cc9e0 con 0x7fbe3c06dd00 2023-10-11T06:13:13.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe28003520 con 0x7fbe3c06dd00 2023-10-11T06:13:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1025517222 0 0) 0x7fbe28003520 con 0x7fbe3c06dd00 2023-10-11T06:13:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 msgr2=0x7fbe3c0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 0x7fbe3c0c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 >> v2:172.21.15.99:3300/0 conn(0x7fbe3c160830 msgr2=0x7fbe3c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:13.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 --2- 172.21.15.84:0/757213809 >> v2:172.21.15.99:3300/0 conn(0x7fbe3c160830 0x7fbe3c0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c06d440 con 0x7fbe3c06dd00 2023-10-11T06:13:13.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fbe340019f0 con 0x7fbe3c06dd00 2023-10-11T06:13:13.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 >> v1:172.21.15.84:6789/0 conn(0x7fbe3c06dd00 legacy=0x7fbe3c164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe28003970 0x7fbe28003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 0x7fbe280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe28005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c06d720 con 0x7fbe28004d00 2023-10-11T06:13:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c06d180 con 0x7fbe28003970 2023-10-11T06:13:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.879+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2800b980 con 0x7fbe3c0cf270 2023-10-11T06:13:13.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3ad9d640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe28005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3a59c640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe28003970 0x7fbe28003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe33fff640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 0x7fbe280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3ad9d640 1 -- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 msgr2=0x7fbe280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3ad9d640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0cf270 0x7fbe280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3ad9d640 1 -- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe28003970 msgr2=0x7fbe28003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3ad9d640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe28003970 0x7fbe28003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3ad9d640 1 -- 172.21.15.84:0/757213809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28003520 con 0x7fbe28004d00 2023-10-11T06:13:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.880+0000 7fbe3a59c640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe28003970 0x7fbe28003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:13:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.881+0000 7fbe3ad9d640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe28005170 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400a860 tx=0x7fbe280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09264e3d3f44628 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34034630 con 0x7fbe28004d00 2023-10-11T06:13:13.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3402f070 con 0x7fbe28004d00 2023-10-11T06:13:13.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe39d9b640 1 -- 172.21.15.84:0/757213809 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340349f0 con 0x7fbe28004d00 2023-10-11T06:13:13.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe40ce2640 1 -- 172.21.15.84:0/757213809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 msgr2=0x7fbe28005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe40ce2640 1 --2- 172.21.15.84:0/757213809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe28005170 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400a860 tx=0x7fbe280057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe3ad9d640 1 -- 172.21.15.84:0/757213809 reap_dead start 2023-10-11T06:13:13.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe40ce2640 1 -- 172.21.15.84:0/757213809 shutdown_connections 2023-10-11T06:13:13.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.882+0000 7fbe40ce2640 1 -- 172.21.15.84:0/757213809 >> 172.21.15.84:0/757213809 conn(0x7fbe3c069c70 msgr2=0x7fbe3c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.883+0000 7fbe40ce2640 1 -- 172.21.15.84:0/757213809 shutdown_connections 2023-10-11T06:13:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.883+0000 7fbe40ce2640 1 -- 172.21.15.84:0/757213809 wait complete. 2023-10-11T06:13:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.884+0000 7fbe40ce2640 1 Processor -- start 2023-10-11T06:13:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.885+0000 7fbe40ce2640 1 -- start start 2023-10-11T06:13:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe40ce2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe3c1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe40ce2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe3c0c9280 0x7fbe3c15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe40ce2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0c96f0 0x7fbe3c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe40ce2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0c9070 con 0x7fbe28004d00 2023-10-11T06:13:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe40ce2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c148800 con 0x7fbe3c0c9280 2023-10-11T06:13:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe40ce2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c148ae0 con 0x7fbe3c0c96f0 2023-10-11T06:13:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe3a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe3c1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe3a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe3c1f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60404/0 (socket says 172.21.15.84:60404) 2023-10-11T06:13:13.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.886+0000 7fbe3a59c640 1 -- 172.21.15.84:0/686175696 learned_addr learned my addr 172.21.15.84:0/686175696 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:13.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe33fff640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe3c0c9280 0x7fbe3c15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe3ad9d640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0c96f0 0x7fbe3c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe3a59c640 1 -- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0c96f0 msgr2=0x7fbe3c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe3a59c640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0c96f0 0x7fbe3c15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe3a59c640 1 -- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe3c0c9280 msgr2=0x7fbe3c15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe3a59c640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe3c0c9280 0x7fbe3c15e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe3a59c640 1 -- 172.21.15.84:0/686175696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c148dc0 con 0x7fbe28004d00 2023-10-11T06:13:13.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe3a59c640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe3c1f59a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0116d0 tx=0x7fbe2c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe32ffd640 1 -- 172.21.15.84:0/686175696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00f9c0 con 0x7fbe28004d00 2023-10-11T06:13:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe32ffd640 1 -- 172.21.15.84:0/686175696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00fb20 con 0x7fbe28004d00 2023-10-11T06:13:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe32ffd640 1 -- 172.21.15.84:0/686175696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00fed0 con 0x7fbe28004d00 2023-10-11T06:13:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c1490a0 con 0x7fbe28004d00 2023-10-11T06:13:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.887+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c1483b0 con 0x7fbe28004d00 2023-10-11T06:13:13.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.888+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c146320 con 0x7fbe28004d00 2023-10-11T06:13:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.890+0000 7fbe32ffd640 1 -- 172.21.15.84:0/686175696 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbe2c015460 con 0x7fbe28004d00 2023-10-11T06:13:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.890+0000 7fbe32ffd640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe1c06a860 0x7fbe1c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.891+0000 7fbe32ffd640 1 -- 172.21.15.84:0/686175696 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fbe2c089b90 con 0x7fbe28004d00 2023-10-11T06:13:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.891+0000 7fbe33fff640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe1c06a860 0x7fbe1c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.891+0000 7fbe33fff640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe1c06a860 0x7fbe1c06cd10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c1461f0 tx=0x7fbe24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:13.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:13.895+0000 7fbe32ffd640 1 -- 172.21.15.84:0/686175696 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fbe2c054bc0 con 0x7fbe28004d00 2023-10-11T06:13:14.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.086+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbe3c073ed0 con 0x7fbe28004d00 2023-10-11T06:13:14.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.087+0000 7fbe32ffd640 1 -- 172.21.15.84:0/686175696 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbe2c0220a0 con 0x7fbe28004d00 2023-10-11T06:13:14.088 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:13:14.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe1c06a860 msgr2=0x7fbe1c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe1c06a860 0x7fbe1c06cd10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c1461f0 tx=0x7fbe24008040 comp rx=0 tx=0).stop 2023-10-11T06:13:14.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 msgr2=0x7fbe3c1f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe3c1f59a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0116d0 tx=0x7fbe2c011b90 comp rx=0 tx=0).stop 2023-10-11T06:13:14.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 shutdown_connections 2023-10-11T06:13:14.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe1c06a860 0x7fbe1c06cd10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe3c0c96f0 0x7fbe3c15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe3c0c9280 0x7fbe3c15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 --2- 172.21.15.84:0/686175696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe28004d00 0x7fbe3c1f59a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 >> 172.21.15.84:0/686175696 conn(0x7fbe3c069c70 msgr2=0x7fbe3c0c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 shutdown_connections 2023-10-11T06:13:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.090+0000 7fbe40ce2640 1 -- 172.21.15.84:0/686175696 wait complete. 2023-10-11T06:13:14.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:13:14.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.323+0000 7f9ce1d18640 1 Processor -- start 2023-10-11T06:13:14.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.323+0000 7f9ce1d18640 1 -- start start 2023-10-11T06:13:14.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.324+0000 7f9ce1d18640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9cdc150300 0x7f9cdc0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.324+0000 7f9ce1d18640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cdc0c9b90 con 0x7f9cdc0ce510 2023-10-11T06:13:14.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.324+0000 7f9ce1d18640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cdc15da10 con 0x7f9cdc14ff30 2023-10-11T06:13:14.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.324+0000 7f9ce1d18640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cdc0cc990 con 0x7f9cdc150300 2023-10-11T06:13:14.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.324+0000 7f9cd3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9cdc150300 0x7f9cdc0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.325+0000 7f9cdbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9cdc0ce510 0x7f9cdc0ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43624/0 (socket says 172.21.15.84:43624) 2023-10-11T06:13:14.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.325+0000 7f9cdbfff640 1 -- 172.21.15.84:0/2947788349 learned_addr learned my addr 172.21.15.84:0/2947788349 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:14.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.325+0000 7f9cd3fff640 1 -- 172.21.15.84:0/2947788349 >> v1:172.21.15.99:6789/0 conn(0x7f9cdc14ff30 legacy=0x7f9cdc0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:14.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.325+0000 7f9cd3fff640 1 -- 172.21.15.84:0/2947788349 >> v1:172.21.15.84:6789/0 conn(0x7f9cdc0ce510 legacy=0x7f9cdc0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.325+0000 7f9cd3fff640 1 -- 172.21.15.84:0/2947788349 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc15ce00 con 0x7f9cdc150300 2023-10-11T06:13:14.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.326+0000 7f9cd3fff640 1 --2- 172.21.15.84:0/2947788349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc150300 0x7f9cdc0c9650 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40097a0 tx=0x7f9cc402da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfb387a0e88867c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:14.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.326+0000 7f9cdaffd640 1 -- 172.21.15.84:0/2947788349 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4031870 con 0x7f9cdc150300 2023-10-11T06:13:14.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.326+0000 7f9cdaffd640 1 -- 172.21.15.84:0/2947788349 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc40319d0 con 0x7f9cdc150300 2023-10-11T06:13:14.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.326+0000 7f9cdaffd640 1 -- 172.21.15.84:0/2947788349 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4031d60 con 0x7f9cdc150300 2023-10-11T06:13:14.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.327+0000 7f9ce1d18640 1 -- 172.21.15.84:0/2947788349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc150300 msgr2=0x7f9cdc0c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.327+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/2947788349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc150300 0x7f9cdc0c9650 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40097a0 tx=0x7f9cc402da50 comp rx=0 tx=0).stop 2023-10-11T06:13:14.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.327+0000 7f9ce1d18640 1 -- 172.21.15.84:0/2947788349 shutdown_connections 2023-10-11T06:13:14.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.327+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/2947788349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc150300 0x7f9cdc0c9650 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.327+0000 7f9ce1d18640 1 -- 172.21.15.84:0/2947788349 >> 172.21.15.84:0/2947788349 conn(0x7f9cdc069c70 msgr2=0x7f9cdc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:14.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.327+0000 7f9ce1d18640 1 -- 172.21.15.84:0/2947788349 shutdown_connections 2023-10-11T06:13:14.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.327+0000 7f9ce1d18640 1 -- 172.21.15.84:0/2947788349 wait complete. 2023-10-11T06:13:14.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.329+0000 7f9ce1d18640 1 Processor -- start 2023-10-11T06:13:14.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.329+0000 7f9ce1d18640 1 -- start start 2023-10-11T06:13:14.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9ce1d18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cdc0ce510 0x7f9cdc14a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9ce1d18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc14ff30 0x7f9cdc0c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9ce1d18640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cdc150300 0x7f9cdc0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9ce1d18640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc150970 con 0x7f9cdc14ff30 2023-10-11T06:13:14.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9ce1d18640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc0c6c50 con 0x7f9cdc0ce510 2023-10-11T06:13:14.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9ce1d18640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc0c6f30 con 0x7f9cdc150300 2023-10-11T06:13:14.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc14ff30 0x7f9cdc0c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc14ff30 0x7f9cdc0c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60418/0 (socket says 172.21.15.84:60418) 2023-10-11T06:13:14.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.330+0000 7f9cd3fff640 1 -- 172.21.15.84:0/3501333675 learned_addr learned my addr 172.21.15.84:0/3501333675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:14.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cdbfff640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cdc150300 0x7f9cdc0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cdb7fe640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cdc0ce510 0x7f9cdc14a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd3fff640 1 -- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cdc150300 msgr2=0x7f9cdc0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd3fff640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cdc150300 0x7f9cdc0cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd3fff640 1 -- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cdc0ce510 msgr2=0x7f9cdc14a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd3fff640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cdc0ce510 0x7f9cdc14a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd3fff640 1 -- 172.21.15.84:0/3501333675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4009400 con 0x7f9cdc14ff30 2023-10-11T06:13:14.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd3fff640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc14ff30 0x7f9cdc0c6190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40098c0 tx=0x7f9cc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:14.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd8ff9640 1 -- 172.21.15.84:0/3501333675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc403e070 con 0x7f9cdc14ff30 2023-10-11T06:13:14.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd8ff9640 1 -- 172.21.15.84:0/3501333675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc40399c0 con 0x7f9cdc14ff30 2023-10-11T06:13:14.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9cd8ff9640 1 -- 172.21.15.84:0/3501333675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4039d70 con 0x7f9cdc14ff30 2023-10-11T06:13:14.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.331+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc0c7210 con 0x7f9cdc14ff30 2023-10-11T06:13:14.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.332+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc14a6b0 con 0x7f9cdc14ff30 2023-10-11T06:13:14.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.333+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cdc06ab90 con 0x7f9cdc14ff30 2023-10-11T06:13:14.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.334+0000 7f9cd8ff9640 1 -- 172.21.15.84:0/3501333675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9cc404b400 con 0x7f9cdc14ff30 2023-10-11T06:13:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.335+0000 7f9cd8ff9640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9cbc06a830 0x7f9cbc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.335+0000 7f9cd8ff9640 1 -- 172.21.15.84:0/3501333675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f9cc40ab610 con 0x7f9cdc14ff30 2023-10-11T06:13:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.335+0000 7f9cdb7fe640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9cbc06a830 0x7f9cbc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.336+0000 7f9cdb7fe640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9cbc06a830 0x7f9cbc06cce0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0015d0 tx=0x7f9ccc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:14.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.340+0000 7f9cd8ff9640 1 -- 172.21.15.84:0/3501333675 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9cc4076890 con 0x7f9cdc14ff30 2023-10-11T06:13:14.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.531+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9cdc073ed0 con 0x7f9cdc14ff30 2023-10-11T06:13:14.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.531+0000 7f9cd8ff9640 1 -- 172.21.15.84:0/3501333675 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9cc4079eb0 con 0x7f9cdc14ff30 2023-10-11T06:13:14.533 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:14.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9cbc06a830 msgr2=0x7f9cbc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9cbc06a830 0x7f9cbc06cce0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0015d0 tx=0x7f9ccc009040 comp rx=0 tx=0).stop 2023-10-11T06:13:14.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc14ff30 msgr2=0x7f9cdc0c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc14ff30 0x7f9cdc0c6190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40098c0 tx=0x7f9cc40046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 shutdown_connections 2023-10-11T06:13:14.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9cbc06a830 0x7f9cbc06cce0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cdc150300 0x7f9cdc0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cdc14ff30 0x7f9cdc0c6190 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 --2- 172.21.15.84:0/3501333675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cdc0ce510 0x7f9cdc14a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 >> 172.21.15.84:0/3501333675 conn(0x7f9cdc069c70 msgr2=0x7f9cdc14d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:14.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 shutdown_connections 2023-10-11T06:13:14.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.534+0000 7f9ce1d18640 1 -- 172.21.15.84:0/3501333675 wait complete. 2023-10-11T06:13:14.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:13:14.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.766+0000 7fb4662a7640 1 Processor -- start 2023-10-11T06:13:14.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.766+0000 7fb4662a7640 1 -- start start 2023-10-11T06:13:14.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb4662a7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb46015a0b0 0x7fb4600cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb4662a7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4600c8d00 0x7fb4600ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb4662a7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4600cefc0 con 0x7fb4600ceb60 2023-10-11T06:13:14.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb4662a7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46006e0f0 con 0x7fb46015a0b0 2023-10-11T06:13:14.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb4662a7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4600cbae0 con 0x7fb4600c8d00 2023-10-11T06:13:14.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb45ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb46015a0b0 0x7fb4600cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb464844640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb4600ceb60 0x7fb46006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43638/0 (socket says 172.21.15.84:43638) 2023-10-11T06:13:14.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.768+0000 7fb464844640 1 -- 172.21.15.84:0/3415370283 learned_addr learned my addr 172.21.15.84:0/3415370283 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:14.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.769+0000 7fb45f7fe640 1 --2- 172.21.15.84:0/3415370283 >> v2:172.21.15.99:3300/0 conn(0x7fb4600c8d00 0x7fb4600ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.769+0000 7fb45ffff640 1 -- 172.21.15.84:0/3415370283 >> v2:172.21.15.99:3300/0 conn(0x7fb4600c8d00 msgr2=0x7fb4600ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:14.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.769+0000 7fb45ffff640 1 --2- 172.21.15.84:0/3415370283 >> v2:172.21.15.99:3300/0 conn(0x7fb4600c8d00 0x7fb4600ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.769+0000 7fb45ffff640 1 -- 172.21.15.84:0/3415370283 >> v1:172.21.15.84:6789/0 conn(0x7fb4600ceb60 legacy=0x7fb46006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.769+0000 7fb45effd640 1 -- 172.21.15.84:0/3415370283 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772241363 0 0) 0x7fb4600cefc0 con 0x7fb4600ceb60 2023-10-11T06:13:14.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.769+0000 7fb45ffff640 1 -- 172.21.15.84:0/3415370283 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460071380 con 0x7fb46015a0b0 2023-10-11T06:13:14.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.770+0000 7fb45ffff640 1 --2- 172.21.15.84:0/3415370283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46015a0b0 0x7fb4600cb5a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b4880 tx=0x7fb45002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ed6b4e288e00a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:14.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.770+0000 7fb45effd640 1 -- 172.21.15.84:0/3415370283 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450031710 con 0x7fb46015a0b0 2023-10-11T06:13:14.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.770+0000 7fb45effd640 1 -- 172.21.15.84:0/3415370283 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45002f050 con 0x7fb46015a0b0 2023-10-11T06:13:14.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.770+0000 7fb45effd640 1 -- 172.21.15.84:0/3415370283 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450031a90 con 0x7fb46015a0b0 2023-10-11T06:13:14.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 -- 172.21.15.84:0/3415370283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46015a0b0 msgr2=0x7fb4600cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 --2- 172.21.15.84:0/3415370283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46015a0b0 0x7fb4600cb5a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b4880 tx=0x7fb45002b7b0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 -- 172.21.15.84:0/3415370283 shutdown_connections 2023-10-11T06:13:14.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 --2- 172.21.15.84:0/3415370283 >> v2:172.21.15.99:3300/0 conn(0x7fb4600c8d00 0x7fb4600ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 --2- 172.21.15.84:0/3415370283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46015a0b0 0x7fb4600cb5a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 -- 172.21.15.84:0/3415370283 >> 172.21.15.84:0/3415370283 conn(0x7fb460069c70 msgr2=0x7fb46006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:14.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 -- 172.21.15.84:0/3415370283 shutdown_connections 2023-10-11T06:13:14.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.771+0000 7fb4662a7640 1 -- 172.21.15.84:0/3415370283 wait complete. 2023-10-11T06:13:14.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.773+0000 7fb4662a7640 1 Processor -- start 2023-10-11T06:13:14.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.773+0000 7fb4662a7640 1 -- start start 2023-10-11T06:13:14.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb4662a7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4600c8d00 0x7fb46006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb4662a7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4600ceb60 0x7fb46006a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb4662a7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb46015a0b0 0x7fb46015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb4662a7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4601f3a50 con 0x7fb4600ceb60 2023-10-11T06:13:14.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb4662a7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4601f30b0 con 0x7fb4600c8d00 2023-10-11T06:13:14.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb4662a7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4601f3390 con 0x7fb46015a0b0 2023-10-11T06:13:14.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb45f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4600ceb60 0x7fb46006a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb45f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4600ceb60 0x7fb46006a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60426/0 (socket says 172.21.15.84:60426) 2023-10-11T06:13:14.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.774+0000 7fb45f7fe640 1 -- 172.21.15.84:0/1219609396 learned_addr learned my addr 172.21.15.84:0/1219609396 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:14.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45f7fe640 1 -- 172.21.15.84:0/1219609396 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb46015a0b0 msgr2=0x7fb46015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45f7fe640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb46015a0b0 0x7fb46015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45f7fe640 1 -- 172.21.15.84:0/1219609396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4600c8d00 msgr2=0x7fb46006a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45f7fe640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4600c8d00 0x7fb46006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45f7fe640 1 -- 172.21.15.84:0/1219609396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009400 con 0x7fb4600ceb60 2023-10-11T06:13:14.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45f7fe640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4600ceb60 0x7fb46006a580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb45400d800 tx=0x7fb45400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:14.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45cff9640 1 -- 172.21.15.84:0/1219609396 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400b6c0 con 0x7fb4600ceb60 2023-10-11T06:13:14.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45cff9640 1 -- 172.21.15.84:0/1219609396 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45400b820 con 0x7fb4600ceb60 2023-10-11T06:13:14.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb45cff9640 1 -- 172.21.15.84:0/1219609396 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400bbb0 con 0x7fb4600ceb60 2023-10-11T06:13:14.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4601f3670 con 0x7fb4600ceb60 2023-10-11T06:13:14.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.775+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4601f3e70 con 0x7fb4600ceb60 2023-10-11T06:13:14.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.777+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb460073ed0 con 0x7fb4600ceb60 2023-10-11T06:13:14.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.777+0000 7fb45cff9640 1 -- 172.21.15.84:0/1219609396 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb45400bd10 con 0x7fb4600ceb60 2023-10-11T06:13:14.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.779+0000 7fb45cff9640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:14.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.779+0000 7fb45cff9640 1 -- 172.21.15.84:0/1219609396 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fb4540896c0 con 0x7fb4600ceb60 2023-10-11T06:13:14.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.779+0000 7fb45ffff640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:14.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.780+0000 7fb45ffff640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb450035000 tx=0x7fb450030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:14.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.783+0000 7fb45cff9640 1 -- 172.21.15.84:0/1219609396 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb454054780 con 0x7fb4600ceb60 2023-10-11T06:13:14.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.973+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb460070760 con 0x7fb4600ceb60 2023-10-11T06:13:14.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.974+0000 7fb45cff9640 1 -- 172.21.15.84:0/1219609396 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb454057da0 con 0x7fb4600ceb60 2023-10-11T06:13:14.976 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:13:14.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.977+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 msgr2=0x7fb44c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.977+0000 7fb4662a7640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb450035000 tx=0x7fb450030000 comp rx=0 tx=0).stop 2023-10-11T06:13:14.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.977+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4600ceb60 msgr2=0x7fb46006a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.977+0000 7fb4662a7640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4600ceb60 0x7fb46006a580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb45400d800 tx=0x7fb45400dcc0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 shutdown_connections 2023-10-11T06:13:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb46015a0b0 0x7fb46015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4600ceb60 0x7fb46006a580 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 --2- 172.21.15.84:0/1219609396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4600c8d00 0x7fb46006a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:14.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 >> 172.21.15.84:0/1219609396 conn(0x7fb460069c70 msgr2=0x7fb4600c3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:14.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 shutdown_connections 2023-10-11T06:13:14.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:14.978+0000 7fb4662a7640 1 -- 172.21.15.84:0/1219609396 wait complete. 2023-10-11T06:13:14.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-11T06:13:15.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.211+0000 7f074823a640 1 Processor -- start 2023-10-11T06:13:15.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.212+0000 7f074823a640 1 -- start start 2023-10-11T06:13:15.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f074823a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f074015f7e0 0x7f07400cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f074823a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 0x7f07400d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f074823a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07400d3460 con 0x7f07400d3000 2023-10-11T06:13:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f074823a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07400cf220 con 0x7f074015f7e0 2023-10-11T06:13:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f074823a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07400cf500 con 0x7f07400cd6c0 2023-10-11T06:13:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f07467d7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f07400d3000 0x7f074015fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43644/0 (socket says 172.21.15.84:43644) 2023-10-11T06:13:15.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f07467d7640 1 -- 172.21.15.84:0/418799788 learned_addr learned my addr 172.21.15.84:0/418799788 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:15.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f0745fd6640 1 --2- 172.21.15.84:0/418799788 >> v2:172.21.15.99:3300/0 conn(0x7f074015f7e0 0x7f07400cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.213+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248183793 0 0) 0x7f07400d3460 con 0x7f07400d3000 2023-10-11T06:13:15.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0728003520 con 0x7f07400d3000 2023-10-11T06:13:15.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f07457d5640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 0x7f07400d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624736394 0 0) 0x7f0728003520 con 0x7f07400d3000 2023-10-11T06:13:15.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 msgr2=0x7f07400d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:15.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 0x7f07400d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 >> v2:172.21.15.99:3300/0 conn(0x7f074015f7e0 msgr2=0x7f07400cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:15.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 --2- 172.21.15.84:0/418799788 >> v2:172.21.15.99:3300/0 conn(0x7f074015f7e0 0x7f07400cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400cf7e0 con 0x7f07400d3000 2023-10-11T06:13:15.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f0738001670 con 0x7f07400d3000 2023-10-11T06:13:15.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 >> v1:172.21.15.84:6789/0 conn(0x7f07400d3000 legacy=0x7f074015fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 0x7f0728003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f07280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728004d00 0x7f0728005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400cf220 con 0x7f0728003f40 2023-10-11T06:13:15.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400cf500 con 0x7f0728004d00 2023-10-11T06:13:15.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072800ba00 con 0x7f07400cd6c0 2023-10-11T06:13:15.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.214+0000 7f07457d5640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f07280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f0745fd6640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 0x7f0728003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f07467d7640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728004d00 0x7f0728005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f07457d5640 1 -- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 msgr2=0x7f0728003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:15.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f07457d5640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400cd6c0 0x7f0728003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f07457d5640 1 -- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728004d00 msgr2=0x7f0728005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:15.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f07457d5640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728004d00 0x7f0728005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f07457d5640 1 -- 172.21.15.84:0/418799788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728003520 con 0x7f0728003f40 2023-10-11T06:13:15.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.215+0000 7f07467d7640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728004d00 0x7f0728005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:15.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.216+0000 7f07457d5640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f07280047c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f073400ba30 tx=0x7f07280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50dff1c7f9a1aa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:15.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.216+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734032640 con 0x7f0728003f40 2023-10-11T06:13:15.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.216+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07340327a0 con 0x7f0728003f40 2023-10-11T06:13:15.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.216+0000 7f0744fd4640 1 -- 172.21.15.84:0/418799788 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734032b30 con 0x7f0728003f40 2023-10-11T06:13:15.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.217+0000 7f074823a640 1 -- 172.21.15.84:0/418799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 msgr2=0x7f07280047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.217+0000 7f074823a640 1 --2- 172.21.15.84:0/418799788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f07280047c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f073400ba30 tx=0x7f07280057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.217+0000 7f07467d7640 1 -- 172.21.15.84:0/418799788 reap_dead start 2023-10-11T06:13:15.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.217+0000 7f074823a640 1 -- 172.21.15.84:0/418799788 shutdown_connections 2023-10-11T06:13:15.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.217+0000 7f074823a640 1 -- 172.21.15.84:0/418799788 >> 172.21.15.84:0/418799788 conn(0x7f07400713a0 msgr2=0x7f07400717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:15.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.217+0000 7f074823a640 1 -- 172.21.15.84:0/418799788 shutdown_connections 2023-10-11T06:13:15.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.217+0000 7f074823a640 1 -- 172.21.15.84:0/418799788 wait complete. 2023-10-11T06:13:15.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.219+0000 7f074823a640 1 Processor -- start 2023-10-11T06:13:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.219+0000 7f074823a640 1 -- start start 2023-10-11T06:13:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f074823a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f074014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f074823a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400c7e80 0x7f07400c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f074823a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07400c9dd0 0x7f07400ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f074823a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400c8810 con 0x7f0728003f40 2023-10-11T06:13:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f074823a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400cb6e0 con 0x7f07400c9dd0 2023-10-11T06:13:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f074823a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07400cab60 con 0x7f07400c7e80 2023-10-11T06:13:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f074014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f074014e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60432/0 (socket says 172.21.15.84:60432) 2023-10-11T06:13:15.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 -- 172.21.15.84:0/2653733532 learned_addr learned my addr 172.21.15.84:0/2653733532 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 -- 172.21.15.84:0/2653733532 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400c7e80 msgr2=0x7f07400c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400c7e80 0x7f07400c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 -- 172.21.15.84:0/2653733532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07400c9dd0 msgr2=0x7f07400ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07400c9dd0 0x7f07400ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 -- 172.21.15.84:0/2653733532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400cae40 con 0x7f0728003f40 2023-10-11T06:13:15.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.220+0000 7f0745fd6640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f074014e060 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f073000f3d0 tx=0x7f073000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:15.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.221+0000 7f073e7fc640 1 -- 172.21.15.84:0/2653733532 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730016620 con 0x7f0728003f40 2023-10-11T06:13:15.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.221+0000 7f073e7fc640 1 -- 172.21.15.84:0/2653733532 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07300167a0 con 0x7f0728003f40 2023-10-11T06:13:15.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.221+0000 7f073e7fc640 1 -- 172.21.15.84:0/2653733532 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730016b50 con 0x7f0728003f40 2023-10-11T06:13:15.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.221+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07400cb120 con 0x7f0728003f40 2023-10-11T06:13:15.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.221+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07400ca780 con 0x7f0728003f40 2023-10-11T06:13:15.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.222+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0740073ed0 con 0x7f0728003f40 2023-10-11T06:13:15.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.223+0000 7f073e7fc640 1 -- 172.21.15.84:0/2653733532 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0730016cb0 con 0x7f0728003f40 2023-10-11T06:13:15.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.224+0000 7f073e7fc640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f072c06a830 0x7f072c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.224+0000 7f073e7fc640 1 -- 172.21.15.84:0/2653733532 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f073008a200 con 0x7f0728003f40 2023-10-11T06:13:15.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.224+0000 7f07457d5640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f072c06a830 0x7f072c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.225+0000 7f07457d5640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f072c06a830 0x7f072c06cce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f073400ba00 tx=0x7f0734032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:15.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.228+0000 7f073e7fc640 1 -- 172.21.15.84:0/2653733532 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0730055240 con 0x7f0728003f40 2023-10-11T06:13:15.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.412+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f074015c230 con 0x7f0728003f40 2023-10-11T06:13:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.413+0000 7f073e7fc640 1 -- 172.21.15.84:0/2653733532 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) v1 ==== 97+0+11 (secure 0 0 0) 0x7f0730058860 con 0x7f0728003f40 2023-10-11T06:13:15.415 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f072c06a830 msgr2=0x7f072c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f072c06a830 0x7f072c06cce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f073400ba00 tx=0x7f0734032000 comp rx=0 tx=0).stop 2023-10-11T06:13:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 msgr2=0x7f074014e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f074014e060 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f073000f3d0 tx=0x7f073000f890 comp rx=0 tx=0).stop 2023-10-11T06:13:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 shutdown_connections 2023-10-11T06:13:15.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f072c06a830 0x7f072c06cce0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07400c9dd0 0x7f07400ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07400c7e80 0x7f07400c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 --2- 172.21.15.84:0/2653733532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0728003f40 0x7f074014e060 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 >> 172.21.15.84:0/2653733532 conn(0x7f07400713a0 msgr2=0x7f07400b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:15.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 shutdown_connections 2023-10-11T06:13:15.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.416+0000 7f074823a640 1 -- 172.21.15.84:0/2653733532 wait complete. 2023-10-11T06:13:15.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:13:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.634+0000 7fc9a1c84640 1 Processor -- start 2023-10-11T06:13:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.634+0000 7fc9a1c84640 1 -- start start 2023-10-11T06:13:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.636+0000 7fc9a1c84640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 0x7fc99c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.636+0000 7fc9a1c84640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c071a90 con 0x7fc99c15d4f0 2023-10-11T06:13:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.636+0000 7fc9a1c84640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c070280 con 0x7fc99c0c9360 2023-10-11T06:13:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.636+0000 7fc9a1c84640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c072310 con 0x7fc99c0ca300 2023-10-11T06:13:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.636+0000 7fc99b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc99c15d4f0 0x7fc99c0c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43660/0 (socket says 172.21.15.84:43660) 2023-10-11T06:13:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.636+0000 7fc99b7fe640 1 -- 172.21.15.84:0/1157143951 learned_addr learned my addr 172.21.15.84:0/1157143951 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:15.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014173726 0 0) 0x7fc99c071a90 con 0x7fc99c15d4f0 2023-10-11T06:13:15.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc993fff640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 0x7fc99c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc988003520 con 0x7fc99c15d4f0 2023-10-11T06:13:15.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2432359634 0 0) 0x7fc988003520 con 0x7fc99c15d4f0 2023-10-11T06:13:15.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 msgr2=0x7fc99c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:15.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 0x7fc99c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 >> v1:172.21.15.99:6789/0 conn(0x7fc99c0c9360 legacy=0x7fc99c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c071cc0 con 0x7fc99c15d4f0 2023-10-11T06:13:15.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fc98c0017f0 con 0x7fc99c15d4f0 2023-10-11T06:13:15.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 >> v1:172.21.15.84:6789/0 conn(0x7fc99c15d4f0 legacy=0x7fc99c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 0x7fc9880039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc988003fd0 0x7fc9880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc988004d00 0x7fc988005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c072310 con 0x7fc988004d00 2023-10-11T06:13:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800bb10 con 0x7fc988003fd0 2023-10-11T06:13:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800bcd0 con 0x7fc99c0ca300 2023-10-11T06:13:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.637+0000 7fc99bfff640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc988004d00 0x7fc988005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc993fff640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc988003fd0 0x7fc9880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc99b7fe640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 0x7fc9880039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc99bfff640 1 -- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 msgr2=0x7fc9880039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc99bfff640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 0x7fc9880039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc99bfff640 1 -- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc988003fd0 msgr2=0x7fc9880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc99bfff640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc988003fd0 0x7fc9880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc99bfff640 1 -- 172.21.15.84:0/1157143951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc988003520 con 0x7fc988004d00 2023-10-11T06:13:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc993fff640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc988003fd0 0x7fc9880047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.638+0000 7fc99b7fe640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc99c0ca300 0x7fc9880039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.639+0000 7fc99bfff640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc988004d00 0x7fc988005170 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc994001e50 tx=0x7fc9880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a03fbf22f1e170 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.639+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994003b70 con 0x7fc988004d00 2023-10-11T06:13:15.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.639+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994003cd0 con 0x7fc988004d00 2023-10-11T06:13:15.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.639+0000 7fc99affd640 1 -- 172.21.15.84:0/1157143951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994005310 con 0x7fc988004d00 2023-10-11T06:13:15.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.640+0000 7fc9a1c84640 1 -- 172.21.15.84:0/1157143951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc988004d00 msgr2=0x7fc988005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.640+0000 7fc9a1c84640 1 --2- 172.21.15.84:0/1157143951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc988004d00 0x7fc988005170 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc994001e50 tx=0x7fc9880057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.640+0000 7fc99bfff640 1 -- 172.21.15.84:0/1157143951 reap_dead start 2023-10-11T06:13:15.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.640+0000 7fc9a1c84640 1 -- 172.21.15.84:0/1157143951 shutdown_connections 2023-10-11T06:13:15.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.640+0000 7fc9a1c84640 1 -- 172.21.15.84:0/1157143951 >> 172.21.15.84:0/1157143951 conn(0x7fc99c069c70 msgr2=0x7fc99c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:15.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.640+0000 7fc9a1c84640 1 -- 172.21.15.84:0/1157143951 shutdown_connections 2023-10-11T06:13:15.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.640+0000 7fc9a1c84640 1 -- 172.21.15.84:0/1157143951 wait complete. 2023-10-11T06:13:15.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.641+0000 7fc9a1c84640 1 Processor -- start 2023-10-11T06:13:15.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.642+0000 7fc9a1c84640 1 -- start start 2023-10-11T06:13:15.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.642+0000 7fc9a1c84640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc988004d00 0x7fc99c155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.642+0000 7fc9a1c84640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1593f0 0x7fc99c0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.642+0000 7fc9a1c84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc99c0af180 0x7fc99c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc9a1c84640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0c69a0 con 0x7fc99c0af180 2023-10-11T06:13:15.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc9a1c84640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0c81b0 con 0x7fc99c1593f0 2023-10-11T06:13:15.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc9a1c84640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0c8490 con 0x7fc988004d00 2023-10-11T06:13:15.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc99c0af180 0x7fc99c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc99c0af180 0x7fc99c154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60452/0 (socket says 172.21.15.84:60452) 2023-10-11T06:13:15.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 -- 172.21.15.84:0/856064982 learned_addr learned my addr 172.21.15.84:0/856064982 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:15.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 -- 172.21.15.84:0/856064982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc988004d00 msgr2=0x7fc99c155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc988004d00 0x7fc99c155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 -- 172.21.15.84:0/856064982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1593f0 msgr2=0x7fc99c0c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1593f0 0x7fc99c0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 -- 172.21.15.84:0/856064982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0c6350 con 0x7fc99c0af180 2023-10-11T06:13:15.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.643+0000 7fc99bfff640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc99c0af180 0x7fc99c154c90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc994001f70 tx=0x7fc9940014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:15.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.644+0000 7fc998ff9640 1 -- 172.21.15.84:0/856064982 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99403e070 con 0x7fc99c0af180 2023-10-11T06:13:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.644+0000 7fc998ff9640 1 -- 172.21.15.84:0/856064982 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994003e40 con 0x7fc99c0af180 2023-10-11T06:13:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.644+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c0c69a0 con 0x7fc99c0af180 2023-10-11T06:13:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.644+0000 7fc998ff9640 1 -- 172.21.15.84:0/856064982 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994006d90 con 0x7fc99c0af180 2023-10-11T06:13:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.644+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c159860 con 0x7fc99c0af180 2023-10-11T06:13:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.645+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c073ed0 con 0x7fc99c0af180 2023-10-11T06:13:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.646+0000 7fc998ff9640 1 -- 172.21.15.84:0/856064982 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc994007030 con 0x7fc99c0af180 2023-10-11T06:13:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.647+0000 7fc998ff9640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc97c06a830 0x7fc97c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.647+0000 7fc998ff9640 1 -- 172.21.15.84:0/856064982 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fc9940aa460 con 0x7fc99c0af180 2023-10-11T06:13:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.647+0000 7fc99b7fe640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc97c06a830 0x7fc97c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.647+0000 7fc99b7fe640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc97c06a830 0x7fc97c06cce0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc98c001920 tx=0x7fc98c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.651+0000 7fc998ff9640 1 -- 172.21.15.84:0/856064982 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc994075520 con 0x7fc99c0af180 2023-10-11T06:13:15.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.839+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc99c0b09b0 con 0x7fc99c0af180 2023-10-11T06:13:15.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.840+0000 7fc998ff9640 1 -- 172.21.15.84:0/856064982 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc994078b40 con 0x7fc99c0af180 2023-10-11T06:13:15.841 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:13:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc97c06a830 msgr2=0x7fc97c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc97c06a830 0x7fc97c06cce0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc98c001920 tx=0x7fc98c009040 comp rx=0 tx=0).stop 2023-10-11T06:13:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc99c0af180 msgr2=0x7fc99c154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:15.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc99c0af180 0x7fc99c154c90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc994001f70 tx=0x7fc9940014a0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 shutdown_connections 2023-10-11T06:13:15.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc97c06a830 0x7fc97c06cce0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc99c0af180 0x7fc99c154c90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1593f0 0x7fc99c0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 --2- 172.21.15.84:0/856064982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc988004d00 0x7fc99c155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 >> 172.21.15.84:0/856064982 conn(0x7fc99c069c70 msgr2=0x7fc99c0c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 shutdown_connections 2023-10-11T06:13:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:15.843+0000 7fc9a1c84640 1 -- 172.21.15.84:0/856064982 wait complete. 2023-10-11T06:13:15.860 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:13:15.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:15.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:15.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[103332]: fuse finished with error 0 and tester_r 0 2023-10-11T06:13:15.950 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:13:16.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:13:16.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:16.028 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:13:16.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-10-11T06:13:16.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.309+0000 7f94988b6640 1 Processor -- start 2023-10-11T06:13:16.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.309+0000 7f94988b6640 1 -- start start 2023-10-11T06:13:16.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.310+0000 7f94988b6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94940d2ac0 0x7f94940d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f94988b6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d0940 0x7f94940d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f94988b6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94940cf220 con 0x7f949415f7e0 2023-10-11T06:13:16.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f94988b6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94940d3480 con 0x7f94940d2ac0 2023-10-11T06:13:16.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f94988b6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94940cf500 con 0x7f94940d0940 2023-10-11T06:13:16.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f9491d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94940d2ac0 0x7f94940d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f9491d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94940d2ac0 0x7f94940d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60466/0 (socket says 172.21.15.84:60466) 2023-10-11T06:13:16.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f9491d9b640 1 -- 172.21.15.84:0/4178813508 learned_addr learned my addr 172.21.15.84:0/4178813508 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:16.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f9491d9b640 1 -- 172.21.15.84:0/4178813508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d0940 msgr2=0x7f94940d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:16.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f9491d9b640 1 --2- 172.21.15.84:0/4178813508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d0940 0x7f94940d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f9491d9b640 1 -- 172.21.15.84:0/4178813508 >> v1:172.21.15.99:6789/0 conn(0x7f949415f7e0 legacy=0x7f94940cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:16.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.311+0000 7f9491d9b640 1 -- 172.21.15.84:0/4178813508 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940cf7e0 con 0x7f94940d2ac0 2023-10-11T06:13:16.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.312+0000 7f9491d9b640 1 --2- 172.21.15.84:0/4178813508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94940d2ac0 0x7f94940d2f40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9488009870 tx=0x7f948802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db848345437df440 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:16.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f949159a640 1 -- 172.21.15.84:0/4178813508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488031640 con 0x7f94940d2ac0 2023-10-11T06:13:16.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f949159a640 1 -- 172.21.15.84:0/4178813508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94880317a0 con 0x7f94940d2ac0 2023-10-11T06:13:16.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f949159a640 1 -- 172.21.15.84:0/4178813508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488031b30 con 0x7f94940d2ac0 2023-10-11T06:13:16.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f94988b6640 1 -- 172.21.15.84:0/4178813508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94940d2ac0 msgr2=0x7f94940d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f94988b6640 1 --2- 172.21.15.84:0/4178813508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94940d2ac0 0x7f94940d2f40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9488009870 tx=0x7f948802b8f0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f94988b6640 1 -- 172.21.15.84:0/4178813508 shutdown_connections 2023-10-11T06:13:16.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f94988b6640 1 --2- 172.21.15.84:0/4178813508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d0940 0x7f94940d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f94988b6640 1 --2- 172.21.15.84:0/4178813508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94940d2ac0 0x7f94940d2f40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f94988b6640 1 -- 172.21.15.84:0/4178813508 >> 172.21.15.84:0/4178813508 conn(0x7f94940713a0 msgr2=0x7f94940717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:16.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.313+0000 7f94988b6640 1 -- 172.21.15.84:0/4178813508 shutdown_connections 2023-10-11T06:13:16.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.314+0000 7f94988b6640 1 -- 172.21.15.84:0/4178813508 wait complete. 2023-10-11T06:13:16.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.315+0000 7f94988b6640 1 Processor -- start 2023-10-11T06:13:16.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.315+0000 7f94988b6640 1 -- start start 2023-10-11T06:13:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.316+0000 7f94988b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94940d0940 0x7f949414df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.316+0000 7f94988b6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d2ac0 0x7f94940b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.316+0000 7f94988b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949415f7e0 0x7f949415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.316+0000 7f94988b6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940b7fe0 con 0x7f949415f7e0 2023-10-11T06:13:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.316+0000 7f94988b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949415b220 con 0x7f94940d0940 2023-10-11T06:13:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.316+0000 7f94988b6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94940b8860 con 0x7f94940d2ac0 2023-10-11T06:13:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.316+0000 7f9492d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949415f7e0 0x7f949415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949415f7e0 0x7f949415ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60482/0 (socket says 172.21.15.84:60482) 2023-10-11T06:13:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 -- 172.21.15.84:0/1656176385 learned_addr learned my addr 172.21.15.84:0/1656176385 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:16.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 -- 172.21.15.84:0/1656176385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d2ac0 msgr2=0x7f94940b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d2ac0 0x7f94940b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 -- 172.21.15.84:0/1656176385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94940d0940 msgr2=0x7f949414df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94940d0940 0x7f949414df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 -- 172.21.15.84:0/1656176385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94880094d0 con 0x7f949415f7e0 2023-10-11T06:13:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9492d9d640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949415f7e0 0x7f949415ce60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f948c00b970 tx=0x7f948c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9482ffd640 1 -- 172.21.15.84:0/1656176385 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00cb40 con 0x7f949415f7e0 2023-10-11T06:13:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9482ffd640 1 -- 172.21.15.84:0/1656176385 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c00ccc0 con 0x7f949415f7e0 2023-10-11T06:13:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f9482ffd640 1 -- 172.21.15.84:0/1656176385 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c014680 con 0x7f949415f7e0 2023-10-11T06:13:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.317+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949415af40 con 0x7f949415f7e0 2023-10-11T06:13:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.318+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94940b84f0 con 0x7f949415f7e0 2023-10-11T06:13:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.319+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494073ed0 con 0x7f949415f7e0 2023-10-11T06:13:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.320+0000 7f9482ffd640 1 -- 172.21.15.84:0/1656176385 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f948c01d3b0 con 0x7f949415f7e0 2023-10-11T06:13:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.320+0000 7f9482ffd640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f947406a830 0x7f947406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.321+0000 7f9482ffd640 1 -- 172.21.15.84:0/1656176385 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f948c0890e0 con 0x7f949415f7e0 2023-10-11T06:13:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.321+0000 7f949259c640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f947406a830 0x7f947406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.321+0000 7f949259c640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f947406a830 0x7f947406cce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f94941609d0 tx=0x7f9484006ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:16.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.325+0000 7f9482ffd640 1 -- 172.21.15.84:0/1656176385 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f948c0541a0 con 0x7f949415f7e0 2023-10-11T06:13:16.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.581+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f94940b5030 con 0x7f949415f7e0 2023-10-11T06:13:16.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.588+0000 7f9482ffd640 1 -- 172.21.15.84:0/1656176385 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v11) v1 ==== 211+0+0 (secure 0 0 0) 0x7f948c019100 con 0x7f949415f7e0 2023-10-11T06:13:16.589 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:13:16.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f947406a830 msgr2=0x7f947406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f947406a830 0x7f947406cce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f94941609d0 tx=0x7f9484006ba0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949415f7e0 msgr2=0x7f949415ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949415f7e0 0x7f949415ce60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f948c00b970 tx=0x7f948c00be30 comp rx=0 tx=0).stop 2023-10-11T06:13:16.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 shutdown_connections 2023-10-11T06:13:16.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f947406a830 0x7f947406cce0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f949415f7e0 0x7f949415ce60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94940d2ac0 0x7f94940b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 --2- 172.21.15.84:0/1656176385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94940d0940 0x7f949414df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 >> 172.21.15.84:0/1656176385 conn(0x7f94940713a0 msgr2=0x7f94940d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:16.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 shutdown_connections 2023-10-11T06:13:16.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.591+0000 7f94988b6640 1 -- 172.21.15.84:0/1656176385 wait complete. 2023-10-11T06:13:16.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:13:16.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.824+0000 7f9151cec640 1 Processor -- start 2023-10-11T06:13:16.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.824+0000 7f9151cec640 1 -- start start 2023-10-11T06:13:16.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.826+0000 7f9151cec640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f914c0adb90 0x7f914c0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.826+0000 7f9151cec640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914c1593f0 con 0x7f914c0ce0d0 2023-10-11T06:13:16.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.826+0000 7f9151cec640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914c1596c0 con 0x7f914c0cd300 2023-10-11T06:13:16.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.826+0000 7f9151cec640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914c15a340 con 0x7f914c0adb90 2023-10-11T06:13:16.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.826+0000 7f914b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f914c0ce0d0 0x7f914c0c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43674/0 (socket says 172.21.15.84:43674) 2023-10-11T06:13:16.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.826+0000 7f914b7fe640 1 -- 172.21.15.84:0/3003275232 learned_addr learned my addr 172.21.15.84:0/3003275232 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:16.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f9143fff640 1 --2- 172.21.15.84:0/3003275232 >> v2:172.21.15.99:3300/0 conn(0x7f914c0adb90 0x7f914c0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056116141 0 0) 0x7f914c1593f0 con 0x7f914c0ce0d0 2023-10-11T06:13:16.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9138005730 con 0x7f914c0ce0d0 2023-10-11T06:13:16.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014140693 0 0) 0x7f9138005730 con 0x7f914c0ce0d0 2023-10-11T06:13:16.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 >> v2:172.21.15.99:3300/0 conn(0x7f914c0adb90 msgr2=0x7f914c0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:16.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 --2- 172.21.15.84:0/3003275232 >> v2:172.21.15.99:3300/0 conn(0x7f914c0adb90 0x7f914c0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 >> v1:172.21.15.99:6789/0 conn(0x7f914c0cd300 legacy=0x7f914c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c1599a0 con 0x7f914c0ce0d0 2023-10-11T06:13:16.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f913c0017f0 con 0x7f914c0ce0d0 2023-10-11T06:13:16.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 >> v1:172.21.15.84:6789/0 conn(0x7f914c0ce0d0 legacy=0x7f914c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9138005b80 0x7f9138005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f9138006e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9138007380 0x7f913800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c15a340 con 0x7f91380069d0 2023-10-11T06:13:16.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913800e200 con 0x7f9138007380 2023-10-11T06:13:16.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.827+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913800e3c0 con 0x7f9138005b80 2023-10-11T06:13:16.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f9143fff640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f9138006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f914bfff640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9138007380 0x7f913800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f9143fff640 1 -- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9138005b80 msgr2=0x7f9138005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:16.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f914b7fe640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9138005b80 0x7f9138005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f9143fff640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9138005b80 0x7f9138005f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f9143fff640 1 -- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9138007380 msgr2=0x7f913800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:16.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f9143fff640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9138007380 0x7f913800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f9143fff640 1 -- 172.21.15.84:0/3003275232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138005730 con 0x7f91380069d0 2023-10-11T06:13:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f914bfff640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9138007380 0x7f913800db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.828+0000 7f914bfff640 1 -- 172.21.15.84:0/3003275232 reap_dead start 2023-10-11T06:13:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.829+0000 7f9143fff640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f9138006e40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9134009ac0 tx=0x7f9138007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8df1cdffc48b9c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:16.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913402c6b0 con 0x7f91380069d0 2023-10-11T06:13:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913402c810 con 0x7f91380069d0 2023-10-11T06:13:16.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f914affd640 1 -- 172.21.15.84:0/3003275232 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913402cba0 con 0x7f91380069d0 2023-10-11T06:13:16.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f9151cec640 1 -- 172.21.15.84:0/3003275232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 msgr2=0x7f9138006e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f9151cec640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f9138006e40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9134009ac0 tx=0x7f9138007a00 comp rx=0 tx=0).stop 2023-10-11T06:13:16.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f9151cec640 1 -- 172.21.15.84:0/3003275232 shutdown_connections 2023-10-11T06:13:16.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f9151cec640 1 --2- 172.21.15.84:0/3003275232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f9138006e40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f9151cec640 1 -- 172.21.15.84:0/3003275232 >> 172.21.15.84:0/3003275232 conn(0x7f914c069c70 msgr2=0x7f914c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:16.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.830+0000 7f9151cec640 1 -- 172.21.15.84:0/3003275232 shutdown_connections 2023-10-11T06:13:16.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.831+0000 7f9151cec640 1 -- 172.21.15.84:0/3003275232 wait complete. 2023-10-11T06:13:16.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.832+0000 7f9151cec640 1 Processor -- start 2023-10-11T06:13:16.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.832+0000 7f9151cec640 1 -- start start 2023-10-11T06:13:16.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.833+0000 7f9151cec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f914c0c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.833+0000 7f9151cec640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f914c155ec0 0x7f914c0c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.833+0000 7f9151cec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f914c0ab950 0x7f914c0abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.833+0000 7f9151cec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c0ac300 con 0x7f91380069d0 2023-10-11T06:13:16.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.833+0000 7f9151cec640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c156330 con 0x7f914c0ab950 2023-10-11T06:13:16.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.833+0000 7f9151cec640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914c0c3080 con 0x7f914c155ec0 2023-10-11T06:13:16.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.833+0000 7f914b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f914c0c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f914c0c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60506/0 (socket says 172.21.15.84:60506) 2023-10-11T06:13:16.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 -- 172.21.15.84:0/2204361714 learned_addr learned my addr 172.21.15.84:0/2204361714 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:16.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 -- 172.21.15.84:0/2204361714 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f914c155ec0 msgr2=0x7f914c0c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f914c155ec0 0x7f914c0c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 -- 172.21.15.84:0/2204361714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f914c0ab950 msgr2=0x7f914c0abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:16.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f914c0ab950 0x7f914c0abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:16.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 -- 172.21.15.84:0/2204361714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c0c3ae0 con 0x7f91380069d0 2023-10-11T06:13:16.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f914b7fe640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f914c0c3450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f913c00d770 tx=0x7f913c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:16.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f9148ff9640 1 -- 172.21.15.84:0/2204361714 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c007480 con 0x7f91380069d0 2023-10-11T06:13:16.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f9148ff9640 1 -- 172.21.15.84:0/2204361714 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c0075e0 con 0x7f91380069d0 2023-10-11T06:13:16.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f9148ff9640 1 -- 172.21.15.84:0/2204361714 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0059c0 con 0x7f91380069d0 2023-10-11T06:13:16.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.834+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914c0c3dc0 con 0x7f91380069d0 2023-10-11T06:13:16.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.835+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c0c4380 con 0x7f91380069d0 2023-10-11T06:13:16.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.836+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914c073ed0 con 0x7f91380069d0 2023-10-11T06:13:16.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.836+0000 7f9148ff9640 1 -- 172.21.15.84:0/2204361714 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f913c005b20 con 0x7f91380069d0 2023-10-11T06:13:16.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.837+0000 7f9148ff9640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f912c06a860 0x7f912c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:16.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.837+0000 7f9148ff9640 1 -- 172.21.15.84:0/2204361714 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f913c089270 con 0x7f91380069d0 2023-10-11T06:13:16.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.838+0000 7f9143fff640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f912c06a860 0x7f912c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:16.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.838+0000 7f9143fff640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f912c06a860 0x7f912c06cd10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9134036000 tx=0x7f913402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:16.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:16.842+0000 7f9148ff9640 1 -- 172.21.15.84:0/2204361714 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f913c054400 con 0x7f91380069d0 2023-10-11T06:13:17.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.066+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f914c0ad350 con 0x7f91380069d0 2023-10-11T06:13:17.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.067+0000 7f9148ff9640 1 -- 172.21.15.84:0/2204361714 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+82 (secure 0 0 0) 0x7f913c057a20 con 0x7f91380069d0 2023-10-11T06:13:17.069 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-10-11T06:13:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f912c06a860 msgr2=0x7f912c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f912c06a860 0x7f912c06cd10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9134036000 tx=0x7f913402e000 comp rx=0 tx=0).stop 2023-10-11T06:13:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 msgr2=0x7f914c0c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f914c0c3450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f913c00d770 tx=0x7f913c00dc30 comp rx=0 tx=0).stop 2023-10-11T06:13:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 shutdown_connections 2023-10-11T06:13:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f912c06a860 0x7f912c06cd10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f914c0ab950 0x7f914c0abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f914c155ec0 0x7f914c0c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 --2- 172.21.15.84:0/2204361714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91380069d0 0x7f914c0c3450 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:17.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 >> 172.21.15.84:0/2204361714 conn(0x7f914c069c70 msgr2=0x7f914c158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:17.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 shutdown_connections 2023-10-11T06:13:17.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:17.070+0000 7f9151cec640 1 -- 172.21.15.84:0/2204361714 wait complete. 2023-10-11T06:13:17.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:13:17.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:13:17.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:13:17.091 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T06:13:17.091 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:17.091 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:13:17.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:13:17.092 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:13:17.092 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:17.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:17.093 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:13:17.143 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:13:17.143 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:13:17.143 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:13:17.144 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:17.144 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:13:17.144 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:17.144 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:17.144 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:17.144 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:13:17.145 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:17.145 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:17.145 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:13:17.145 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:17.145 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:17.146 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:13:17.146 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:17.146 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:13:17.146 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42280sec preferred_lft 42280sec 2023-10-11T06:13:17.146 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:13:17.146 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:17.147 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:13:17.147 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:17.147 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:13:17.147 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:17.147 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1c33:d5ff:fea1:21f9/64 scope link 2023-10-11T06:13:17.148 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:17.148 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:13:17.148 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:13:17.148 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::d4ff:bcff:feac:c798/64 scope link 2023-10-11T06:13:17.148 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:17.148 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:13:17.149 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:13:17.149 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4088:59ff:feb7:538e/64 scope link 2023-10-11T06:13:17.149 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:17.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:17.150 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:13:17.281 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:13:17.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:17.285 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:13:17.339 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:13:17.339 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:13:17.340 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:13:17.340 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:13:17.340 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:13:17.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:17.341 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:13:17.341 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:13:17.341 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:13:17.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:13:17.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:13:17.343 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:13:17.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:17.531 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:13:17.582 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T06:13:17.583 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:13:17.584 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:17.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:17.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-11T06:13:17.680 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T06:13:17.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:13:17.681 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:17.735 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:17.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:13:17.736 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:17.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:17.874 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:17.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:17.967 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:17.971 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:17.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:17.972 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:18.026 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:13:18.026 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T06:13:18.032 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:18.107 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:18.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:18.183 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:18.187 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:18.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:18.189 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:18.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-10-11T06:13:18.217+0000 7fabfe03f480 -1 init, newargv = 0x556e8f063940 newargc=15 2023-10-11T06:13:18.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[104533]: starting ceph client 2023-10-11T06:13:18.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[104533]: starting fuse 2023-10-11T06:13:18.233 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:13:18.233 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-10-11T06:13:18.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:13:18.234 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:13:18.234 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:13:18.234 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:13:18.235 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:13:18.235 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:18.235 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:13:18.236 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T06:13:18.236 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:13:18.236 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T06:13:18.237 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:18.237 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:13:18.237 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:13:18.238 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:13:18.238 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:13:18.238 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:18.239 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:13:18.239 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:13:18.239 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:13:18.240 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:13:18.240 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:13:18.241 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:13:18.241 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:13:18.241 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:13:18.242 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:18.242 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-10-11T06:13:18.242 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:13:18.437 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.104533.asok 2023-10-11T06:13:18.618 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.104533.asok 2023-10-11T06:13:18.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:18.619 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.104533.asok status 2023-10-11T06:13:18.863 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:13:18.863 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:13:18.863 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:13:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:13:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-10-11T06:13:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:13:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T06:13:18.864 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "104533", 2023-10-11T06:13:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:13:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:13:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:13:18.865 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5754, 2023-10-11T06:13:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:13:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:13:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:13:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 5754 2023-10-11T06:13:18.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:13:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:13:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:13:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3911694390 2023-10-11T06:13:18.867 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:13:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:13:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:13:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3911694390 2023-10-11T06:13:18.868 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.5754 192.168.0.1:0/3911694390", 2023-10-11T06:13:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/3911694390", 2023-10-11T06:13:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:13:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 16, 2023-10-11T06:13:18.869 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 41, 2023-10-11T06:13:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:13:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:13:18.870 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:13:18.870 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:13:18.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:18.877 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:13:18.877 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:18.931 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:13:18.931 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:18.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:18.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:19.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:19.029 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-10-11T06:13:19.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:19.094 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-10-11T06:13:19.162 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:19.162 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:19.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:19.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.429+0000 7f490602d640 1 Processor -- start 2023-10-11T06:13:19.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.429+0000 7f490602d640 1 -- start start 2023-10-11T06:13:19.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f490602d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49000cafb0 0x7f49000cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f490602d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f49000cc410 0x7f49000cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f490602d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000cb970 con 0x7f490015c290 2023-10-11T06:13:19.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f490602d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49000cad70 con 0x7f49000cafb0 2023-10-11T06:13:19.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f490602d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490015fe40 con 0x7f49000cc410 2023-10-11T06:13:19.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f48f6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49000cafb0 0x7f49000cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f48f6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49000cafb0 0x7f49000cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46000/0 (socket says 172.21.15.84:46000) 2023-10-11T06:13:19.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.431+0000 7f48f6ffd640 1 -- 172.21.15.84:0/2676572180 learned_addr learned my addr 172.21.15.84:0/2676572180 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:19.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.432+0000 7f48f6ffd640 1 -- 172.21.15.84:0/2676572180 >> v2:172.21.15.99:3300/0 conn(0x7f49000cc410 msgr2=0x7f49000cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:19.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.432+0000 7f48f6ffd640 1 --2- 172.21.15.84:0/2676572180 >> v2:172.21.15.99:3300/0 conn(0x7f49000cc410 0x7f49000cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.432+0000 7f48f6ffd640 1 -- 172.21.15.84:0/2676572180 >> v1:172.21.15.99:6789/0 conn(0x7f490015c290 legacy=0x7f49000c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:19.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.432+0000 7f48f6ffd640 1 -- 172.21.15.84:0/2676572180 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490006ff30 con 0x7f49000cafb0 2023-10-11T06:13:19.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.433+0000 7f48f6ffd640 1 --2- 172.21.15.84:0/2676572180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000cafb0 0x7f49000cb430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f490006fd30 tx=0x7f48e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c5a9a9cef66acc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:19.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.433+0000 7f48feffd640 1 -- 172.21.15.84:0/2676572180 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8031870 con 0x7f49000cafb0 2023-10-11T06:13:19.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.433+0000 7f48feffd640 1 -- 172.21.15.84:0/2676572180 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e80319d0 con 0x7f49000cafb0 2023-10-11T06:13:19.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.433+0000 7f48feffd640 1 -- 172.21.15.84:0/2676572180 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8031d60 con 0x7f49000cafb0 2023-10-11T06:13:19.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.433+0000 7f490602d640 1 -- 172.21.15.84:0/2676572180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000cafb0 msgr2=0x7f49000cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.433+0000 7f490602d640 1 --2- 172.21.15.84:0/2676572180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000cafb0 0x7f49000cb430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f490006fd30 tx=0x7f48e802b910 comp rx=0 tx=0).stop 2023-10-11T06:13:19.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.434+0000 7f490602d640 1 -- 172.21.15.84:0/2676572180 shutdown_connections 2023-10-11T06:13:19.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.434+0000 7f490602d640 1 --2- 172.21.15.84:0/2676572180 >> v2:172.21.15.99:3300/0 conn(0x7f49000cc410 0x7f49000cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.434+0000 7f490602d640 1 --2- 172.21.15.84:0/2676572180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49000cafb0 0x7f49000cb430 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.434+0000 7f490602d640 1 -- 172.21.15.84:0/2676572180 >> 172.21.15.84:0/2676572180 conn(0x7f4900069c70 msgr2=0x7f490006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:19.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.434+0000 7f490602d640 1 -- 172.21.15.84:0/2676572180 shutdown_connections 2023-10-11T06:13:19.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.434+0000 7f490602d640 1 -- 172.21.15.84:0/2676572180 wait complete. 2023-10-11T06:13:19.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.435+0000 7f490602d640 1 Processor -- start 2023-10-11T06:13:19.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.436+0000 7f490602d640 1 -- start start 2023-10-11T06:13:19.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.436+0000 7f490602d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49000cafb0 0x7f490015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f490602d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49000cc410 0x7f49000c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f490602d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490015c290 0x7f490014cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f490602d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490015e950 con 0x7f490015c290 2023-10-11T06:13:19.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f490602d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4900157380 con 0x7f49000cafb0 2023-10-11T06:13:19.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f490602d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000c26d0 con 0x7f49000cc410 2023-10-11T06:13:19.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490015c290 0x7f490014cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490015c290 0x7f490014cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46004/0 (socket says 172.21.15.84:46004) 2023-10-11T06:13:19.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 -- 172.21.15.84:0/2534774821 learned_addr learned my addr 172.21.15.84:0/2534774821 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:19.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48ff7fe640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49000cafb0 0x7f490015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48f6ffd640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49000cc410 0x7f49000c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 -- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49000cc410 msgr2=0x7f49000c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49000cc410 0x7f49000c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 -- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49000cafb0 msgr2=0x7f490015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49000cafb0 0x7f490015ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.437+0000 7f48fffff640 1 -- 172.21.15.84:0/2534774821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e80094d0 con 0x7f490015c290 2023-10-11T06:13:19.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.438+0000 7f48fffff640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490015c290 0x7f490014cad0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f48f800b970 tx=0x7f48f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:19.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.438+0000 7f48fcff9640 1 -- 172.21.15.84:0/2534774821 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f800cb40 con 0x7f490015c290 2023-10-11T06:13:19.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.438+0000 7f48fcff9640 1 -- 172.21.15.84:0/2534774821 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f800ccc0 con 0x7f490015c290 2023-10-11T06:13:19.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.438+0000 7f48fcff9640 1 -- 172.21.15.84:0/2534774821 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8014680 con 0x7f490015c290 2023-10-11T06:13:19.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.438+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49000c29b0 con 0x7f490015c290 2023-10-11T06:13:19.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.438+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49000c2f70 con 0x7f490015c290 2023-10-11T06:13:19.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.439+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900073ed0 con 0x7f490015c290 2023-10-11T06:13:19.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.440+0000 7f48fcff9640 1 -- 172.21.15.84:0/2534774821 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f48f801d3b0 con 0x7f490015c290 2023-10-11T06:13:19.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.441+0000 7f48fcff9640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f48e006a830 0x7f48e006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.441+0000 7f48fcff9640 1 -- 172.21.15.84:0/2534774821 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f48f8089180 con 0x7f490015c290 2023-10-11T06:13:19.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.442+0000 7f48ff7fe640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f48e006a830 0x7f48e006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.442+0000 7f48ff7fe640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f48e006a830 0x7f48e006cce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001740 tx=0x7f48f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:19.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.445+0000 7f48fcff9640 1 -- 172.21.15.84:0/2534774821 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f48f80541b0 con 0x7f490015c290 2023-10-11T06:13:19.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.638+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49000b4580 con 0x7f490015c290 2023-10-11T06:13:19.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.641+0000 7f48fcff9640 1 -- 172.21.15.84:0/2534774821 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7f48f80577d0 con 0x7f490015c290 2023-10-11T06:13:19.642 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:19.643 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:19.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f48e006a830 msgr2=0x7f48e006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f48e006a830 0x7f48e006cce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001740 tx=0x7f48f0009040 comp rx=0 tx=0).stop 2023-10-11T06:13:19.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490015c290 msgr2=0x7f490014cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490015c290 0x7f490014cad0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f48f800b970 tx=0x7f48f800be30 comp rx=0 tx=0).stop 2023-10-11T06:13:19.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 shutdown_connections 2023-10-11T06:13:19.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f48e006a830 0x7f48e006cce0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f490015c290 0x7f490014cad0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49000cc410 0x7f49000c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 --2- 172.21.15.84:0/2534774821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49000cafb0 0x7f490015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.643+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 >> 172.21.15.84:0/2534774821 conn(0x7f4900069c70 msgr2=0x7f49000ac170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:19.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.644+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 shutdown_connections 2023-10-11T06:13:19.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.644+0000 7f490602d640 1 -- 172.21.15.84:0/2534774821 wait complete. 2023-10-11T06:13:19.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:12:38.260176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:12:39.011036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-11T06:12:50.853138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:13:19.663 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:13:19.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.877+0000 7f0390d0a640 1 Processor -- start 2023-10-11T06:13:19.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.877+0000 7f0390d0a640 1 -- start start 2023-10-11T06:13:19.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.879+0000 7f0390d0a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f038c156be0 0x7f038c06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.879+0000 7f0390d0a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c0cde80 con 0x7f038c06c880 2023-10-11T06:13:19.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.879+0000 7f0390d0a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c06df80 con 0x7f038c156810 2023-10-11T06:13:19.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.879+0000 7f0390d0a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c0ce140 con 0x7f038c156be0 2023-10-11T06:13:19.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.879+0000 7f0383fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f038c06c880 0x7f038c0ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55584/0 (socket says 172.21.15.84:55584) 2023-10-11T06:13:19.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.880+0000 7f0383fff640 1 -- 172.21.15.84:0/1304127130 learned_addr learned my addr 172.21.15.84:0/1304127130 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:19.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.880+0000 7f038ad9d640 1 --2- 172.21.15.84:0/1304127130 >> v2:172.21.15.84:3300/0 conn(0x7f038c156be0 0x7f038c06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.880+0000 7f0389d9b640 1 -- 172.21.15.84:0/1304127130 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645339706 0 0) 0x7f038c0cde80 con 0x7f038c06c880 2023-10-11T06:13:19.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.880+0000 7f0389d9b640 1 -- 172.21.15.84:0/1304127130 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0378003520 con 0x7f038c06c880 2023-10-11T06:13:19.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.880+0000 7f038ad9d640 1 -- 172.21.15.84:0/1304127130 >> v1:172.21.15.99:6789/0 conn(0x7f038c156810 legacy=0x7f038c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.880+0000 7f038ad9d640 1 -- 172.21.15.84:0/1304127130 >> v1:172.21.15.84:6789/0 conn(0x7f038c06c880 legacy=0x7f038c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.880+0000 7f038ad9d640 1 -- 172.21.15.84:0/1304127130 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0cc3b0 con 0x7f038c156be0 2023-10-11T06:13:19.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.881+0000 7f038ad9d640 1 --2- 172.21.15.84:0/1304127130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c06d8e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0384009d90 tx=0x7f038402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddd6f5c32025cc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0389d9b640 1 -- 172.21.15.84:0/1304127130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038402cc60 con 0x7f038c156be0 2023-10-11T06:13:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0389d9b640 1 -- 172.21.15.84:0/1304127130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038402cdc0 con 0x7f038c156be0 2023-10-11T06:13:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0389d9b640 1 -- 172.21.15.84:0/1304127130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384034660 con 0x7f038c156be0 2023-10-11T06:13:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0390d0a640 1 -- 172.21.15.84:0/1304127130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 msgr2=0x7f038c06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0390d0a640 1 --2- 172.21.15.84:0/1304127130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c06d8e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0384009d90 tx=0x7f038402beb0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0390d0a640 1 -- 172.21.15.84:0/1304127130 shutdown_connections 2023-10-11T06:13:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0390d0a640 1 --2- 172.21.15.84:0/1304127130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c06d8e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0390d0a640 1 -- 172.21.15.84:0/1304127130 >> 172.21.15.84:0/1304127130 conn(0x7f038c069c70 msgr2=0x7f038c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0390d0a640 1 -- 172.21.15.84:0/1304127130 shutdown_connections 2023-10-11T06:13:19.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.882+0000 7f0390d0a640 1 -- 172.21.15.84:0/1304127130 wait complete. 2023-10-11T06:13:19.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.884+0000 7f0390d0a640 1 Processor -- start 2023-10-11T06:13:19.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.884+0000 7f0390d0a640 1 -- start start 2023-10-11T06:13:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f0390d0a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f038c06c880 0x7f038c151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f0390d0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f038c156810 0x7f038c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f0390d0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f0390d0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c1524b0 con 0x7f038c156be0 2023-10-11T06:13:19.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f0390d0a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0ace50 con 0x7f038c156810 2023-10-11T06:13:19.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f0390d0a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0ad130 con 0x7f038c06c880 2023-10-11T06:13:19.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f038ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f038ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c0af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46036/0 (socket says 172.21.15.84:46036) 2023-10-11T06:13:19.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.885+0000 7f038ad9d640 1 -- 172.21.15.84:0/4204121716 learned_addr learned my addr 172.21.15.84:0/4204121716 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:19.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.886+0000 7f038ad9d640 1 -- 172.21.15.84:0/4204121716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f038c06c880 msgr2=0x7f038c151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.886+0000 7f038ad9d640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f038c06c880 0x7f038c151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.886+0000 7f0383fff640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f038c156810 0x7f038c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.886+0000 7f038ad9d640 1 -- 172.21.15.84:0/4204121716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f038c156810 msgr2=0x7f038c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:19.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.886+0000 7f038ad9d640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f038c156810 0x7f038c0aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.886+0000 7f038ad9d640 1 -- 172.21.15.84:0/4204121716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384009a70 con 0x7f038c156be0 2023-10-11T06:13:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.886+0000 7f038ad9d640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c0af2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0384009d90 tx=0x7f0384004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.887+0000 7f0382ffd640 1 -- 172.21.15.84:0/4204121716 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384040650 con 0x7f038c156be0 2023-10-11T06:13:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.887+0000 7f0382ffd640 1 -- 172.21.15.84:0/4204121716 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03840407b0 con 0x7f038c156be0 2023-10-11T06:13:19.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.887+0000 7f0382ffd640 1 -- 172.21.15.84:0/4204121716 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384040b60 con 0x7f038c156be0 2023-10-11T06:13:19.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.887+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c0ad410 con 0x7f038c156be0 2023-10-11T06:13:19.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.887+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c0ad9d0 con 0x7f038c156be0 2023-10-11T06:13:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.888+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c0cbd30 con 0x7f038c156be0 2023-10-11T06:13:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.889+0000 7f0382ffd640 1 -- 172.21.15.84:0/4204121716 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0384040cc0 con 0x7f038c156be0 2023-10-11T06:13:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.890+0000 7f0382ffd640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f036c06a880 0x7f036c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.890+0000 7f0382ffd640 1 -- 172.21.15.84:0/4204121716 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f03840abf50 con 0x7f038c156be0 2023-10-11T06:13:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.890+0000 7f038a59c640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f036c06a880 0x7f036c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:19.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.891+0000 7f038a59c640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f036c06a880 0x7f036c06cd30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f037c000f60 tx=0x7f037c0079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:19.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:19.894+0000 7f0382ffd640 1 -- 172.21.15.84:0/4204121716 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0384077000 con 0x7f038c156be0 2023-10-11T06:13:20.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.088+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f038c0adbc0 con 0x7f038c156be0 2023-10-11T06:13:20.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.089+0000 7f0382ffd640 1 -- 172.21.15.84:0/4204121716 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0384009450 con 0x7f038c156be0 2023-10-11T06:13:20.091 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:13:20.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f036c06a880 msgr2=0x7f036c06cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f036c06a880 0x7f036c06cd30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f037c000f60 tx=0x7f037c0079b0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 msgr2=0x7f038c0af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c0af2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0384009d90 tx=0x7f0384004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:20.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 shutdown_connections 2023-10-11T06:13:20.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f036c06a880 0x7f036c06cd30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f038c156be0 0x7f038c0af2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f038c156810 0x7f038c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 --2- 172.21.15.84:0/4204121716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f038c06c880 0x7f038c151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 >> 172.21.15.84:0/4204121716 conn(0x7f038c069c70 msgr2=0x7f038c06c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:20.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 shutdown_connections 2023-10-11T06:13:20.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.092+0000 7f0390d0a640 1 -- 172.21.15.84:0/4204121716 wait complete. 2023-10-11T06:13:20.111 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:13:20.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.326+0000 7fd934527640 1 Processor -- start 2023-10-11T06:13:20.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.326+0000 7fd934527640 1 -- start start 2023-10-11T06:13:20.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.328+0000 7fd934527640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd92c0ced30 0x7fd92c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.328+0000 7fd934527640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.328+0000 7fd934527640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c071380 con 0x7fd92c15a0b0 2023-10-11T06:13:20.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.328+0000 7fd934527640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92c06e0f0 con 0x7fd92c0ced30 2023-10-11T06:13:20.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.328+0000 7fd934527640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0cbae0 con 0x7fd92c0c8d00 2023-10-11T06:13:20.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.329+0000 7fd931ac2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd92c0ced30 0x7fd92c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.329+0000 7fd932ac4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.329+0000 7fd931ac2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd92c0ced30 0x7fd92c0cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:47626/0 (socket says 172.21.15.84:47626) 2023-10-11T06:13:20.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.329+0000 7fd931ac2640 1 -- 172.21.15.84:0/482597437 learned_addr learned my addr 172.21.15.84:0/482597437 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.329+0000 7fd931ac2640 1 -- 172.21.15.84:0/482597437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 msgr2=0x7fd92c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.329+0000 7fd931ac2640 1 --2- 172.21.15.84:0/482597437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.329+0000 7fd931ac2640 1 -- 172.21.15.84:0/482597437 >> v1:172.21.15.99:6789/0 conn(0x7fd92c15a0b0 legacy=0x7fd92c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.330+0000 7fd931ac2640 1 -- 172.21.15.84:0/482597437 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c06de10 con 0x7fd92c0ced30 2023-10-11T06:13:20.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.330+0000 7fd9312c1640 1 -- 172.21.15.84:0/482597437 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442018401 0 0) 0x7fd92c071380 con 0x7fd92c15a0b0 2023-10-11T06:13:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.331+0000 7fd931ac2640 1 --2- 172.21.15.84:0/482597437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 0x7fd92c0cb5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0098a0 tx=0x7fd91c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4233cc69a70d84 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.332+0000 7fd9312c1640 1 -- 172.21.15.84:0/482597437 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0317c0 con 0x7fd92c0ced30 2023-10-11T06:13:20.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.332+0000 7fd9312c1640 1 -- 172.21.15.84:0/482597437 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c031920 con 0x7fd92c0ced30 2023-10-11T06:13:20.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.332+0000 7fd9312c1640 1 -- 172.21.15.84:0/482597437 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c031cb0 con 0x7fd92c0ced30 2023-10-11T06:13:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 -- 172.21.15.84:0/482597437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 msgr2=0x7fd92c0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 --2- 172.21.15.84:0/482597437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 0x7fd92c0cb5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0098a0 tx=0x7fd91c02b860 comp rx=0 tx=0).stop 2023-10-11T06:13:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 -- 172.21.15.84:0/482597437 shutdown_connections 2023-10-11T06:13:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 --2- 172.21.15.84:0/482597437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 --2- 172.21.15.84:0/482597437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 0x7fd92c0cb5a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 -- 172.21.15.84:0/482597437 >> 172.21.15.84:0/482597437 conn(0x7fd92c069c70 msgr2=0x7fd92c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:20.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 -- 172.21.15.84:0/482597437 shutdown_connections 2023-10-11T06:13:20.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.333+0000 7fd934527640 1 -- 172.21.15.84:0/482597437 wait complete. 2023-10-11T06:13:20.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.335+0000 7fd934527640 1 Processor -- start 2023-10-11T06:13:20.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.335+0000 7fd934527640 1 -- start start 2023-10-11T06:13:20.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd934527640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd934527640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 0x7fd92c147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd934527640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92c15a0b0 0x7fd92c158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd934527640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0b1b00 con 0x7fd92c15a0b0 2023-10-11T06:13:20.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd934527640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0b3be0 con 0x7fd92c0ced30 2023-10-11T06:13:20.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd934527640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0b1d10 con 0x7fd92c0c8d00 2023-10-11T06:13:20.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92c15a0b0 0x7fd92c158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92c15a0b0 0x7fd92c158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46048/0 (socket says 172.21.15.84:46048) 2023-10-11T06:13:20.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 -- 172.21.15.84:0/2880313127 learned_addr learned my addr 172.21.15.84:0/2880313127 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:20.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd931ac2640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 0x7fd92c147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd9322c3640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 -- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 msgr2=0x7fd92c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 -- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 msgr2=0x7fd92c147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 0x7fd92c147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd932ac4640 1 -- 172.21.15.84:0/2880313127 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0094d0 con 0x7fd92c15a0b0 2023-10-11T06:13:20.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.336+0000 7fd9322c3640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:20.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.337+0000 7fd932ac4640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92c15a0b0 0x7fd92c158f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd92400efc0 tx=0x7fd92400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:20.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.337+0000 7fd922ffd640 1 -- 172.21.15.84:0/2880313127 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924014480 con 0x7fd92c15a0b0 2023-10-11T06:13:20.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.337+0000 7fd922ffd640 1 -- 172.21.15.84:0/2880313127 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92400f040 con 0x7fd92c15a0b0 2023-10-11T06:13:20.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.337+0000 7fd922ffd640 1 -- 172.21.15.84:0/2880313127 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924014820 con 0x7fd92c15a0b0 2023-10-11T06:13:20.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.337+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c0b1ff0 con 0x7fd92c15a0b0 2023-10-11T06:13:20.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.337+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c0b25b0 con 0x7fd92c15a0b0 2023-10-11T06:13:20.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.338+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c0cb090 con 0x7fd92c15a0b0 2023-10-11T06:13:20.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.339+0000 7fd922ffd640 1 -- 172.21.15.84:0/2880313127 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd924014980 con 0x7fd92c15a0b0 2023-10-11T06:13:20.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.340+0000 7fd922ffd640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd91006a830 0x7fd91006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.340+0000 7fd922ffd640 1 -- 172.21.15.84:0/2880313127 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fd924089dc0 con 0x7fd92c15a0b0 2023-10-11T06:13:20.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.341+0000 7fd9322c3640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd91006a830 0x7fd91006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.341+0000 7fd9322c3640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd91006a830 0x7fd91006cce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd928000f60 tx=0x7fd928009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:20.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.345+0000 7fd922ffd640 1 -- 172.21.15.84:0/2880313127 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd924054df0 con 0x7fd92c15a0b0 2023-10-11T06:13:20.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.534+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd92c073ed0 con 0x7fd92c15a0b0 2023-10-11T06:13:20.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.535+0000 7fd922ffd640 1 -- 172.21.15.84:0/2880313127 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd9240180a0 con 0x7fd92c15a0b0 2023-10-11T06:13:20.536 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:13:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.537+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd91006a830 msgr2=0x7fd91006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.537+0000 7fd934527640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd91006a830 0x7fd91006cce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd928000f60 tx=0x7fd928009040 comp rx=0 tx=0).stop 2023-10-11T06:13:20.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92c15a0b0 msgr2=0x7fd92c158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92c15a0b0 0x7fd92c158f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd92400efc0 tx=0x7fd92400c490 comp rx=0 tx=0).stop 2023-10-11T06:13:20.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 shutdown_connections 2023-10-11T06:13:20.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd91006a830 0x7fd91006cce0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd92c15a0b0 0x7fd92c158f30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd92c0ced30 0x7fd92c147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 --2- 172.21.15.84:0/2880313127 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd92c0c8d00 0x7fd92c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 >> 172.21.15.84:0/2880313127 conn(0x7fd92c069c70 msgr2=0x7fd92c0c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:20.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 shutdown_connections 2023-10-11T06:13:20.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.538+0000 7fd934527640 1 -- 172.21.15.84:0/2880313127 wait complete. 2023-10-11T06:13:20.558 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:13:20.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.777+0000 7fb74d96e640 1 Processor -- start 2023-10-11T06:13:20.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.777+0000 7fb74d96e640 1 -- start start 2023-10-11T06:13:20.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb74d96e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1b10 0x7fb7480cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb74d96e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb74806c530 0x7fb7480cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb74d96e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74815cba0 con 0x7fb7480b1ee0 2023-10-11T06:13:20.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb74d96e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74809c220 con 0x7fb74806c530 2023-10-11T06:13:20.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb74d96e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7480d1360 con 0x7fb7480b1b10 2023-10-11T06:13:20.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb74d16d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb74806c530 0x7fb7480cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb747fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb7480b1ee0 0x7fb7480ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55592/0 (socket says 172.21.15.84:55592) 2023-10-11T06:13:20.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.779+0000 7fb747fff640 1 -- 172.21.15.84:0/3800961103 learned_addr learned my addr 172.21.15.84:0/3800961103 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.780+0000 7fb74c96c640 1 --2- 172.21.15.84:0/3800961103 >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1b10 0x7fb7480cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.780+0000 7fb74d16d640 1 -- 172.21.15.84:0/3800961103 >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1b10 msgr2=0x7fb7480cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.780+0000 7fb74d16d640 1 --2- 172.21.15.84:0/3800961103 >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1b10 0x7fb7480cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.780+0000 7fb74d16d640 1 -- 172.21.15.84:0/3800961103 >> v1:172.21.15.84:6789/0 conn(0x7fb7480b1ee0 legacy=0x7fb7480ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.780+0000 7fb74d16d640 1 -- 172.21.15.84:0/3800961103 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480d0ac0 con 0x7fb74806c530 2023-10-11T06:13:20.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.780+0000 7fb74c96c640 1 --2- 172.21.15.84:0/3800961103 >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1b10 0x7fb7480cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:20.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.781+0000 7fb74d16d640 1 --2- 172.21.15.84:0/3800961103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74806c530 0x7fb7480cf740 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb748073b40 tx=0x7fb74002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89ae34cdf1989ee server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:20.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.781+0000 7fb7477fe640 1 -- 172.21.15.84:0/3800961103 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74002cd10 con 0x7fb74806c530 2023-10-11T06:13:20.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.781+0000 7fb7477fe640 1 -- 172.21.15.84:0/3800961103 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74002ce70 con 0x7fb74806c530 2023-10-11T06:13:20.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.781+0000 7fb7477fe640 1 -- 172.21.15.84:0/3800961103 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740034640 con 0x7fb74806c530 2023-10-11T06:13:20.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 -- 172.21.15.84:0/3800961103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74806c530 msgr2=0x7fb7480cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3800961103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74806c530 0x7fb7480cf740 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb748073b40 tx=0x7fb74002bf60 comp rx=0 tx=0).stop 2023-10-11T06:13:20.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 -- 172.21.15.84:0/3800961103 shutdown_connections 2023-10-11T06:13:20.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3800961103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74806c530 0x7fb7480cf740 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3800961103 >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1b10 0x7fb7480cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 -- 172.21.15.84:0/3800961103 >> 172.21.15.84:0/3800961103 conn(0x7fb74809bc20 msgr2=0x7fb7480d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:20.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 -- 172.21.15.84:0/3800961103 shutdown_connections 2023-10-11T06:13:20.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.782+0000 7fb74d96e640 1 -- 172.21.15.84:0/3800961103 wait complete. 2023-10-11T06:13:20.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.784+0000 7fb74d96e640 1 Processor -- start 2023-10-11T06:13:20.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.784+0000 7fb74d96e640 1 -- start start 2023-10-11T06:13:20.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb74d96e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74806c530 0x7fb748159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb74d96e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7480b1b10 0x7fb7480c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb74d96e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7480b1ee0 0x7fb7480b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb74d96e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480b5750 con 0x7fb7480b1b10 2023-10-11T06:13:20.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb74d96e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480b5a30 con 0x7fb74806c530 2023-10-11T06:13:20.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb74d96e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480b2d00 con 0x7fb7480b1ee0 2023-10-11T06:13:20.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb747fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7480b1b10 0x7fb7480c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb747fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7480b1b10 0x7fb7480c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46074/0 (socket says 172.21.15.84:46074) 2023-10-11T06:13:20.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.785+0000 7fb747fff640 1 -- 172.21.15.84:0/3326503810 learned_addr learned my addr 172.21.15.84:0/3326503810 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:20.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb747fff640 1 -- 172.21.15.84:0/3326503810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7480b1ee0 msgr2=0x7fb7480b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb747fff640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7480b1ee0 0x7fb7480b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb747fff640 1 -- 172.21.15.84:0/3326503810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74806c530 msgr2=0x7fb748159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb747fff640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74806c530 0x7fb748159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb747fff640 1 -- 172.21.15.84:0/3326503810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740009a70 con 0x7fb7480b1b10 2023-10-11T06:13:20.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb747fff640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7480b1b10 0x7fb7480c9db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c480 tx=0x7fb73400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:20.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb7457fa640 1 -- 172.21.15.84:0/3326503810 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734005ce0 con 0x7fb7480b1b10 2023-10-11T06:13:20.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb7457fa640 1 -- 172.21.15.84:0/3326503810 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734005e40 con 0x7fb7480b1b10 2023-10-11T06:13:20.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb7457fa640 1 -- 172.21.15.84:0/3326503810 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340072f0 con 0x7fb7480b1b10 2023-10-11T06:13:20.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7480b2fe0 con 0x7fb7480b1b10 2023-10-11T06:13:20.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.786+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7481597b0 con 0x7fb7480b1b10 2023-10-11T06:13:20.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.787+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74804e890 con 0x7fb7480b1b10 2023-10-11T06:13:20.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.788+0000 7fb7457fa640 1 -- 172.21.15.84:0/3326503810 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb734007510 con 0x7fb7480b1b10 2023-10-11T06:13:20.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.789+0000 7fb7457fa640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb72c06a830 0x7fb72c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:20.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.789+0000 7fb7457fa640 1 -- 172.21.15.84:0/3326503810 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fb7340885e0 con 0x7fb7480b1b10 2023-10-11T06:13:20.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.790+0000 7fb74c96c640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb72c06a830 0x7fb72c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:20.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.790+0000 7fb74c96c640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb72c06a830 0x7fb72c06cce0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00a8b0 tx=0x7fb73c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:20.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.794+0000 7fb7457fa640 1 -- 172.21.15.84:0/3326503810 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb7340537f0 con 0x7fb7480b1b10 2023-10-11T06:13:20.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.985+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7480b5bc0 con 0x7fb7480b1b10 2023-10-11T06:13:20.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.986+0000 7fb7457fa640 1 -- 172.21.15.84:0/3326503810 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb734056e10 con 0x7fb7480b1b10 2023-10-11T06:13:20.987 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.988+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb72c06a830 msgr2=0x7fb72c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.988+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb72c06a830 0x7fb72c06cce0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00a8b0 tx=0x7fb73c008040 comp rx=0 tx=0).stop 2023-10-11T06:13:20.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.988+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7480b1b10 msgr2=0x7fb7480c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.988+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7480b1b10 0x7fb7480c9db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c480 tx=0x7fb73400c940 comp rx=0 tx=0).stop 2023-10-11T06:13:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 shutdown_connections 2023-10-11T06:13:20.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb72c06a830 0x7fb72c06cce0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7480b1ee0 0x7fb7480b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7480b1b10 0x7fb7480c9db0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 --2- 172.21.15.84:0/3326503810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74806c530 0x7fb748159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 >> 172.21.15.84:0/3326503810 conn(0x7fb74809bc20 msgr2=0x7fb7481e4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 shutdown_connections 2023-10-11T06:13:20.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:20.989+0000 7fb74d96e640 1 -- 172.21.15.84:0/3326503810 wait complete. 2023-10-11T06:13:21.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:13:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.207+0000 7f4cbf1ab640 1 Processor -- start 2023-10-11T06:13:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.207+0000 7f4cbf1ab640 1 -- start start 2023-10-11T06:13:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbf1ab640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4cb815a0b0 0x7f4cb80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbf1ab640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80c8d00 0x7f4cb80ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbf1ab640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb80cefc0 con 0x7f4cb80ceb60 2023-10-11T06:13:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbf1ab640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb806e0f0 con 0x7f4cb815a0b0 2023-10-11T06:13:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbf1ab640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80cbae0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbcf47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4cb815a0b0 0x7f4cb80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:21.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbcf47640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4cb815a0b0 0x7f4cb80cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46076/0 (socket says 172.21.15.84:46076) 2023-10-11T06:13:21.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbcf47640 1 -- 172.21.15.84:0/2280001369 learned_addr learned my addr 172.21.15.84:0/2280001369 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbcf47640 1 -- 172.21.15.84:0/2280001369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80c8d00 msgr2=0x7f4cb80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbcf47640 1 --2- 172.21.15.84:0/2280001369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80c8d00 0x7f4cb80ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbcf47640 1 -- 172.21.15.84:0/2280001369 >> v1:172.21.15.99:6789/0 conn(0x7f4cb80ceb60 legacy=0x7f4cb806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.209+0000 7f4cbcf47640 1 -- 172.21.15.84:0/2280001369 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb8071380 con 0x7f4cb815a0b0 2023-10-11T06:13:21.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.210+0000 7f4cbcf47640 1 --2- 172.21.15.84:0/2280001369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb815a0b0 0x7f4cb80cb5a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0098a0 tx=0x7f4cac02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2e81c191d93da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:21.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cb77fe640 1 -- 172.21.15.84:0/2280001369 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac031740 con 0x7f4cb815a0b0 2023-10-11T06:13:21.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cb77fe640 1 -- 172.21.15.84:0/2280001369 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac0318a0 con 0x7f4cb815a0b0 2023-10-11T06:13:21.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cb77fe640 1 -- 172.21.15.84:0/2280001369 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac031c30 con 0x7f4cb815a0b0 2023-10-11T06:13:21.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/2280001369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb815a0b0 msgr2=0x7f4cb80cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/2280001369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb815a0b0 0x7f4cb80cb5a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0098a0 tx=0x7f4cac02b7e0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/2280001369 shutdown_connections 2023-10-11T06:13:21.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/2280001369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80c8d00 0x7f4cb80ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/2280001369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb815a0b0 0x7f4cb80cb5a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/2280001369 >> 172.21.15.84:0/2280001369 conn(0x7f4cb8069c70 msgr2=0x7f4cb806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:21.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/2280001369 shutdown_connections 2023-10-11T06:13:21.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.211+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/2280001369 wait complete. 2023-10-11T06:13:21.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.213+0000 7f4cbf1ab640 1 Processor -- start 2023-10-11T06:13:21.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.213+0000 7f4cbf1ab640 1 -- start start 2023-10-11T06:13:21.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbf1ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb80c8d00 0x7f4cb80c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbf1ab640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80ceb60 0x7f4cb813f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbf1ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb815a0b0 0x7f4cb814dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbf1ab640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80c6ef0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbf1ab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80c7100 con 0x7f4cb815a0b0 2023-10-11T06:13:21.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbf1ab640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb80c73e0 con 0x7f4cb80ceb60 2023-10-11T06:13:21.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbcf47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb80c8d00 0x7f4cb80c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:21.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbcf47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb80c8d00 0x7f4cb80c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46078/0 (socket says 172.21.15.84:46078) 2023-10-11T06:13:21.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.214+0000 7f4cbcf47640 1 -- 172.21.15.84:0/1635407327 learned_addr learned my addr 172.21.15.84:0/1635407327 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:21.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cbcf47640 1 -- 172.21.15.84:0/1635407327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80ceb60 msgr2=0x7f4cb813f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cbcf47640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80ceb60 0x7f4cb813f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cbcf47640 1 -- 172.21.15.84:0/1635407327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb815a0b0 msgr2=0x7f4cb814dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cbcf47640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb815a0b0 0x7f4cb814dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cbcf47640 1 -- 172.21.15.84:0/1635407327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac0094d0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cbcf47640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb80c8d00 0x7f4cb80c7ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4cac000c00 tx=0x7f4cac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:21.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cb57fa640 1 -- 172.21.15.84:0/1635407327 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac033c70 con 0x7f4cb80c8d00 2023-10-11T06:13:21.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cb57fa640 1 -- 172.21.15.84:0/1635407327 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac033dd0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb80c76c0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.215+0000 7f4cb57fa640 1 -- 172.21.15.84:0/1635407327 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac03d6b0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.216+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb813f5b0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.216+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb80c7be0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.217+0000 7f4cb57fa640 1 -- 172.21.15.84:0/1635407327 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4cac03a040 con 0x7f4cb80c8d00 2023-10-11T06:13:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.219+0000 7f4cb57fa640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4c9c06a830 0x7f4c9c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.219+0000 7f4cb57fa640 1 -- 172.21.15.84:0/1635407327 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f4cac0ab100 con 0x7f4cb80c8d00 2023-10-11T06:13:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.219+0000 7f4cb7fff640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4c9c06a830 0x7f4c9c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.219+0000 7f4cb7fff640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4c9c06a830 0x7f4c9c06cce0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80c8810 tx=0x7f4ca4006c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:21.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.223+0000 7f4cb57fa640 1 -- 172.21.15.84:0/1635407327 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4cb80c7be0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.411+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cb80c7be0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.412+0000 7f4cb57fa640 1 -- 172.21.15.84:0/1635407327 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cb80c7be0 con 0x7f4cb80c8d00 2023-10-11T06:13:21.414 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:13:21.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4c9c06a830 msgr2=0x7f4c9c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4c9c06a830 0x7f4c9c06cce0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80c8810 tx=0x7f4ca4006c90 comp rx=0 tx=0).stop 2023-10-11T06:13:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb80c8d00 msgr2=0x7f4cb80c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb80c8d00 0x7f4cb80c7ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4cac000c00 tx=0x7f4cac0046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 shutdown_connections 2023-10-11T06:13:21.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4c9c06a830 0x7f4c9c06cce0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb815a0b0 0x7f4cb814dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cb80ceb60 0x7f4cb813f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 --2- 172.21.15.84:0/1635407327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cb80c8d00 0x7f4cb80c7ec0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 >> 172.21.15.84:0/1635407327 conn(0x7f4cb8069c70 msgr2=0x7f4cb814f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:21.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 shutdown_connections 2023-10-11T06:13:21.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.415+0000 7f4cbf1ab640 1 -- 172.21.15.84:0/1635407327 wait complete. 2023-10-11T06:13:21.435 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-11T06:13:21.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.644+0000 7f8a27e1a640 1 Processor -- start 2023-10-11T06:13:21.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.644+0000 7f8a27e1a640 1 -- start start 2023-10-11T06:13:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.645+0000 7f8a27e1a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a200d2ac0 0x7f8a200d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.645+0000 7f8a27e1a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a200d0940 0x7f8a200d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.645+0000 7f8a27e1a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a200cf220 con 0x7f8a2015f7e0 2023-10-11T06:13:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.645+0000 7f8a27e1a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a200d3480 con 0x7f8a200d0940 2023-10-11T06:13:21.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.645+0000 7f8a27e1a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a200cf500 con 0x7f8a200d2ac0 2023-10-11T06:13:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.646+0000 7f8a263b7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a200d0940 0x7f8a200d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.646+0000 7f8a263b7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a200d0940 0x7f8a200d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46092/0 (socket says 172.21.15.84:46092) 2023-10-11T06:13:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.646+0000 7f8a263b7640 1 -- 172.21.15.84:0/2567535205 learned_addr learned my addr 172.21.15.84:0/2567535205 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:21.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.646+0000 7f8a263b7640 1 -- 172.21.15.84:0/2567535205 >> v2:172.21.15.99:3300/0 conn(0x7f8a200d2ac0 msgr2=0x7f8a200d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:21.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.646+0000 7f8a263b7640 1 --2- 172.21.15.84:0/2567535205 >> v2:172.21.15.99:3300/0 conn(0x7f8a200d2ac0 0x7f8a200d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.646+0000 7f8a263b7640 1 -- 172.21.15.84:0/2567535205 >> v1:172.21.15.99:6789/0 conn(0x7f8a2015f7e0 legacy=0x7f8a200cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:21.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.646+0000 7f8a263b7640 1 -- 172.21.15.84:0/2567535205 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a200cf7e0 con 0x7f8a200d0940 2023-10-11T06:13:21.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.647+0000 7f8a263b7640 1 --2- 172.21.15.84:0/2567535205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d0940 0x7f8a200d0d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8a2008adb0 tx=0x7f8a1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288ac3a1ba38de11 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:21.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.647+0000 7f8a24bb4640 1 -- 172.21.15.84:0/2567535205 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18031870 con 0x7f8a200d0940 2023-10-11T06:13:21.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.647+0000 7f8a24bb4640 1 -- 172.21.15.84:0/2567535205 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a180319d0 con 0x7f8a200d0940 2023-10-11T06:13:21.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.647+0000 7f8a24bb4640 1 -- 172.21.15.84:0/2567535205 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18031d60 con 0x7f8a200d0940 2023-10-11T06:13:21.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 -- 172.21.15.84:0/2567535205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d0940 msgr2=0x7f8a200d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/2567535205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d0940 0x7f8a200d0d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8a2008adb0 tx=0x7f8a1802b910 comp rx=0 tx=0).stop 2023-10-11T06:13:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 -- 172.21.15.84:0/2567535205 shutdown_connections 2023-10-11T06:13:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/2567535205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d0940 0x7f8a200d0d60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/2567535205 >> v2:172.21.15.99:3300/0 conn(0x7f8a200d2ac0 0x7f8a200d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 -- 172.21.15.84:0/2567535205 >> 172.21.15.84:0/2567535205 conn(0x7f8a200713a0 msgr2=0x7f8a200717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 -- 172.21.15.84:0/2567535205 shutdown_connections 2023-10-11T06:13:21.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.648+0000 7f8a27e1a640 1 -- 172.21.15.84:0/2567535205 wait complete. 2023-10-11T06:13:21.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.650+0000 7f8a27e1a640 1 Processor -- start 2023-10-11T06:13:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.650+0000 7f8a27e1a640 1 -- start start 2023-10-11T06:13:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a27e1a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a200d0940 0x7f8a2014d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a27e1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d2ac0 0x7f8a200c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a27e1a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a2015f7e0 0x7f8a200c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a27e1a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a20157a90 con 0x7f8a200d2ac0 2023-10-11T06:13:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a27e1a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2014d640 con 0x7f8a2015f7e0 2023-10-11T06:13:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a27e1a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a201571f0 con 0x7f8a200d0940 2023-10-11T06:13:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a253b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d2ac0 0x7f8a200c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a253b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d2ac0 0x7f8a200c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46108/0 (socket says 172.21.15.84:46108) 2023-10-11T06:13:21.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.651+0000 7f8a253b5640 1 -- 172.21.15.84:0/1618884471 learned_addr learned my addr 172.21.15.84:0/1618884471 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a253b5640 1 -- 172.21.15.84:0/1618884471 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a200d0940 msgr2=0x7f8a2014d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a253b5640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a200d0940 0x7f8a2014d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a253b5640 1 -- 172.21.15.84:0/1618884471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a2015f7e0 msgr2=0x7f8a200c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a253b5640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a2015f7e0 0x7f8a200c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a253b5640 1 -- 172.21.15.84:0/1618884471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a180094d0 con 0x7f8a200d2ac0 2023-10-11T06:13:21.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a253b5640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d2ac0 0x7f8a200c76e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400b4a0 tx=0x7f8a1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a1e7fc640 1 -- 172.21.15.84:0/1618884471 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14010610 con 0x7f8a200d2ac0 2023-10-11T06:13:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a1e7fc640 1 -- 172.21.15.84:0/1618884471 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14010770 con 0x7f8a200d2ac0 2023-10-11T06:13:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a1e7fc640 1 -- 172.21.15.84:0/1618884471 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14010b20 con 0x7f8a200d2ac0 2023-10-11T06:13:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a201574d0 con 0x7f8a200d2ac0 2023-10-11T06:13:21.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.652+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a201599f0 con 0x7f8a200d2ac0 2023-10-11T06:13:21.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.654+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a200b2300 con 0x7f8a200d2ac0 2023-10-11T06:13:21.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.655+0000 7f8a1e7fc640 1 -- 172.21.15.84:0/1618884471 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8a14010c80 con 0x7f8a200d2ac0 2023-10-11T06:13:21.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.655+0000 7f8a1e7fc640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8a0c06a830 0x7f8a0c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:21.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.656+0000 7f8a1e7fc640 1 -- 172.21.15.84:0/1618884471 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f8a14088ea0 con 0x7f8a200d2ac0 2023-10-11T06:13:21.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.656+0000 7f8a25bb6640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8a0c06a830 0x7f8a0c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:21.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.656+0000 7f8a25bb6640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8a0c06a830 0x7f8a0c06cce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a10001c60 tx=0x7f8a10007d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:21.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.660+0000 7f8a1e7fc640 1 -- 172.21.15.84:0/1618884471 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8a14053ee0 con 0x7f8a200d2ac0 2023-10-11T06:13:21.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.834+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f8a20159be0 con 0x7f8a200d2ac0 2023-10-11T06:13:21.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.834+0000 7f8a1e7fc640 1 -- 172.21.15.84:0/1618884471 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) v1 ==== 97+0+11 (secure 0 0 0) 0x7f8a1401e020 con 0x7f8a200d2ac0 2023-10-11T06:13:21.836 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:21.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8a0c06a830 msgr2=0x7f8a0c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8a0c06a830 0x7f8a0c06cce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a10001c60 tx=0x7f8a10007d10 comp rx=0 tx=0).stop 2023-10-11T06:13:21.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d2ac0 msgr2=0x7f8a200c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:21.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d2ac0 0x7f8a200c76e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400b4a0 tx=0x7f8a1400b960 comp rx=0 tx=0).stop 2023-10-11T06:13:21.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 shutdown_connections 2023-10-11T06:13:21.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8a0c06a830 0x7f8a0c06cce0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a2015f7e0 0x7f8a200c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a200d2ac0 0x7f8a200c76e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 --2- 172.21.15.84:0/1618884471 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a200d0940 0x7f8a2014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:21.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 >> 172.21.15.84:0/1618884471 conn(0x7f8a200713a0 msgr2=0x7f8a2015ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:21.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 shutdown_connections 2023-10-11T06:13:21.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:21.837+0000 7f8a27e1a640 1 -- 172.21.15.84:0/1618884471 wait complete. 2023-10-11T06:13:21.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:13:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.066+0000 7fce858a4640 1 Processor -- start 2023-10-11T06:13:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.067+0000 7fce858a4640 1 -- start start 2023-10-11T06:13:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.068+0000 7fce858a4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce800ca300 0x7fce800cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.068+0000 7fce858a4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce800c9360 0x7fce800ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.068+0000 7fce858a4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce80071a90 con 0x7fce8015d4f0 2023-10-11T06:13:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.068+0000 7fce858a4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce80072310 con 0x7fce800c9360 2023-10-11T06:13:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.068+0000 7fce858a4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce80070280 con 0x7fce800ca300 2023-10-11T06:13:22.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce7f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce800c9360 0x7fce800ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce7f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce800c9360 0x7fce800ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46116/0 (socket says 172.21.15.84:46116) 2023-10-11T06:13:22.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce7f7fe640 1 -- 172.21.15.84:0/3915054124 learned_addr learned my addr 172.21.15.84:0/3915054124 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:22.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce77fff640 1 --2- 172.21.15.84:0/3915054124 >> v2:172.21.15.99:3300/0 conn(0x7fce800ca300 0x7fce800cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce7f7fe640 1 -- 172.21.15.84:0/3915054124 >> v2:172.21.15.99:3300/0 conn(0x7fce800ca300 msgr2=0x7fce800cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:22.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce7f7fe640 1 --2- 172.21.15.84:0/3915054124 >> v2:172.21.15.99:3300/0 conn(0x7fce800ca300 0x7fce800cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce7f7fe640 1 -- 172.21.15.84:0/3915054124 >> v1:172.21.15.99:6789/0 conn(0x7fce8015d4f0 legacy=0x7fce800c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:22.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.069+0000 7fce7f7fe640 1 -- 172.21.15.84:0/3915054124 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80071cc0 con 0x7fce800c9360 2023-10-11T06:13:22.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.070+0000 7fce7f7fe640 1 --2- 172.21.15.84:0/3915054124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce800ce550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fce800b0980 tx=0x7fce7802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6b30cbaef8571a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:22.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.070+0000 7fce7e7fc640 1 -- 172.21.15.84:0/3915054124 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7802ca80 con 0x7fce800c9360 2023-10-11T06:13:22.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce7e7fc640 1 -- 172.21.15.84:0/3915054124 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7802cbe0 con 0x7fce800c9360 2023-10-11T06:13:22.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce7e7fc640 1 -- 172.21.15.84:0/3915054124 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780343f0 con 0x7fce800c9360 2023-10-11T06:13:22.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 -- 172.21.15.84:0/3915054124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 msgr2=0x7fce800ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 --2- 172.21.15.84:0/3915054124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce800ce550 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fce800b0980 tx=0x7fce7802bee0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 -- 172.21.15.84:0/3915054124 shutdown_connections 2023-10-11T06:13:22.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 --2- 172.21.15.84:0/3915054124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce800ce550 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 --2- 172.21.15.84:0/3915054124 >> v2:172.21.15.99:3300/0 conn(0x7fce800ca300 0x7fce800cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 -- 172.21.15.84:0/3915054124 >> 172.21.15.84:0/3915054124 conn(0x7fce80069c70 msgr2=0x7fce8006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:22.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 -- 172.21.15.84:0/3915054124 shutdown_connections 2023-10-11T06:13:22.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.071+0000 7fce858a4640 1 -- 172.21.15.84:0/3915054124 wait complete. 2023-10-11T06:13:22.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.073+0000 7fce858a4640 1 Processor -- start 2023-10-11T06:13:22.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.073+0000 7fce858a4640 1 -- start start 2023-10-11T06:13:22.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.074+0000 7fce858a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce801ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.074+0000 7fce858a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce800ca300 0x7fce8014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.074+0000 7fce858a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce8015d4f0 0x7fce800a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.074+0000 7fce858a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce800c62b0 con 0x7fce800c9360 2023-10-11T06:13:22.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.074+0000 7fce858a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce801efbe0 con 0x7fce800ca300 2023-10-11T06:13:22.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.074+0000 7fce858a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800c6500 con 0x7fce8015d4f0 2023-10-11T06:13:22.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.074+0000 7fce7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce801ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce801ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46124/0 (socket says 172.21.15.84:46124) 2023-10-11T06:13:22.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 -- 172.21.15.84:0/1618327100 learned_addr learned my addr 172.21.15.84:0/1618327100 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:22.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 -- 172.21.15.84:0/1618327100 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce8015d4f0 msgr2=0x7fce800a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce8015d4f0 0x7fce800a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 -- 172.21.15.84:0/1618327100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce800ca300 msgr2=0x7fce8014db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce800ca300 0x7fce8014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 -- 172.21.15.84:0/1618327100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78009b30 con 0x7fce800c9360 2023-10-11T06:13:22.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7effd640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce801ef6a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c400 tx=0x7fce7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:22.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7cff9640 1 -- 172.21.15.84:0/1618327100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700054e0 con 0x7fce800c9360 2023-10-11T06:13:22.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.075+0000 7fce7cff9640 1 -- 172.21.15.84:0/1618327100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70003640 con 0x7fce800c9360 2023-10-11T06:13:22.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.076+0000 7fce7cff9640 1 -- 172.21.15.84:0/1618327100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700039f0 con 0x7fce800c9360 2023-10-11T06:13:22.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.076+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce800c67e0 con 0x7fce800c9360 2023-10-11T06:13:22.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.076+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce800c6da0 con 0x7fce800c9360 2023-10-11T06:13:22.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.077+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce80073ed0 con 0x7fce800c9360 2023-10-11T06:13:22.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.078+0000 7fce7cff9640 1 -- 172.21.15.84:0/1618327100 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fce70003b50 con 0x7fce800c9360 2023-10-11T06:13:22.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.078+0000 7fce7cff9640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce6006a860 0x7fce6006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.079+0000 7fce7cff9640 1 -- 172.21.15.84:0/1618327100 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fce70088490 con 0x7fce800c9360 2023-10-11T06:13:22.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.079+0000 7fce77fff640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce6006a860 0x7fce6006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.079+0000 7fce77fff640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce6006a860 0x7fce6006cd10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce68006fd0 tx=0x7fce68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:22.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.083+0000 7fce7cff9640 1 -- 172.21.15.84:0/1618327100 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fce700534c0 con 0x7fce800c9360 2023-10-11T06:13:22.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.285+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fce800b09b0 con 0x7fce800c9360 2023-10-11T06:13:22.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.286+0000 7fce7cff9640 1 -- 172.21.15.84:0/1618327100 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7fce70056ae0 con 0x7fce800c9360 2023-10-11T06:13:22.287 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:13:22.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce6006a860 msgr2=0x7fce6006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce6006a860 0x7fce6006cd10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce68006fd0 tx=0x7fce68008040 comp rx=0 tx=0).stop 2023-10-11T06:13:22.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 msgr2=0x7fce801ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce801ef6a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c400 tx=0x7fce7000c8c0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 shutdown_connections 2023-10-11T06:13:22.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fce6006a860 0x7fce6006cd10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce8015d4f0 0x7fce800a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce800ca300 0x7fce8014db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 --2- 172.21.15.84:0/1618327100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce800c9360 0x7fce801ef6a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 >> 172.21.15.84:0/1618327100 conn(0x7fce80069c70 msgr2=0x7fce800ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:22.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 shutdown_connections 2023-10-11T06:13:22.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.288+0000 7fce858a4640 1 -- 172.21.15.84:0/1618327100 wait complete. 2023-10-11T06:13:22.308 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:22.308 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:22.365 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:22.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.577+0000 7ff3d08f6640 1 Processor -- start 2023-10-11T06:13:22.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.577+0000 7ff3d08f6640 1 -- start start 2023-10-11T06:13:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3d08f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3cc06c000 0x7ff3cc0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3d08f6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc158e40 0x7ff3cc0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3d08f6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3cc0adc40 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3d08f6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3cc0c97e0 con 0x7ff3cc06c000 2023-10-11T06:13:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3d08f6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc0c9ac0 con 0x7ff3cc158e40 2023-10-11T06:13:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3cad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3cc06c000 0x7ff3cc0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3cad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3cc06c000 0x7ff3cc0c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46134/0 (socket says 172.21.15.84:46134) 2023-10-11T06:13:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.578+0000 7ff3cad9d640 1 -- 172.21.15.84:0/498722351 learned_addr learned my addr 172.21.15.84:0/498722351 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.579+0000 7ff3cad9d640 1 -- 172.21.15.84:0/498722351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc158e40 msgr2=0x7ff3cc0c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:22.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.579+0000 7ff3cad9d640 1 --2- 172.21.15.84:0/498722351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc158e40 0x7ff3cc0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.579+0000 7ff3cad9d640 1 -- 172.21.15.84:0/498722351 >> v1:172.21.15.99:6789/0 conn(0x7ff3cc06c3d0 legacy=0x7ff3cc158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:22.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.579+0000 7ff3cad9d640 1 -- 172.21.15.84:0/498722351 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc0c9da0 con 0x7ff3cc06c000 2023-10-11T06:13:22.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.580+0000 7ff3cad9d640 1 --2- 172.21.15.84:0/498722351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c000 0x7ff3cc0c8c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009870 tx=0x7ff3bc02db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36df1e67890e7c00 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:22.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.580+0000 7ff3ca59c640 1 -- 172.21.15.84:0/498722351 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc02f8c0 con 0x7ff3cc06c000 2023-10-11T06:13:22.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.580+0000 7ff3ca59c640 1 -- 172.21.15.84:0/498722351 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc02fa20 con 0x7ff3cc06c000 2023-10-11T06:13:22.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.580+0000 7ff3ca59c640 1 -- 172.21.15.84:0/498722351 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc02fdb0 con 0x7ff3cc06c000 2023-10-11T06:13:22.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 -- 172.21.15.84:0/498722351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c000 msgr2=0x7ff3cc0c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/498722351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c000 0x7ff3cc0c8c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009870 tx=0x7ff3bc02db00 comp rx=0 tx=0).stop 2023-10-11T06:13:22.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 -- 172.21.15.84:0/498722351 shutdown_connections 2023-10-11T06:13:22.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/498722351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc158e40 0x7ff3cc0c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/498722351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c000 0x7ff3cc0c8c20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 -- 172.21.15.84:0/498722351 >> 172.21.15.84:0/498722351 conn(0x7ff3cc069c70 msgr2=0x7ff3cc06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:22.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 -- 172.21.15.84:0/498722351 shutdown_connections 2023-10-11T06:13:22.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.581+0000 7ff3d08f6640 1 -- 172.21.15.84:0/498722351 wait complete. 2023-10-11T06:13:22.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.583+0000 7ff3d08f6640 1 Processor -- start 2023-10-11T06:13:22.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.583+0000 7ff3d08f6640 1 -- start start 2023-10-11T06:13:22.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3d08f6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc06c000 0x7ff3cc1f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3d08f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c3d0 0x7ff3cc0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3d08f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc158e40 0x7ff3cc0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3d08f6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1f2280 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3d08f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc0abe80 con 0x7ff3cc158e40 2023-10-11T06:13:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3d08f6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc0ac160 con 0x7ff3cc06c000 2023-10-11T06:13:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c3d0 0x7ff3cc0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c3d0 0x7ff3cc0b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46140/0 (socket says 172.21.15.84:46140) 2023-10-11T06:13:22.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 -- 172.21.15.84:0/2484995566 learned_addr learned my addr 172.21.15.84:0/2484995566 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3cad9d640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc06c000 0x7ff3cc1f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 -- 172.21.15.84:0/2484995566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc06c000 msgr2=0x7ff3cc1f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc06c000 0x7ff3cc1f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 -- 172.21.15.84:0/2484995566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc158e40 msgr2=0x7ff3cc0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc158e40 0x7ff3cc0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.584+0000 7ff3c259c640 1 -- 172.21.15.84:0/2484995566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc0094d0 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.585+0000 7ff3c259c640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c3d0 0x7ff3cc0b34a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c7a0 tx=0x7ff3b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.585+0000 7ff3c3fff640 1 -- 172.21.15.84:0/2484995566 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4003d80 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.585+0000 7ff3c3fff640 1 -- 172.21.15.84:0/2484995566 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b4005070 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.585+0000 7ff3c3fff640 1 -- 172.21.15.84:0/2484995566 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4005460 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.585+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc0ac440 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.585+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc0ab8e0 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.586+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc1f25f0 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.587+0000 7ff3c3fff640 1 -- 172.21.15.84:0/2484995566 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff3b40055c0 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.588+0000 7ff3c3fff640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3ac06a830 0x7ff3ac06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:22.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.588+0000 7ff3c3fff640 1 -- 172.21.15.84:0/2484995566 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7ff3b4088670 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.588+0000 7ff3cad9d640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3ac06a830 0x7ff3ac06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:22.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.589+0000 7ff3cad9d640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3ac06a830 0x7ff3ac06cce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc038000 tx=0x7ff3bc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:22.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.592+0000 7ff3c3fff640 1 -- 172.21.15.84:0/2484995566 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff3cc1f25f0 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.780+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3cc1f25f0 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.783+0000 7ff3c3fff640 1 -- 172.21.15.84:0/2484995566 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7ff3cc1f25f0 con 0x7ff3cc06c3d0 2023-10-11T06:13:22.785 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:22.785 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:22.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3ac06a830 msgr2=0x7ff3ac06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3ac06a830 0x7ff3ac06cce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc038000 tx=0x7ff3bc008030 comp rx=0 tx=0).stop 2023-10-11T06:13:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c3d0 msgr2=0x7ff3cc0b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c3d0 0x7ff3cc0b34a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c7a0 tx=0x7ff3b400cc60 comp rx=0 tx=0).stop 2023-10-11T06:13:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 shutdown_connections 2023-10-11T06:13:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3ac06a830 0x7ff3ac06cce0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3cc158e40 0x7ff3cc0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3cc06c3d0 0x7ff3cc0b34a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 --2- 172.21.15.84:0/2484995566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3cc06c000 0x7ff3cc1f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 >> 172.21.15.84:0/2484995566 conn(0x7ff3cc069c70 msgr2=0x7ff3cc0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:22.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 shutdown_connections 2023-10-11T06:13:22.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:22.787+0000 7ff3d08f6640 1 -- 172.21.15.84:0/2484995566 wait complete. 2023-10-11T06:13:22.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:12:38.260176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:12:39.011036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-11T06:12:50.853138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:13:22.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:13:23.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.014+0000 7fa81e939640 1 Processor -- start 2023-10-11T06:13:23.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.014+0000 7fa81e939640 1 -- start start 2023-10-11T06:13:23.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.016+0000 7fa81e939640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8180b8040 0x7fa8180cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.016+0000 7fa81e939640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180d05f0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.016+0000 7fa81e939640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180cf920 con 0x7fa8180d3150 2023-10-11T06:13:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.016+0000 7fa81e939640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8180cf730 con 0x7fa8180b8040 2023-10-11T06:13:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.016+0000 7fa817fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8180b8040 0x7fa8180cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.016+0000 7fa817fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8180b8040 0x7fa8180cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46146/0 (socket says 172.21.15.84:46146) 2023-10-11T06:13:23.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.016+0000 7fa817fff640 1 -- 172.21.15.84:0/2649283004 learned_addr learned my addr 172.21.15.84:0/2649283004 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:23.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.017+0000 7fa817fff640 1 -- 172.21.15.84:0/2649283004 >> v1:172.21.15.99:6789/0 conn(0x7fa8180d3150 legacy=0x7fa8180d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:23.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.017+0000 7fa817fff640 1 -- 172.21.15.84:0/2649283004 >> v1:172.21.15.84:6789/0 conn(0x7fa8180cd5b0 legacy=0x7fa8180d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.017+0000 7fa817fff640 1 -- 172.21.15.84:0/2649283004 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8180cfc00 con 0x7fa8180b8040 2023-10-11T06:13:23.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.018+0000 7fa817fff640 1 --2- 172.21.15.84:0/2649283004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180b8040 0x7fa8180cd070 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede236bf856d21f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.018+0000 7fa8177fe640 1 -- 172.21.15.84:0/2649283004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808030860 con 0x7fa8180b8040 2023-10-11T06:13:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.018+0000 7fa8177fe640 1 -- 172.21.15.84:0/2649283004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8080309c0 con 0x7fa8180b8040 2023-10-11T06:13:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.018+0000 7fa8177fe640 1 -- 172.21.15.84:0/2649283004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808030d50 con 0x7fa8180b8040 2023-10-11T06:13:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.019+0000 7fa81e939640 1 -- 172.21.15.84:0/2649283004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180b8040 msgr2=0x7fa8180cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.019+0000 7fa81e939640 1 --2- 172.21.15.84:0/2649283004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180b8040 0x7fa8180cd070 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80802b9a0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.019+0000 7fa81e939640 1 -- 172.21.15.84:0/2649283004 shutdown_connections 2023-10-11T06:13:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.019+0000 7fa81e939640 1 --2- 172.21.15.84:0/2649283004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180b8040 0x7fa8180cd070 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.019+0000 7fa81e939640 1 -- 172.21.15.84:0/2649283004 >> 172.21.15.84:0/2649283004 conn(0x7fa81806afe0 msgr2=0x7fa81806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.019+0000 7fa81e939640 1 -- 172.21.15.84:0/2649283004 shutdown_connections 2023-10-11T06:13:23.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.019+0000 7fa81e939640 1 -- 172.21.15.84:0/2649283004 wait complete. 2023-10-11T06:13:23.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.020+0000 7fa81e939640 1 Processor -- start 2023-10-11T06:13:23.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.020+0000 7fa81e939640 1 -- start start 2023-10-11T06:13:23.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.021+0000 7fa81e939640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8180b8040 0x7fa81815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.021+0000 7fa81e939640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180cd5b0 0x7fa818155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.021+0000 7fa81e939640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8180d3150 0x7fa8181561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.021+0000 7fa81e939640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818157500 con 0x7fa8180cd5b0 2023-10-11T06:13:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.021+0000 7fa81e939640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c8500 con 0x7fa8180b8040 2023-10-11T06:13:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.021+0000 7fa81e939640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa818156830 con 0x7fa8180d3150 2023-10-11T06:13:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180cd5b0 0x7fa818155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180cd5b0 0x7fa818155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46156/0 (socket says 172.21.15.84:46156) 2023-10-11T06:13:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 -- 172.21.15.84:0/3943682967 learned_addr learned my addr 172.21.15.84:0/3943682967 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 -- 172.21.15.84:0/3943682967 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8180d3150 msgr2=0x7fa8181561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8180d3150 0x7fa8181561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 -- 172.21.15.84:0/3943682967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8180b8040 msgr2=0x7fa81815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8180b8040 0x7fa81815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 -- 172.21.15.84:0/3943682967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080094d0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa80f7fe640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180cd5b0 0x7fa818155c80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c480 tx=0x7fa80000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa8157fa640 1 -- 172.21.15.84:0/3943682967 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8000052d0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.022+0000 7fa8157fa640 1 -- 172.21.15.84:0/3943682967 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800005430 con 0x7fa8180cd5b0 2023-10-11T06:13:23.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.023+0000 7fa8157fa640 1 -- 172.21.15.84:0/3943682967 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800012870 con 0x7fa8180cd5b0 2023-10-11T06:13:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.023+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180c7d40 con 0x7fa8180cd5b0 2023-10-11T06:13:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.023+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8180c7fe0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.024+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8180c81d0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.025+0000 7fa8157fa640 1 -- 172.21.15.84:0/3943682967 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa8000129d0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.026+0000 7fa8157fa640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa7f806a830 0x7fa7f806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.026+0000 7fa8157fa640 1 -- 172.21.15.84:0/3943682967 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fa800088d30 con 0x7fa8180cd5b0 2023-10-11T06:13:23.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.026+0000 7fa817fff640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa7f806a830 0x7fa7f806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.027+0000 7fa817fff640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa7f806a830 0x7fa7f806cce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa808036000 tx=0x7fa80803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.030+0000 7fa8157fa640 1 -- 172.21.15.84:0/3943682967 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa800053df0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.220+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa818073ed0 con 0x7fa8180cd5b0 2023-10-11T06:13:23.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.221+0000 7fa8157fa640 1 -- 172.21.15.84:0/3943682967 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa800057410 con 0x7fa8180cd5b0 2023-10-11T06:13:23.222 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:13:23.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa7f806a830 msgr2=0x7fa7f806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa7f806a830 0x7fa7f806cce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa808036000 tx=0x7fa80803b040 comp rx=0 tx=0).stop 2023-10-11T06:13:23.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180cd5b0 msgr2=0x7fa818155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180cd5b0 0x7fa818155c80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c480 tx=0x7fa80000c940 comp rx=0 tx=0).stop 2023-10-11T06:13:23.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 shutdown_connections 2023-10-11T06:13:23.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa7f806a830 0x7fa7f806cce0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8180d3150 0x7fa8181561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8180cd5b0 0x7fa818155c80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 --2- 172.21.15.84:0/3943682967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8180b8040 0x7fa81815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.223+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 >> 172.21.15.84:0/3943682967 conn(0x7fa81806afe0 msgr2=0x7fa81815fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:23.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.224+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 shutdown_connections 2023-10-11T06:13:23.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.224+0000 7fa81e939640 1 -- 172.21.15.84:0/3943682967 wait complete. 2023-10-11T06:13:23.242 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:13:23.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.462+0000 7f530622d640 1 Processor -- start 2023-10-11T06:13:23.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.462+0000 7f530622d640 1 -- start start 2023-10-11T06:13:23.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.463+0000 7f530622d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f53000cec20 0x7f53000c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.463+0000 7f530622d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53000c9b90 con 0x7f530014ff30 2023-10-11T06:13:23.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.463+0000 7f530622d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530015da10 con 0x7f5300150300 2023-10-11T06:13:23.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.463+0000 7f530622d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53000cc990 con 0x7f53000cec20 2023-10-11T06:13:23.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.464+0000 7f52ff7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f530014ff30 0x7f53000cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55616/0 (socket says 172.21.15.84:55616) 2023-10-11T06:13:23.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.464+0000 7f52ff7fe640 1 -- 172.21.15.84:0/3586763492 learned_addr learned my addr 172.21.15.84:0/3586763492 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:23.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.464+0000 7f52fffff640 1 --2- 172.21.15.84:0/3586763492 >> v2:172.21.15.84:3300/0 conn(0x7f53000cec20 0x7f53000c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.464+0000 7f52feffd640 1 -- 172.21.15.84:0/3586763492 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418540588 0 0) 0x7f53000c9b90 con 0x7f530014ff30 2023-10-11T06:13:23.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.464+0000 7f52feffd640 1 -- 172.21.15.84:0/3586763492 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52ec003520 con 0x7f530014ff30 2023-10-11T06:13:23.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.464+0000 7f52fffff640 1 -- 172.21.15.84:0/3586763492 >> v1:172.21.15.99:6789/0 conn(0x7f5300150300 legacy=0x7f53000ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.464+0000 7f52fffff640 1 -- 172.21.15.84:0/3586763492 >> v1:172.21.15.84:6789/0 conn(0x7f530014ff30 legacy=0x7f53000cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.465+0000 7f52fffff640 1 -- 172.21.15.84:0/3586763492 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530015ce00 con 0x7f53000cec20 2023-10-11T06:13:23.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.466+0000 7f52fffff640 1 --2- 172.21.15.84:0/3586763492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53000cec20 0x7f53000c9650 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791d9b0da8e8c087 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.466+0000 7f52feffd640 1 -- 172.21.15.84:0/3586763492 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f802cc60 con 0x7f53000cec20 2023-10-11T06:13:23.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.466+0000 7f52feffd640 1 -- 172.21.15.84:0/3586763492 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f802cdc0 con 0x7f53000cec20 2023-10-11T06:13:23.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.466+0000 7f52feffd640 1 -- 172.21.15.84:0/3586763492 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8034660 con 0x7f53000cec20 2023-10-11T06:13:23.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.466+0000 7f530622d640 1 -- 172.21.15.84:0/3586763492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53000cec20 msgr2=0x7f53000c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.466+0000 7f530622d640 1 --2- 172.21.15.84:0/3586763492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53000cec20 0x7f53000c9650 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f802beb0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.467+0000 7f530622d640 1 -- 172.21.15.84:0/3586763492 shutdown_connections 2023-10-11T06:13:23.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.467+0000 7f530622d640 1 --2- 172.21.15.84:0/3586763492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53000cec20 0x7f53000c9650 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.467+0000 7f530622d640 1 -- 172.21.15.84:0/3586763492 >> 172.21.15.84:0/3586763492 conn(0x7f5300069c70 msgr2=0x7f530006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:23.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.467+0000 7f530622d640 1 -- 172.21.15.84:0/3586763492 shutdown_connections 2023-10-11T06:13:23.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.467+0000 7f530622d640 1 -- 172.21.15.84:0/3586763492 wait complete. 2023-10-11T06:13:23.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.468+0000 7f530622d640 1 Processor -- start 2023-10-11T06:13:23.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.468+0000 7f530622d640 1 -- start start 2023-10-11T06:13:23.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.469+0000 7f530622d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53000cec20 0x7f530014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.469+0000 7f530622d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f530014ff30 0x7f53000c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.469+0000 7f530622d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5300150300 0x7f53000c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.469+0000 7f530622d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300151b90 con 0x7f5300150300 2023-10-11T06:13:23.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.469+0000 7f530622d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300151e70 con 0x7f530014ff30 2023-10-11T06:13:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.469+0000 7f530622d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5300152150 con 0x7f53000cec20 2023-10-11T06:13:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5300150300 0x7f53000c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5300150300 0x7f53000c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46160/0 (socket says 172.21.15.84:46160) 2023-10-11T06:13:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 -- 172.21.15.84:0/331344829 learned_addr learned my addr 172.21.15.84:0/331344829 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:23.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 -- 172.21.15.84:0/331344829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53000cec20 msgr2=0x7f530014e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53000cec20 0x7f530014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 -- 172.21.15.84:0/331344829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f530014ff30 msgr2=0x7f53000c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52f6ffd640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f530014ff30 0x7f53000c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f530014ff30 0x7f53000c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.470+0000 7f52fffff640 1 -- 172.21.15.84:0/331344829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f8009a70 con 0x7f5300150300 2023-10-11T06:13:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.471+0000 7f52fffff640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5300150300 0x7f53000c6390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.471+0000 7f52fcff9640 1 -- 172.21.15.84:0/331344829 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8040650 con 0x7f5300150300 2023-10-11T06:13:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.471+0000 7f52fcff9640 1 -- 172.21.15.84:0/331344829 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f80407b0 con 0x7f5300150300 2023-10-11T06:13:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.471+0000 7f52fcff9640 1 -- 172.21.15.84:0/331344829 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8040b60 con 0x7f5300150300 2023-10-11T06:13:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.471+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5300152430 con 0x7f5300150300 2023-10-11T06:13:23.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.471+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530014e6f0 con 0x7f5300150300 2023-10-11T06:13:23.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.472+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530006ab90 con 0x7f5300150300 2023-10-11T06:13:23.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.473+0000 7f52fcff9640 1 -- 172.21.15.84:0/331344829 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f52f8040cc0 con 0x7f5300150300 2023-10-11T06:13:23.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.474+0000 7f52fcff9640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f52e006a830 0x7f52e006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.475+0000 7f52fcff9640 1 -- 172.21.15.84:0/331344829 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f52f80abe90 con 0x7f5300150300 2023-10-11T06:13:23.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.475+0000 7f52ff7fe640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f52e006a830 0x7f52e006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.475+0000 7f52ff7fe640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f52e006a830 0x7f52e006cce0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52f0001610 tx=0x7f52f0007930 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.479+0000 7f52fcff9640 1 -- 172.21.15.84:0/331344829 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f52f8076ff0 con 0x7f5300150300 2023-10-11T06:13:23.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.668+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5300073ed0 con 0x7f5300150300 2023-10-11T06:13:23.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.669+0000 7f52fcff9640 1 -- 172.21.15.84:0/331344829 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52f8044020 con 0x7f5300150300 2023-10-11T06:13:23.671 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:13:23.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f52e006a830 msgr2=0x7f52e006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f52e006a830 0x7f52e006cce0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52f0001610 tx=0x7f52f0007930 comp rx=0 tx=0).stop 2023-10-11T06:13:23.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5300150300 msgr2=0x7f53000c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5300150300 0x7f53000c6390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f8004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:23.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 shutdown_connections 2023-10-11T06:13:23.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f52e006a830 0x7f52e006cce0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5300150300 0x7f53000c6390 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f530014ff30 0x7f53000c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 --2- 172.21.15.84:0/331344829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53000cec20 0x7f530014e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 >> 172.21.15.84:0/331344829 conn(0x7f5300069c70 msgr2=0x7f53000b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 shutdown_connections 2023-10-11T06:13:23.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.672+0000 7f530622d640 1 -- 172.21.15.84:0/331344829 wait complete. 2023-10-11T06:13:23.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:13:23.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.911+0000 7f29f16b4640 1 Processor -- start 2023-10-11T06:13:23.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.912+0000 7f29f16b4640 1 -- start start 2023-10-11T06:13:23.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.913+0000 7f29f16b4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec15f7e0 0x7f29ec0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.913+0000 7f29f16b4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29ec0cd6c0 0x7f29ec0d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.913+0000 7f29f16b4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f29ec0d3000 0x7f29ec15fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.913+0000 7f29f16b4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec0d3460 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.913+0000 7f29f16b4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec0cf220 con 0x7f29ec0d3000 2023-10-11T06:13:23.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.913+0000 7f29f16b4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0cf500 con 0x7f29ec15f7e0 2023-10-11T06:13:23.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29ec0cd6c0 0x7f29ec0d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29ec0cd6c0 0x7f29ec0d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46168/0 (socket says 172.21.15.84:46168) 2023-10-11T06:13:23.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 -- 172.21.15.84:0/2069045904 learned_addr learned my addr 172.21.15.84:0/2069045904 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:23.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29eaffd640 1 --2- 172.21.15.84:0/2069045904 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec15f7e0 0x7f29ec0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29eb7fe640 1 --2- 172.21.15.84:0/2069045904 >> v2:172.21.15.99:3300/0 conn(0x7f29ec0d3000 0x7f29ec15fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 -- 172.21.15.84:0/2069045904 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec15f7e0 msgr2=0x7f29ec0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:23.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 --2- 172.21.15.84:0/2069045904 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec15f7e0 0x7f29ec0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 -- 172.21.15.84:0/2069045904 >> v2:172.21.15.99:3300/0 conn(0x7f29ec0d3000 msgr2=0x7f29ec15fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:23.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 --2- 172.21.15.84:0/2069045904 >> v2:172.21.15.99:3300/0 conn(0x7f29ec0d3000 0x7f29ec15fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.914+0000 7f29e27fc640 1 -- 172.21.15.84:0/2069045904 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0cf7e0 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.915+0000 7f29e27fc640 1 --2- 172.21.15.84:0/2069045904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0d2ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29ec08bef0 tx=0x7f29d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42fa0a414fbb522 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29ea7fc640 1 -- 172.21.15.84:0/2069045904 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4031870 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29ea7fc640 1 -- 172.21.15.84:0/2069045904 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d40319d0 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29ea7fc640 1 -- 172.21.15.84:0/2069045904 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4031d60 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 -- 172.21.15.84:0/2069045904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 msgr2=0x7f29ec0d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2069045904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0d2ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29ec08bef0 tx=0x7f29d402b910 comp rx=0 tx=0).stop 2023-10-11T06:13:23.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 -- 172.21.15.84:0/2069045904 shutdown_connections 2023-10-11T06:13:23.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2069045904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0d2ac0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2069045904 >> v2:172.21.15.99:3300/0 conn(0x7f29ec0d3000 0x7f29ec15fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2069045904 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec15f7e0 0x7f29ec0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 -- 172.21.15.84:0/2069045904 >> 172.21.15.84:0/2069045904 conn(0x7f29ec0713a0 msgr2=0x7f29ec0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:23.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.916+0000 7f29f16b4640 1 -- 172.21.15.84:0/2069045904 shutdown_connections 2023-10-11T06:13:23.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.917+0000 7f29f16b4640 1 -- 172.21.15.84:0/2069045904 wait complete. 2023-10-11T06:13:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.918+0000 7f29f16b4640 1 Processor -- start 2023-10-11T06:13:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.918+0000 7f29f16b4640 1 -- start start 2023-10-11T06:13:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.919+0000 7f29f16b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.919+0000 7f29f16b4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec0d3000 0x7f29ec0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.919+0000 7f29f16b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15f7e0 0x7f29ec1f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.919+0000 7f29f16b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0b72d0 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.919+0000 7f29f16b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0b6370 con 0x7f29ec15f7e0 2023-10-11T06:13:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.919+0000 7f29f16b4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0b6650 con 0x7f29ec0d3000 2023-10-11T06:13:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46180/0 (socket says 172.21.15.84:46180) 2023-10-11T06:13:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 -- 172.21.15.84:0/2898585990 learned_addr learned my addr 172.21.15.84:0/2898585990 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29e27fc640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec0d3000 0x7f29ec0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 -- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec0d3000 msgr2=0x7f29ec0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec0d3000 0x7f29ec0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eb7fe640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15f7e0 0x7f29ec1f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 -- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15f7e0 msgr2=0x7f29ec1f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15f7e0 0x7f29ec1f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 -- 172.21.15.84:0/2898585990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40094d0 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.920+0000 7f29eaffd640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0b8230 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00d5d0 tx=0x7f29dc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.921+0000 7f29e3fff640 1 -- 172.21.15.84:0/2898585990 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0125f0 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.921+0000 7f29e3fff640 1 -- 172.21.15.84:0/2898585990 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc012770 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.921+0000 7f29e3fff640 1 -- 172.21.15.84:0/2898585990 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc012b20 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.921+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec0b6930 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.921+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec0b6ef0 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.922+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec0cee40 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.923+0000 7f29e3fff640 1 -- 172.21.15.84:0/2898585990 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f29dc012c80 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.924+0000 7f29e3fff640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f29cc06a830 0x7f29cc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.925+0000 7f29e3fff640 1 -- 172.21.15.84:0/2898585990 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f29dc089630 con 0x7f29ec0cd6c0 2023-10-11T06:13:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.925+0000 7f29e27fc640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f29cc06a830 0x7f29cc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.925+0000 7f29e27fc640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f29cc06a830 0x7f29cc06cce0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f29d4036000 tx=0x7f29d403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:23.929+0000 7f29e3fff640 1 -- 172.21.15.84:0/2898585990 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f29dc0546f0 con 0x7f29ec0cd6c0 2023-10-11T06:13:24.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.118+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29ec0c9be0 con 0x7f29ec0cd6c0 2023-10-11T06:13:24.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.119+0000 7f29e3fff640 1 -- 172.21.15.84:0/2898585990 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29dc057d10 con 0x7f29ec0cd6c0 2023-10-11T06:13:24.121 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:24.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f29cc06a830 msgr2=0x7f29cc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f29cc06a830 0x7f29cc06cce0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f29d4036000 tx=0x7f29d403c040 comp rx=0 tx=0).stop 2023-10-11T06:13:24.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 msgr2=0x7f29ec0b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0b8230 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00d5d0 tx=0x7f29dc00da90 comp rx=0 tx=0).stop 2023-10-11T06:13:24.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 shutdown_connections 2023-10-11T06:13:24.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f29cc06a830 0x7f29cc06cce0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15f7e0 0x7f29ec1f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29ec0d3000 0x7f29ec0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 --2- 172.21.15.84:0/2898585990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29ec0cd6c0 0x7f29ec0b8230 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 >> 172.21.15.84:0/2898585990 conn(0x7f29ec0713a0 msgr2=0x7f29ec15ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:24.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 shutdown_connections 2023-10-11T06:13:24.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.122+0000 7f29f16b4640 1 -- 172.21.15.84:0/2898585990 wait complete. 2023-10-11T06:13:24.143 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:13:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.361+0000 7f115961c640 1 Processor -- start 2023-10-11T06:13:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.361+0000 7f115961c640 1 -- start start 2023-10-11T06:13:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.362+0000 7f115961c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f115414ff30 0x7f11540c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.362+0000 7f115961c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1154150300 0x7f11540cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f115961c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11540c9b90 con 0x7f11540cc7c0 2023-10-11T06:13:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f115961c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115415da10 con 0x7f1154150300 2023-10-11T06:13:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f115961c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115415ce00 con 0x7f115414ff30 2023-10-11T06:13:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f11527fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1154150300 0x7f11540cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f11537fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f11540cc7c0 0x7f11540ce510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55632/0 (socket says 172.21.15.84:55632) 2023-10-11T06:13:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f11537fe640 1 -- 172.21.15.84:0/131171216 learned_addr learned my addr 172.21.15.84:0/131171216 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f1152ffd640 1 --2- 172.21.15.84:0/131171216 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f115414ff30 0x7f11540c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f11527fc640 1 -- 172.21.15.84:0/131171216 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f115414ff30 msgr2=0x7f11540c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f11527fc640 1 --2- 172.21.15.84:0/131171216 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f115414ff30 0x7f11540c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f11527fc640 1 -- 172.21.15.84:0/131171216 >> v1:172.21.15.84:6789/0 conn(0x7f11540cc7c0 legacy=0x7f11540ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f11527fc640 1 -- 172.21.15.84:0/131171216 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115415d0e0 con 0x7f1154150300 2023-10-11T06:13:24.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.363+0000 7f1151ffb640 1 -- 172.21.15.84:0/131171216 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438402149 0 0) 0x7f11540c9b90 con 0x7f11540cc7c0 2023-10-11T06:13:24.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.365+0000 7f11527fc640 1 --2- 172.21.15.84:0/131171216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540cc280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f113c0097d0 tx=0x7f113c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d91237dd12508f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:24.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.365+0000 7f1151ffb640 1 -- 172.21.15.84:0/131171216 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c02e070 con 0x7f1154150300 2023-10-11T06:13:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.365+0000 7f1151ffb640 1 -- 172.21.15.84:0/131171216 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c0315a0 con 0x7f1154150300 2023-10-11T06:13:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.365+0000 7f1151ffb640 1 -- 172.21.15.84:0/131171216 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c031930 con 0x7f1154150300 2023-10-11T06:13:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.365+0000 7f115961c640 1 -- 172.21.15.84:0/131171216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 msgr2=0x7f11540cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.365+0000 7f115961c640 1 --2- 172.21.15.84:0/131171216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540cc280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f113c0097d0 tx=0x7f113c02b840 comp rx=0 tx=0).stop 2023-10-11T06:13:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.366+0000 7f115961c640 1 -- 172.21.15.84:0/131171216 shutdown_connections 2023-10-11T06:13:24.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.366+0000 7f115961c640 1 --2- 172.21.15.84:0/131171216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540cc280 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.366+0000 7f115961c640 1 --2- 172.21.15.84:0/131171216 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f115414ff30 0x7f11540c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.366+0000 7f115961c640 1 -- 172.21.15.84:0/131171216 >> 172.21.15.84:0/131171216 conn(0x7f1154069c70 msgr2=0x7f115406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:24.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.366+0000 7f115961c640 1 -- 172.21.15.84:0/131171216 shutdown_connections 2023-10-11T06:13:24.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.366+0000 7f115961c640 1 -- 172.21.15.84:0/131171216 wait complete. 2023-10-11T06:13:24.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.367+0000 7f115961c640 1 Processor -- start 2023-10-11T06:13:24.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.368+0000 7f115961c640 1 -- start start 2023-10-11T06:13:24.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.368+0000 7f115961c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115414ff30 0x7f115414dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.368+0000 7f115961c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.368+0000 7f115961c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11540af050 0x7f11540cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.368+0000 7f115961c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1154151350 con 0x7f1154150300 2023-10-11T06:13:24.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.368+0000 7f115961c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11540af4d0 con 0x7f115414ff30 2023-10-11T06:13:24.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.368+0000 7f115961c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1154151770 con 0x7f11540af050 2023-10-11T06:13:24.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46192/0 (socket says 172.21.15.84:46192) 2023-10-11T06:13:24.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 -- 172.21.15.84:0/2995229383 learned_addr learned my addr 172.21.15.84:0/2995229383 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:24.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11537fe640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11540af050 0x7f11540cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f1152ffd640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115414ff30 0x7f115414dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 -- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11540af050 msgr2=0x7f11540cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11540af050 0x7f11540cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 -- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115414ff30 msgr2=0x7f115414dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115414ff30 0x7f115414dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11527fc640 1 -- 172.21.15.84:0/2995229383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c009400 con 0x7f1154150300 2023-10-11T06:13:24.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.369+0000 7f11537fe640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11540af050 0x7f11540cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:24.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.370+0000 7f11527fc640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540aeb10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f113c000c00 tx=0x7f113c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:24.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.370+0000 7f114b7fe640 1 -- 172.21.15.84:0/2995229383 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c032900 con 0x7f1154150300 2023-10-11T06:13:24.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.370+0000 7f114b7fe640 1 -- 172.21.15.84:0/2995229383 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c032a80 con 0x7f1154150300 2023-10-11T06:13:24.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.370+0000 7f114b7fe640 1 -- 172.21.15.84:0/2995229383 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c032e30 con 0x7f1154150300 2023-10-11T06:13:24.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.370+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11541506d0 con 0x7f1154150300 2023-10-11T06:13:24.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.370+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1154150c90 con 0x7f1154150300 2023-10-11T06:13:24.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.371+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1154151490 con 0x7f1154150300 2023-10-11T06:13:24.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.372+0000 7f114b7fe640 1 -- 172.21.15.84:0/2995229383 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f113c04d3f0 con 0x7f1154150300 2023-10-11T06:13:24.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.373+0000 7f114b7fe640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f113406a830 0x7f113406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.373+0000 7f114b7fe640 1 -- 172.21.15.84:0/2995229383 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f113c0aad10 con 0x7f1154150300 2023-10-11T06:13:24.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.373+0000 7f1152ffd640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f113406a830 0x7f113406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.374+0000 7f1152ffd640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f113406a830 0x7f113406cce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1144006fd0 tx=0x7f1144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:24.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.378+0000 7f114b7fe640 1 -- 172.21.15.84:0/2995229383 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1154151490 con 0x7f1154150300 2023-10-11T06:13:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.567+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1154151490 con 0x7f1154150300 2023-10-11T06:13:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.568+0000 7f114b7fe640 1 -- 172.21.15.84:0/2995229383 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1154151490 con 0x7f1154150300 2023-10-11T06:13:24.569 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:13:24.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f113406a830 msgr2=0x7f113406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f113406a830 0x7f113406cce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1144006fd0 tx=0x7f1144008040 comp rx=0 tx=0).stop 2023-10-11T06:13:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 msgr2=0x7f11540aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540aeb10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f113c000c00 tx=0x7f113c0046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 shutdown_connections 2023-10-11T06:13:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f113406a830 0x7f113406cce0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11540af050 0x7f11540cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1154150300 0x7f11540aeb10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 --2- 172.21.15.84:0/2995229383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115414ff30 0x7f115414dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 >> 172.21.15.84:0/2995229383 conn(0x7f1154069c70 msgr2=0x7f11540cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 shutdown_connections 2023-10-11T06:13:24.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.571+0000 7f115961c640 1 -- 172.21.15.84:0/2995229383 wait complete. 2023-10-11T06:13:24.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-11T06:13:24.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.801+0000 7f1e88077640 1 Processor -- start 2023-10-11T06:13:24.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.801+0000 7f1e88077640 1 -- start start 2023-10-11T06:13:24.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.803+0000 7f1e88077640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e800cb5a0 0x7f1e800c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.803+0000 7f1e88077640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e800c9240 con 0x7f1e800ced30 2023-10-11T06:13:24.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.803+0000 7f1e88077640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e80071380 con 0x7f1e8015a0b0 2023-10-11T06:13:24.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.803+0000 7f1e88077640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8006e0f0 con 0x7f1e800cb5a0 2023-10-11T06:13:24.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.803+0000 7f1e85612640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e800ced30 0x7f1e8006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55640/0 (socket says 172.21.15.84:55640) 2023-10-11T06:13:24.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.803+0000 7f1e85612640 1 -- 172.21.15.84:0/4182369800 learned_addr learned my addr 172.21.15.84:0/4182369800 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:24.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.803+0000 7f1e86614640 1 --2- 172.21.15.84:0/4182369800 >> v2:172.21.15.99:3300/0 conn(0x7f1e800cb5a0 0x7f1e800c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290280381 0 0) 0x7f1e70001110 con 0x7f1e800ced30 2023-10-11T06:13:24.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e6c003520 con 0x7f1e800ced30 2023-10-11T06:13:24.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4076447943 0 0) 0x7f1e6c003520 con 0x7f1e800ced30 2023-10-11T06:13:24.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 >> v2:172.21.15.99:3300/0 conn(0x7f1e800cb5a0 msgr2=0x7f1e800c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:24.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 --2- 172.21.15.84:0/4182369800 >> v2:172.21.15.99:3300/0 conn(0x7f1e800cb5a0 0x7f1e800c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 >> v1:172.21.15.99:6789/0 conn(0x7f1e8015a0b0 legacy=0x7f1e800ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:24.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8006de10 con 0x7f1e800ced30 2023-10-11T06:13:24.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f1e70001800 con 0x7f1e800ced30 2023-10-11T06:13:24.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 >> v1:172.21.15.84:6789/0 conn(0x7f1e800ced30 legacy=0x7f1e8006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c003970 0x7f1e6c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c0047c0 0x7f1e6c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c0051e0 0x7f1e6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8006e0f0 con 0x7f1e6c003970 2023-10-11T06:13:24.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c00c120 con 0x7f1e6c0047c0 2023-10-11T06:13:24.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c00c2e0 con 0x7f1e6c0051e0 2023-10-11T06:13:24.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.804+0000 7f1e85e13640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c003970 0x7f1e6c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.805+0000 7f1e85e13640 1 -- 172.21.15.84:0/4182369800 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c0051e0 msgr2=0x7f1e6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:24.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.805+0000 7f1e85e13640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c0051e0 0x7f1e6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.805+0000 7f1e85e13640 1 -- 172.21.15.84:0/4182369800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c0047c0 msgr2=0x7f1e6c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:24.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.805+0000 7f1e85e13640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6c0047c0 0x7f1e6c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.805+0000 7f1e85e13640 1 -- 172.21.15.84:0/4182369800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c003520 con 0x7f1e6c003970 2023-10-11T06:13:24.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.805+0000 7f1e86614640 1 -- 172.21.15.84:0/4182369800 reap_dead start 2023-10-11T06:13:24.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.806+0000 7f1e85e13640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c003970 0x7f1e6c003d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c003db0 tx=0x7f1e6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d644f6213d9d5c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:24.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.806+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c032600 con 0x7f1e6c003970 2023-10-11T06:13:24.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.806+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c02e050 con 0x7f1e6c003970 2023-10-11T06:13:24.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.806+0000 7f1e84e11640 1 -- 172.21.15.84:0/4182369800 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0329e0 con 0x7f1e6c003970 2023-10-11T06:13:24.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.807+0000 7f1e88077640 1 -- 172.21.15.84:0/4182369800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c003970 msgr2=0x7f1e6c003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.807+0000 7f1e88077640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c003970 0x7f1e6c003d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c003db0 tx=0x7f1e6c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.807+0000 7f1e88077640 1 -- 172.21.15.84:0/4182369800 shutdown_connections 2023-10-11T06:13:24.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.807+0000 7f1e88077640 1 --2- 172.21.15.84:0/4182369800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c003970 0x7f1e6c003d60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.807+0000 7f1e88077640 1 -- 172.21.15.84:0/4182369800 >> 172.21.15.84:0/4182369800 conn(0x7f1e80069c70 msgr2=0x7f1e8006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:24.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.807+0000 7f1e88077640 1 -- 172.21.15.84:0/4182369800 shutdown_connections 2023-10-11T06:13:24.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.807+0000 7f1e88077640 1 -- 172.21.15.84:0/4182369800 wait complete. 2023-10-11T06:13:24.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.808+0000 7f1e88077640 1 Processor -- start 2023-10-11T06:13:24.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.809+0000 7f1e88077640 1 -- start start 2023-10-11T06:13:24.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.809+0000 7f1e88077640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c003970 0x7f1e8014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.809+0000 7f1e88077640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e800cd2d0 0x7f1e80155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e88077640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e800c63b0 0x7f1e800c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e88077640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e801571a0 con 0x7f1e800c63b0 2023-10-11T06:13:24.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e88077640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800c6d60 con 0x7f1e800cd2d0 2023-10-11T06:13:24.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e88077640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800c5780 con 0x7f1e6c003970 2023-10-11T06:13:24.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e800c63b0 0x7f1e800c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e800c63b0 0x7f1e800c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46210/0 (socket says 172.21.15.84:46210) 2023-10-11T06:13:24.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 -- 172.21.15.84:0/2005098712 learned_addr learned my addr 172.21.15.84:0/2005098712 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:24.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e85e13640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c003970 0x7f1e8014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e85612640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e800cd2d0 0x7f1e80155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 -- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c003970 msgr2=0x7f1e8014dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c003970 0x7f1e8014dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 -- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e800cd2d0 msgr2=0x7f1e80155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:24.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e800cd2d0 0x7f1e80155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:24.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.810+0000 7f1e86614640 1 -- 172.21.15.84:0/2005098712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800c5a60 con 0x7f1e800c63b0 2023-10-11T06:13:24.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.811+0000 7f1e86614640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e800c63b0 0x7f1e800c6820 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1e800c8c80 tx=0x7f1e7800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:24.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.811+0000 7f1e767fc640 1 -- 172.21.15.84:0/2005098712 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78010040 con 0x7f1e800c63b0 2023-10-11T06:13:24.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.811+0000 7f1e767fc640 1 -- 172.21.15.84:0/2005098712 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e78014810 con 0x7f1e800c63b0 2023-10-11T06:13:24.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.811+0000 7f1e767fc640 1 -- 172.21.15.84:0/2005098712 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78009b10 con 0x7f1e800c63b0 2023-10-11T06:13:24.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.811+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e800c5d40 con 0x7f1e800c63b0 2023-10-11T06:13:24.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.811+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e80156c80 con 0x7f1e800c63b0 2023-10-11T06:13:24.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.812+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e80073ed0 con 0x7f1e800c63b0 2023-10-11T06:13:24.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.813+0000 7f1e767fc640 1 -- 172.21.15.84:0/2005098712 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1e78014bc0 con 0x7f1e800c63b0 2023-10-11T06:13:24.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.814+0000 7f1e767fc640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e6406a830 0x7f1e6406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:24.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.814+0000 7f1e767fc640 1 -- 172.21.15.84:0/2005098712 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f1e78088d90 con 0x7f1e800c63b0 2023-10-11T06:13:24.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.814+0000 7f1e85e13640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e6406a830 0x7f1e6406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:24.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.815+0000 7f1e85e13640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e6406a830 0x7f1e6406cce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c034000 tx=0x7f1e7c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:24.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:24.818+0000 7f1e767fc640 1 -- 172.21.15.84:0/2005098712 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1e78053dc0 con 0x7f1e800c63b0 2023-10-11T06:13:25.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.010+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f1e80070760 con 0x7f1e800c63b0 2023-10-11T06:13:25.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.011+0000 7f1e767fc640 1 -- 172.21.15.84:0/2005098712 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) v1 ==== 97+0+11 (secure 0 0 0) 0x7f1e780573e0 con 0x7f1e800c63b0 2023-10-11T06:13:25.013 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:25.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.014+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e6406a830 msgr2=0x7f1e6406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.014+0000 7f1e88077640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e6406a830 0x7f1e6406cce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c034000 tx=0x7f1e7c009030 comp rx=0 tx=0).stop 2023-10-11T06:13:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.014+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e800c63b0 msgr2=0x7f1e800c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.014+0000 7f1e88077640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e800c63b0 0x7f1e800c6820 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1e800c8c80 tx=0x7f1e7800ef10 comp rx=0 tx=0).stop 2023-10-11T06:13:25.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 shutdown_connections 2023-10-11T06:13:25.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f1e6406a830 0x7f1e6406cce0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e800c63b0 0x7f1e800c6820 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e800cd2d0 0x7f1e80155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 --2- 172.21.15.84:0/2005098712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e6c003970 0x7f1e8014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 >> 172.21.15.84:0/2005098712 conn(0x7f1e80069c70 msgr2=0x7f1e8006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:25.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 shutdown_connections 2023-10-11T06:13:25.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.015+0000 7f1e88077640 1 -- 172.21.15.84:0/2005098712 wait complete. 2023-10-11T06:13:25.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:13:25.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.237+0000 7f9e5c57e640 1 Processor -- start 2023-10-11T06:13:25.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.237+0000 7f9e5c57e640 1 -- start start 2023-10-11T06:13:25.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e5c57e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9e5415c960 0x7f9e540ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e5c57e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e541610d0 0x7f9e540cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e5c57e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e5406cb00 0x7f9e540d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e5c57e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e540cdc90 con 0x7f9e541610d0 2023-10-11T06:13:25.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e5c57e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5415cd30 con 0x7f9e5415c960 2023-10-11T06:13:25.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e5c57e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541603a0 con 0x7f9e5406cb00 2023-10-11T06:13:25.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e59b19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e541610d0 0x7f9e540cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e59b19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e541610d0 0x7f9e540cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46212/0 (socket says 172.21.15.84:46212) 2023-10-11T06:13:25.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.239+0000 7f9e59b19640 1 -- 172.21.15.84:0/450931654 learned_addr learned my addr 172.21.15.84:0/450931654 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:25.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.240+0000 7f9e59b19640 1 -- 172.21.15.84:0/450931654 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e5406cb00 msgr2=0x7f9e540d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:25.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.240+0000 7f9e5ab1b640 1 --2- 172.21.15.84:0/450931654 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e5406cb00 0x7f9e540d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.240+0000 7f9e5a31a640 1 --2- 172.21.15.84:0/450931654 >> v2:172.21.15.99:3300/0 conn(0x7f9e5415c960 0x7f9e540ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.240+0000 7f9e59b19640 1 --2- 172.21.15.84:0/450931654 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e5406cb00 0x7f9e540d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.240+0000 7f9e59b19640 1 -- 172.21.15.84:0/450931654 >> v2:172.21.15.99:3300/0 conn(0x7f9e5415c960 msgr2=0x7f9e540ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:25.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.240+0000 7f9e59b19640 1 --2- 172.21.15.84:0/450931654 >> v2:172.21.15.99:3300/0 conn(0x7f9e5415c960 0x7f9e540ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.240+0000 7f9e59b19640 1 -- 172.21.15.84:0/450931654 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5406c060 con 0x7f9e541610d0 2023-10-11T06:13:25.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.241+0000 7f9e59b19640 1 --2- 172.21.15.84:0/450931654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e541610d0 0x7f9e540cea10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9e5415ce70 tx=0x7f9e4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eab42dfe0f0615 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:25.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.241+0000 7f9e59318640 1 -- 172.21.15.84:0/450931654 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44031640 con 0x7f9e541610d0 2023-10-11T06:13:25.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.241+0000 7f9e59318640 1 -- 172.21.15.84:0/450931654 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e440317a0 con 0x7f9e541610d0 2023-10-11T06:13:25.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.241+0000 7f9e59318640 1 -- 172.21.15.84:0/450931654 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44031b30 con 0x7f9e541610d0 2023-10-11T06:13:25.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 -- 172.21.15.84:0/450931654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e541610d0 msgr2=0x7f9e540cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/450931654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e541610d0 0x7f9e540cea10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9e5415ce70 tx=0x7f9e4402b8f0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 -- 172.21.15.84:0/450931654 shutdown_connections 2023-10-11T06:13:25.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/450931654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e541610d0 0x7f9e540cea10 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/450931654 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e5406cb00 0x7f9e540d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/450931654 >> v2:172.21.15.99:3300/0 conn(0x7f9e5415c960 0x7f9e540ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 -- 172.21.15.84:0/450931654 >> 172.21.15.84:0/450931654 conn(0x7f9e5406e2a0 msgr2=0x7f9e5406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:25.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 -- 172.21.15.84:0/450931654 shutdown_connections 2023-10-11T06:13:25.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.242+0000 7f9e5c57e640 1 -- 172.21.15.84:0/450931654 wait complete. 2023-10-11T06:13:25.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.244+0000 7f9e5c57e640 1 Processor -- start 2023-10-11T06:13:25.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.244+0000 7f9e5c57e640 1 -- start start 2023-10-11T06:13:25.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5c57e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5406cb00 0x7f9e54156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5c57e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5415c960 0x7f9e540ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5c57e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e541610d0 0x7f9e5415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5c57e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e54156f60 con 0x7f9e5406cb00 2023-10-11T06:13:25.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5c57e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541f1d40 con 0x7f9e5415c960 2023-10-11T06:13:25.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5c57e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541f1fe0 con 0x7f9e541610d0 2023-10-11T06:13:25.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5a31a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5406cb00 0x7f9e54156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5a31a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5406cb00 0x7f9e54156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46214/0 (socket says 172.21.15.84:46214) 2023-10-11T06:13:25.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5a31a640 1 -- 172.21.15.84:0/2523244566 learned_addr learned my addr 172.21.15.84:0/2523244566 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:25.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.245+0000 7f9e5a31a640 1 -- 172.21.15.84:0/2523244566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e541610d0 msgr2=0x7f9e5415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e5a31a640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e541610d0 0x7f9e5415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e5a31a640 1 -- 172.21.15.84:0/2523244566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5415c960 msgr2=0x7f9e540ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e59b19640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5415c960 0x7f9e540ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e5a31a640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5415c960 0x7f9e540ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e5a31a640 1 -- 172.21.15.84:0/2523244566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440094d0 con 0x7f9e5406cb00 2023-10-11T06:13:25.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e5a31a640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5406cb00 0x7f9e54156a20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000c7f0 tx=0x7f9e5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:25.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e4affd640 1 -- 172.21.15.84:0/2523244566 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50004220 con 0x7f9e5406cb00 2023-10-11T06:13:25.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e4affd640 1 -- 172.21.15.84:0/2523244566 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e500043a0 con 0x7f9e5406cb00 2023-10-11T06:13:25.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e541f22c0 con 0x7f9e5406cb00 2023-10-11T06:13:25.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e4affd640 1 -- 172.21.15.84:0/2523244566 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50007860 con 0x7f9e5406cb00 2023-10-11T06:13:25.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.246+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e541f2ac0 con 0x7f9e5406cb00 2023-10-11T06:13:25.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.247+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e54065cd0 con 0x7f9e5406cb00 2023-10-11T06:13:25.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.248+0000 7f9e4affd640 1 -- 172.21.15.84:0/2523244566 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9e50007bd0 con 0x7f9e5406cb00 2023-10-11T06:13:25.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.249+0000 7f9e4affd640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9e3806a830 0x7f9e3806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.249+0000 7f9e4affd640 1 -- 172.21.15.84:0/2523244566 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f9e50089220 con 0x7f9e5406cb00 2023-10-11T06:13:25.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.249+0000 7f9e59b19640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9e3806a830 0x7f9e3806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.250+0000 7f9e59b19640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9e3806a830 0x7f9e3806cce0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e44036000 tx=0x7f9e4403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:25.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.254+0000 7f9e4affd640 1 -- 172.21.15.84:0/2523244566 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9e500542e0 con 0x7f9e5406cb00 2023-10-11T06:13:25.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.457+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9e54073ed0 con 0x7f9e5406cb00 2023-10-11T06:13:25.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.459+0000 7f9e4affd640 1 -- 172.21.15.84:0/2523244566 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9e50057900 con 0x7f9e5406cb00 2023-10-11T06:13:25.460 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:13:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9e3806a830 msgr2=0x7f9e3806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9e3806a830 0x7f9e3806cce0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e44036000 tx=0x7f9e4403c040 comp rx=0 tx=0).stop 2023-10-11T06:13:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5406cb00 msgr2=0x7f9e54156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5406cb00 0x7f9e54156a20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000c7f0 tx=0x7f9e5000ccb0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 shutdown_connections 2023-10-11T06:13:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f9e3806a830 0x7f9e3806cce0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e541610d0 0x7f9e5415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5415c960 0x7f9e540ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 --2- 172.21.15.84:0/2523244566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e5406cb00 0x7f9e54156a20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.461+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 >> 172.21.15.84:0/2523244566 conn(0x7f9e5406e2a0 msgr2=0x7f9e540ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:25.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.462+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 shutdown_connections 2023-10-11T06:13:25.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.462+0000 7f9e5c57e640 1 -- 172.21.15.84:0/2523244566 wait complete. 2023-10-11T06:13:25.480 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:13:25.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:25.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:25.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[104533]: fuse finished with error 0 and tester_r 0 2023-10-11T06:13:25.576 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:13:25.615 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:13:25.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:25.616 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:13:25.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-10-11T06:13:25.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.935+0000 7f961c577640 1 Processor -- start 2023-10-11T06:13:25.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.935+0000 7f961c577640 1 -- start start 2023-10-11T06:13:25.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.937+0000 7f961c577640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96140d0c10 0x7f961415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.937+0000 7f961c577640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961406b790 con 0x7f96141602d0 2023-10-11T06:13:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.937+0000 7f961c577640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140af580 con 0x7f96141568a0 2023-10-11T06:13:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.937+0000 7f961c577640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9614160b00 con 0x7f96140d0c10 2023-10-11T06:13:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.937+0000 7f9619b12640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96140d0c10 0x7f961415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.937+0000 7f961ab14640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f96141602d0 0x7f961406bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55656/0 (socket says 172.21.15.84:55656) 2023-10-11T06:13:25.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.938+0000 7f961ab14640 1 -- 172.21.15.84:0/1663418569 learned_addr learned my addr 172.21.15.84:0/1663418569 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.938+0000 7f9619b12640 1 -- 172.21.15.84:0/1663418569 >> v1:172.21.15.99:6789/0 conn(0x7f96141568a0 legacy=0x7f96140d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.938+0000 7f9619b12640 1 -- 172.21.15.84:0/1663418569 >> v1:172.21.15.84:6789/0 conn(0x7f96141602d0 legacy=0x7f961406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.938+0000 7f9619b12640 1 -- 172.21.15.84:0/1663418569 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961406ae60 con 0x7f96140d0c10 2023-10-11T06:13:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.938+0000 7f9619311640 1 -- 172.21.15.84:0/1663418569 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955060424 0 0) 0x7f961406b790 con 0x7f96141602d0 2023-10-11T06:13:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.939+0000 7f9619b12640 1 --2- 172.21.15.84:0/1663418569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96140d0c10 0x7f961415fd90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f96040097a0 tx=0x7f960402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7d0df2da4a122d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.939+0000 7f9619311640 1 -- 172.21.15.84:0/1663418569 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604031830 con 0x7f96140d0c10 2023-10-11T06:13:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.939+0000 7f9619311640 1 -- 172.21.15.84:0/1663418569 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9604031990 con 0x7f96140d0c10 2023-10-11T06:13:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.939+0000 7f9619311640 1 -- 172.21.15.84:0/1663418569 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604031d20 con 0x7f96140d0c10 2023-10-11T06:13:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.940+0000 7f961c577640 1 -- 172.21.15.84:0/1663418569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96140d0c10 msgr2=0x7f961415fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.940+0000 7f961c577640 1 --2- 172.21.15.84:0/1663418569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96140d0c10 0x7f961415fd90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f96040097a0 tx=0x7f960402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.940+0000 7f961c577640 1 -- 172.21.15.84:0/1663418569 shutdown_connections 2023-10-11T06:13:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.940+0000 7f961c577640 1 --2- 172.21.15.84:0/1663418569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96140d0c10 0x7f961415fd90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.940+0000 7f961c577640 1 -- 172.21.15.84:0/1663418569 >> 172.21.15.84:0/1663418569 conn(0x7f9614071cc0 msgr2=0x7f961409bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:25.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.940+0000 7f961c577640 1 -- 172.21.15.84:0/1663418569 shutdown_connections 2023-10-11T06:13:25.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.940+0000 7f961c577640 1 -- 172.21.15.84:0/1663418569 wait complete. 2023-10-11T06:13:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.942+0000 7f961c577640 1 Processor -- start 2023-10-11T06:13:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.942+0000 7f961c577640 1 -- start start 2023-10-11T06:13:25.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.943+0000 7f961c577640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96140d0c10 0x7f96140b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.943+0000 7f961c577640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96141568a0 0x7f96141e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.943+0000 7f961c577640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96141602d0 0x7f96141519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.943+0000 7f961c577640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141e8120 con 0x7f96141602d0 2023-10-11T06:13:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.943+0000 7f961c577640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141e7510 con 0x7f96140d0c10 2023-10-11T06:13:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.943+0000 7f961c577640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96141e77f0 con 0x7f96141568a0 2023-10-11T06:13:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.943+0000 7f961ab14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96141602d0 0x7f96141519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96141602d0 0x7f96141519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46230/0 (socket says 172.21.15.84:46230) 2023-10-11T06:13:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 -- 172.21.15.84:0/2616478809 learned_addr learned my addr 172.21.15.84:0/2616478809 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 -- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96141568a0 msgr2=0x7f96141e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f9619b12640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96141568a0 0x7f96141e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961a313640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96140d0c10 0x7f96140b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96141568a0 0x7f96141e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 -- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96140d0c10 msgr2=0x7f96140b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96140d0c10 0x7f96140b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:25.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 -- 172.21.15.84:0/2616478809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9604009400 con 0x7f96141602d0 2023-10-11T06:13:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f961ab14640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96141602d0 0x7f96141519b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c7e0 tx=0x7f960c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f960affd640 1 -- 172.21.15.84:0/2616478809 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c00a640 con 0x7f96141602d0 2023-10-11T06:13:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.944+0000 7f960affd640 1 -- 172.21.15.84:0/2616478809 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c00a7c0 con 0x7f96141602d0 2023-10-11T06:13:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.945+0000 7f960affd640 1 -- 172.21.15.84:0/2616478809 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c00ab70 con 0x7f96141602d0 2023-10-11T06:13:25.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.945+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96141e7ad0 con 0x7f96141602d0 2023-10-11T06:13:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.945+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96140b46d0 con 0x7f96141602d0 2023-10-11T06:13:25.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.946+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96140b48c0 con 0x7f96141602d0 2023-10-11T06:13:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.947+0000 7f960affd640 1 -- 172.21.15.84:0/2616478809 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f960c00acd0 con 0x7f96141602d0 2023-10-11T06:13:25.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.948+0000 7f960affd640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95f806a830 0x7f95f806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.948+0000 7f960affd640 1 -- 172.21.15.84:0/2616478809 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f960c088750 con 0x7f96141602d0 2023-10-11T06:13:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.948+0000 7f961a313640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95f806a830 0x7f95f806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.949+0000 7f961a313640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95f806a830 0x7f95f806cce0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9610003e00 tx=0x7f9610009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:25.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:25.953+0000 7f960affd640 1 -- 172.21.15.84:0/2616478809 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f960c053810 con 0x7f96141602d0 2023-10-11T06:13:26.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.216+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f96140c6460 con 0x7f96141602d0 2023-10-11T06:13:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.224+0000 7f960affd640 1 -- 172.21.15.84:0/2616478809 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f96140c6460 con 0x7f96141602d0 2023-10-11T06:13:26.225 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:13:26.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95f806a830 msgr2=0x7f95f806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:26.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95f806a830 0x7f95f806cce0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9610003e00 tx=0x7f9610009040 comp rx=0 tx=0).stop 2023-10-11T06:13:26.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96141602d0 msgr2=0x7f96141519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:26.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96141602d0 0x7f96141519b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c7e0 tx=0x7f960c00cca0 comp rx=0 tx=0).stop 2023-10-11T06:13:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 shutdown_connections 2023-10-11T06:13:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95f806a830 0x7f95f806cce0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96141602d0 0x7f96141519b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96141568a0 0x7f96141e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 --2- 172.21.15.84:0/2616478809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96140d0c10 0x7f96140b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:26.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 >> 172.21.15.84:0/2616478809 conn(0x7f9614071cc0 msgr2=0x7f96140b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:26.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 shutdown_connections 2023-10-11T06:13:26.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:26.227+0000 7f961c577640 1 -- 172.21.15.84:0/2616478809 wait complete. 2023-10-11T06:13:26.246 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T06:13:26.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:13:26.361 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:13:26.485 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T06:13:26.485 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:26.486 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:26.546 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053598,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T06:13:26.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:13:26.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:26.548 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:13:26.600 INFO:teuthology.orchestra.run.smithi084.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-11T06:13:26.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:13:26.601 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:13:26.601 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:26.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:13:26.662 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:13:26.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:26.663 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T06:13:26.716 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-11T06:13:26.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:13:26.718 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:26.718 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:26.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053598,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T06:13:26.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:13:26.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:26.779 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T06:13:26.832 INFO:teuthology.orchestra.run.smithi084.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-11T06:13:26.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:13:26.834 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:13:26.835 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:26.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:13:26.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:13:26.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:13:26.896 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:13:26.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:26.897 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T06:13:26.985 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-11T06:13:26.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:13:26.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:13:27.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.203+0000 7ff515a15640 1 Processor -- start 2023-10-11T06:13:27.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.204+0000 7ff515a15640 1 -- start start 2023-10-11T06:13:27.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.205+0000 7ff515a15640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5100d0b00 0x7ff5100b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.205+0000 7ff515a15640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51015c700 con 0x7ff51015c330 2023-10-11T06:13:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.205+0000 7ff515a15640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100b02a0 con 0x7ff5100afe10 2023-10-11T06:13:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.205+0000 7ff515a15640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100b0f20 con 0x7ff5100d0b00 2023-10-11T06:13:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.205+0000 7ff514a13640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff51015c330 0x7ff5100d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55660/0 (socket says 172.21.15.84:55660) 2023-10-11T06:13:27.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.205+0000 7ff514a13640 1 -- 172.21.15.84:0/3579251891 learned_addr learned my addr 172.21.15.84:0/3579251891 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:27.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834547929 0 0) 0x7ff504001290 con 0x7ff51015c330 2023-10-11T06:13:27.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff500003520 con 0x7ff51015c330 2023-10-11T06:13:27.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff515214640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5100d0b00 0x7ff5100b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707790507 0 0) 0x7ff500003520 con 0x7ff51015c330 2023-10-11T06:13:27.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5100d0b00 msgr2=0x7ff5100b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:27.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5100d0b00 0x7ff5100b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 >> v1:172.21.15.99:6789/0 conn(0x7ff5100afe10 legacy=0x7ff5100cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100b0580 con 0x7ff51015c330 2023-10-11T06:13:27.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7ff504001980 con 0x7ff51015c330 2023-10-11T06:13:27.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 >> v1:172.21.15.84:6789/0 conn(0x7ff51015c330 legacy=0x7ff5100d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff500003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5000047c0 0x7ff500004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5100d0b00 0x7ff50000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100b0f20 con 0x7ff5000039a0 2023-10-11T06:13:27.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff500005520 con 0x7ff5000047c0 2023-10-11T06:13:27.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.206+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50000bec0 con 0x7ff5100d0b00 2023-10-11T06:13:27.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.207+0000 7ff514a13640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff500003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.207+0000 7ff50ffff640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5000047c0 0x7ff500004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.207+0000 7ff514a13640 1 -- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5100d0b00 msgr2=0x7ff50000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:27.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.207+0000 7ff514a13640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5100d0b00 0x7ff50000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.207+0000 7ff514a13640 1 -- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5000047c0 msgr2=0x7ff500004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:27.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.207+0000 7ff514a13640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5000047c0 0x7ff500004c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.207+0000 7ff514a13640 1 -- 172.21.15.84:0/3579251891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff500003520 con 0x7ff5000039a0 2023-10-11T06:13:27.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.208+0000 7ff514a13640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff500003db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c210 tx=0x7ff5000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e76eb910906888 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:27.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.208+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504007270 con 0x7ff5000039a0 2023-10-11T06:13:27.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.208+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5040073f0 con 0x7ff5000039a0 2023-10-11T06:13:27.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.208+0000 7ff50f7fe640 1 -- 172.21.15.84:0/3579251891 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50402c890 con 0x7ff5000039a0 2023-10-11T06:13:27.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.209+0000 7ff515a15640 1 -- 172.21.15.84:0/3579251891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 msgr2=0x7ff500003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.209+0000 7ff515a15640 1 --2- 172.21.15.84:0/3579251891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff500003db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c210 tx=0x7ff5000057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.209+0000 7ff515214640 1 -- 172.21.15.84:0/3579251891 reap_dead start 2023-10-11T06:13:27.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.209+0000 7ff515a15640 1 -- 172.21.15.84:0/3579251891 shutdown_connections 2023-10-11T06:13:27.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.209+0000 7ff515a15640 1 -- 172.21.15.84:0/3579251891 >> 172.21.15.84:0/3579251891 conn(0x7ff51009bb10 msgr2=0x7ff5100d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:27.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.209+0000 7ff515a15640 1 -- 172.21.15.84:0/3579251891 shutdown_connections 2023-10-11T06:13:27.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.209+0000 7ff515a15640 1 -- 172.21.15.84:0/3579251891 wait complete. 2023-10-11T06:13:27.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.211+0000 7ff515a15640 1 Processor -- start 2023-10-11T06:13:27.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.211+0000 7ff515a15640 1 -- start start 2023-10-11T06:13:27.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff515a15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff5100c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff515a15640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff510148050 0x7ff5100c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff515a15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5100c85d0 0x7ff5100c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff515a15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c65c0 con 0x7ff5000039a0 2023-10-11T06:13:27.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff515a15640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c6820 con 0x7ff5100c85d0 2023-10-11T06:13:27.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff515a15640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c6b00 con 0x7ff510148050 2023-10-11T06:13:27.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff514a13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff5100c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff514a13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff5100c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46246/0 (socket says 172.21.15.84:46246) 2023-10-11T06:13:27.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.212+0000 7ff514a13640 1 -- 172.21.15.84:0/2157315703 learned_addr learned my addr 172.21.15.84:0/2157315703 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:27.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff514a13640 1 -- 172.21.15.84:0/2157315703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff510148050 msgr2=0x7ff5100c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff514a13640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff510148050 0x7ff5100c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff514a13640 1 -- 172.21.15.84:0/2157315703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5100c85d0 msgr2=0x7ff5100c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff514a13640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5100c85d0 0x7ff5100c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff514a13640 1 -- 172.21.15.84:0/2157315703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100c6de0 con 0x7ff5000039a0 2023-10-11T06:13:27.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff514a13640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff5100c6080 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff504000ef0 tx=0x7ff504001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:27.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff50d7fa640 1 -- 172.21.15.84:0/2157315703 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504039070 con 0x7ff5000039a0 2023-10-11T06:13:27.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff50d7fa640 1 -- 172.21.15.84:0/2157315703 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff504007270 con 0x7ff5000039a0 2023-10-11T06:13:27.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff50d7fa640 1 -- 172.21.15.84:0/2157315703 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50403c040 con 0x7ff5000039a0 2023-10-11T06:13:27.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.213+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100c70c0 con 0x7ff5000039a0 2023-10-11T06:13:27.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.214+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5100c78c0 con 0x7ff5000039a0 2023-10-11T06:13:27.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.215+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51004e890 con 0x7ff5000039a0 2023-10-11T06:13:27.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.216+0000 7ff50d7fa640 1 -- 172.21.15.84:0/2157315703 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff504032070 con 0x7ff5000039a0 2023-10-11T06:13:27.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.217+0000 7ff50d7fa640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff4f406a830 0x7ff4f406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.217+0000 7ff50d7fa640 1 -- 172.21.15.84:0/2157315703 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7ff5040aa410 con 0x7ff5000039a0 2023-10-11T06:13:27.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.217+0000 7ff50ffff640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff4f406a830 0x7ff4f406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.217+0000 7ff50ffff640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff4f406a830 0x7ff4f406cce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff510148970 tx=0x7ff4fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:27.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.221+0000 7ff50d7fa640 1 -- 172.21.15.84:0/2157315703 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff504075690 con 0x7ff5000039a0 2023-10-11T06:13:27.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.447+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff51015c700 con 0x7ff5000039a0 2023-10-11T06:13:27.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.448+0000 7ff50d7fa640 1 -- 172.21.15.84:0/2157315703 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4644 (secure 0 0 0) 0x7ff51015c700 con 0x7ff5000039a0 2023-10-11T06:13:27.449 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:27.450 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:12:40.002715+0000","modified":"2023-10-11T06:12:51.393935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5334":{"gid":5334,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/2172719017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2172719017},{"type":"v1","addr":"172.21.15.99:6835","nonce":2172719017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T06:13:27.450 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-10-11T06:13:27.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff4f406a830 msgr2=0x7ff4f406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff4f406a830 0x7ff4f406cce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff510148970 tx=0x7ff4fc009040 comp rx=0 tx=0).stop 2023-10-11T06:13:27.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 msgr2=0x7ff5100c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff5100c6080 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff504000ef0 tx=0x7ff504001d10 comp rx=0 tx=0).stop 2023-10-11T06:13:27.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 shutdown_connections 2023-10-11T06:13:27.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff4f406a830 0x7ff4f406cce0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5100c85d0 0x7ff5100c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff510148050 0x7ff5100c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 --2- 172.21.15.84:0/2157315703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5000039a0 0x7ff5100c6080 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 >> 172.21.15.84:0/2157315703 conn(0x7ff51009bb10 msgr2=0x7ff5100b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:27.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 shutdown_connections 2023-10-11T06:13:27.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.451+0000 7ff515a15640 1 -- 172.21.15.84:0/2157315703 wait complete. 2023-10-11T06:13:27.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:27.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.687+0000 7f9668958640 1 Processor -- start 2023-10-11T06:13:27.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.687+0000 7f9668958640 1 -- start start 2023-10-11T06:13:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.688+0000 7f9668958640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96640afe10 0x7f96640b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.688+0000 7f9668958640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96640d0b00 0x7f96640cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.688+0000 7f9668958640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966415c700 con 0x7f966415c330 2023-10-11T06:13:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.688+0000 7f9668958640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96640b0f20 con 0x7f96640afe10 2023-10-11T06:13:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.688+0000 7f9668958640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640d1100 con 0x7f96640d0b00 2023-10-11T06:13:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.689+0000 7f966259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96640afe10 0x7f96640b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.689+0000 7f966259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96640afe10 0x7f96640b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46262/0 (socket says 172.21.15.84:46262) 2023-10-11T06:13:27.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.689+0000 7f966259c640 1 -- 172.21.15.84:0/878777816 learned_addr learned my addr 172.21.15.84:0/878777816 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:27.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.689+0000 7f966259c640 1 -- 172.21.15.84:0/878777816 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96640d0b00 msgr2=0x7f96640cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:27.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.689+0000 7f966259c640 1 --2- 172.21.15.84:0/878777816 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96640d0b00 0x7f96640cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.689+0000 7f966259c640 1 -- 172.21.15.84:0/878777816 >> v1:172.21.15.99:6789/0 conn(0x7f966415c330 legacy=0x7f96640d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:27.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.689+0000 7f966259c640 1 -- 172.21.15.84:0/878777816 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640b02a0 con 0x7f96640afe10 2023-10-11T06:13:27.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.690+0000 7f966259c640 1 --2- 172.21.15.84:0/878777816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640afe10 0x7f96640b1340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9658009e50 tx=0x7f965802bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e811f3d4461c44 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:27.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.691+0000 7f9661d9b640 1 -- 172.21.15.84:0/878777816 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965802c980 con 0x7f96640afe10 2023-10-11T06:13:27.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.691+0000 7f9661d9b640 1 -- 172.21.15.84:0/878777816 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965802cae0 con 0x7f96640afe10 2023-10-11T06:13:27.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.691+0000 7f9661d9b640 1 -- 172.21.15.84:0/878777816 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965802ce70 con 0x7f96640afe10 2023-10-11T06:13:27.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.691+0000 7f9668958640 1 -- 172.21.15.84:0/878777816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640afe10 msgr2=0x7f96640b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.691+0000 7f9668958640 1 --2- 172.21.15.84:0/878777816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640afe10 0x7f96640b1340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9658009e50 tx=0x7f965802bde0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.692+0000 7f9668958640 1 -- 172.21.15.84:0/878777816 shutdown_connections 2023-10-11T06:13:27.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.692+0000 7f9668958640 1 --2- 172.21.15.84:0/878777816 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96640d0b00 0x7f96640cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.692+0000 7f9668958640 1 --2- 172.21.15.84:0/878777816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640afe10 0x7f96640b1340 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.692+0000 7f9668958640 1 -- 172.21.15.84:0/878777816 >> 172.21.15.84:0/878777816 conn(0x7f966409bb10 msgr2=0x7f96640d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:27.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.692+0000 7f9668958640 1 -- 172.21.15.84:0/878777816 shutdown_connections 2023-10-11T06:13:27.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.692+0000 7f9668958640 1 -- 172.21.15.84:0/878777816 wait complete. 2023-10-11T06:13:27.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.693+0000 7f9668958640 1 Processor -- start 2023-10-11T06:13:27.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.694+0000 7f9668958640 1 -- start start 2023-10-11T06:13:27.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9668958640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640afe10 0x7f96641f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9668958640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640d0b00 0x7f96641e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9668958640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f966415c330 0x7f96641f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9668958640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641e6880 con 0x7f96640d0b00 2023-10-11T06:13:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9668958640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640c70d0 con 0x7f96640afe10 2023-10-11T06:13:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9668958640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96641e6b20 con 0x7f966415c330 2023-10-11T06:13:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640d0b00 0x7f96641e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640d0b00 0x7f96641e6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46272/0 (socket says 172.21.15.84:46272) 2023-10-11T06:13:27.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 -- 172.21.15.84:0/2714172566 learned_addr learned my addr 172.21.15.84:0/2714172566 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:27.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9662d9d640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640afe10 0x7f96641f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966359e640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f966415c330 0x7f96641f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 -- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f966415c330 msgr2=0x7f96641f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f966415c330 0x7f96641f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 -- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640afe10 msgr2=0x7f96641f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640afe10 0x7f96641f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966259c640 1 -- 172.21.15.84:0/2714172566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9658009b30 con 0x7f96640d0b00 2023-10-11T06:13:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f9662d9d640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640afe10 0x7f96641f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:27.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.695+0000 7f966359e640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f966415c330 0x7f96641f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:27.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.696+0000 7f966259c640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640d0b00 0x7f96641e6340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f96580093e0 tx=0x7f9658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:27.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.696+0000 7f964b7fe640 1 -- 172.21.15.84:0/2714172566 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965802c940 con 0x7f96640d0b00 2023-10-11T06:13:27.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.696+0000 7f964b7fe640 1 -- 172.21.15.84:0/2714172566 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9658007d30 con 0x7f96640d0b00 2023-10-11T06:13:27.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.696+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96640c7390 con 0x7f96640d0b00 2023-10-11T06:13:27.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.696+0000 7f964b7fe640 1 -- 172.21.15.84:0/2714172566 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658007ea0 con 0x7f96640d0b00 2023-10-11T06:13:27.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.696+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96640c7950 con 0x7f96640d0b00 2023-10-11T06:13:27.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.697+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96640d1d60 con 0x7f96640d0b00 2023-10-11T06:13:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.698+0000 7f964b7fe640 1 -- 172.21.15.84:0/2714172566 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f965803e6b0 con 0x7f96640d0b00 2023-10-11T06:13:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.699+0000 7f964b7fe640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f965006a830 0x7f965006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.699+0000 7f964b7fe640 1 -- 172.21.15.84:0/2714172566 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f96580abc00 con 0x7f96640d0b00 2023-10-11T06:13:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.700+0000 7f9662d9d640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f965006a830 0x7f965006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:27.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.700+0000 7f9662d9d640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f965006a830 0x7f965006cce0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f96640a0750 tx=0x7f9654008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:27.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.704+0000 7f964b7fe640 1 -- 172.21.15.84:0/2714172566 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9658076cc0 con 0x7f96640d0b00 2023-10-11T06:13:27.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.893+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f966415c700 con 0x7f96640d0b00 2023-10-11T06:13:27.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.896+0000 7f964b7fe640 1 -- 172.21.15.84:0/2714172566 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7f966415c700 con 0x7f96640d0b00 2023-10-11T06:13:27.898 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:27.898 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f965006a830 msgr2=0x7f965006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f965006a830 0x7f965006cce0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f96640a0750 tx=0x7f9654008040 comp rx=0 tx=0).stop 2023-10-11T06:13:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640d0b00 msgr2=0x7f96641e6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640d0b00 0x7f96641e6340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f96580093e0 tx=0x7f9658004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:27.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 shutdown_connections 2023-10-11T06:13:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f965006a830 0x7f965006cce0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f966415c330 0x7f96641f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96640d0b00 0x7f96641e6340 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 --2- 172.21.15.84:0/2714172566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640afe10 0x7f96641f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:27.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.899+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 >> 172.21.15.84:0/2714172566 conn(0x7f966409bb10 msgr2=0x7f96640cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.900+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 shutdown_connections 2023-10-11T06:13:27.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:27.900+0000 7f9668958640 1 -- 172.21.15.84:0/2714172566 wait complete. 2023-10-11T06:13:27.918 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:12:38.260176+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:12:39.011036+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-10-11T06:12:50.853138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:13:27.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:13:28.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.137+0000 7fdd2cc5f640 1 Processor -- start 2023-10-11T06:13:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.137+0000 7fdd2cc5f640 1 -- start start 2023-10-11T06:13:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2cc5f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd2815a0b0 0x7fdd280cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2cc5f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd280c8d00 0x7fdd280ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2cc5f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd280cefc0 con 0x7fdd280ceb60 2023-10-11T06:13:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2cc5f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2806e0f0 con 0x7fdd2815a0b0 2023-10-11T06:13:28.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2cc5f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd280cbae0 con 0x7fdd280c8d00 2023-10-11T06:13:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd2815a0b0 0x7fdd280cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd2815a0b0 0x7fdd280cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46286/0 (socket says 172.21.15.84:46286) 2023-10-11T06:13:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.139+0000 7fdd2659c640 1 -- 172.21.15.84:0/408051855 learned_addr learned my addr 172.21.15.84:0/408051855 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:28.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.140+0000 7fdd1ffff640 1 --2- 172.21.15.84:0/408051855 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd280c8d00 0x7fdd280ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.140+0000 7fdd2659c640 1 -- 172.21.15.84:0/408051855 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd280c8d00 msgr2=0x7fdd280ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.140+0000 7fdd2659c640 1 --2- 172.21.15.84:0/408051855 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd280c8d00 0x7fdd280ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.140+0000 7fdd2659c640 1 -- 172.21.15.84:0/408051855 >> v1:172.21.15.99:6789/0 conn(0x7fdd280ceb60 legacy=0x7fdd2806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.140+0000 7fdd2659c640 1 -- 172.21.15.84:0/408051855 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd28071380 con 0x7fdd2815a0b0 2023-10-11T06:13:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.141+0000 7fdd2659c640 1 --2- 172.21.15.84:0/408051855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2815a0b0 0x7fdd280cb5a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd180098a0 tx=0x7fdd1802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd616d28394156b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.142+0000 7fdd25d9b640 1 -- 172.21.15.84:0/408051855 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd180317c0 con 0x7fdd2815a0b0 2023-10-11T06:13:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.142+0000 7fdd25d9b640 1 -- 172.21.15.84:0/408051855 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18031920 con 0x7fdd2815a0b0 2023-10-11T06:13:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.142+0000 7fdd25d9b640 1 -- 172.21.15.84:0/408051855 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18031cb0 con 0x7fdd2815a0b0 2023-10-11T06:13:28.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.142+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/408051855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2815a0b0 msgr2=0x7fdd280cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.142+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/408051855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2815a0b0 0x7fdd280cb5a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd180098a0 tx=0x7fdd1802b860 comp rx=0 tx=0).stop 2023-10-11T06:13:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.143+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/408051855 shutdown_connections 2023-10-11T06:13:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.143+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/408051855 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd280c8d00 0x7fdd280ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.143+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/408051855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd2815a0b0 0x7fdd280cb5a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.143+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/408051855 >> 172.21.15.84:0/408051855 conn(0x7fdd28069c70 msgr2=0x7fdd2806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:28.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.143+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/408051855 shutdown_connections 2023-10-11T06:13:28.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.143+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/408051855 wait complete. 2023-10-11T06:13:28.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.144+0000 7fdd2cc5f640 1 Processor -- start 2023-10-11T06:13:28.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.145+0000 7fdd2cc5f640 1 -- start start 2023-10-11T06:13:28.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.145+0000 7fdd2cc5f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd280c8d00 0x7fdd2813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.145+0000 7fdd2cc5f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd280ceb60 0x7fdd280c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd2cc5f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2815a0b0 0x7fdd28137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd2cc5f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd280c8550 con 0x7fdd280ceb60 2023-10-11T06:13:28.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd2cc5f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2813b6e0 con 0x7fdd280c8d00 2023-10-11T06:13:28.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd2cc5f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd280c5a40 con 0x7fdd2815a0b0 2023-10-11T06:13:28.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd280ceb60 0x7fdd280c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd280ceb60 0x7fdd280c37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46302/0 (socket says 172.21.15.84:46302) 2023-10-11T06:13:28.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 -- 172.21.15.84:0/1698834532 learned_addr learned my addr 172.21.15.84:0/1698834532 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:28.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd2659c640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd280c8d00 0x7fdd2813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd26d9d640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2815a0b0 0x7fdd28137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 -- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2815a0b0 msgr2=0x7fdd28137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2815a0b0 0x7fdd28137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 -- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd280c8d00 msgr2=0x7fdd2813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd280c8d00 0x7fdd2813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.146+0000 7fdd1ffff640 1 -- 172.21.15.84:0/1698834532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180094d0 con 0x7fdd280ceb60 2023-10-11T06:13:28.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.147+0000 7fdd1ffff640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd280ceb60 0x7fdd280c37a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000d800 tx=0x7fdd1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:28.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.147+0000 7fdd1effd640 1 -- 172.21.15.84:0/1698834532 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000b6c0 con 0x7fdd280ceb60 2023-10-11T06:13:28.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.147+0000 7fdd1effd640 1 -- 172.21.15.84:0/1698834532 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1000b820 con 0x7fdd280ceb60 2023-10-11T06:13:28.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.147+0000 7fdd1effd640 1 -- 172.21.15.84:0/1698834532 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000bbb0 con 0x7fdd280ceb60 2023-10-11T06:13:28.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.147+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd280c78d0 con 0x7fdd280ceb60 2023-10-11T06:13:28.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.147+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd280c8170 con 0x7fdd280ceb60 2023-10-11T06:13:28.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.148+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd28073ed0 con 0x7fdd280ceb60 2023-10-11T06:13:28.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.149+0000 7fdd1effd640 1 -- 172.21.15.84:0/1698834532 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdd1000bd10 con 0x7fdd280ceb60 2023-10-11T06:13:28.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.150+0000 7fdd1effd640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd0806a830 0x7fdd0806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.150+0000 7fdd1effd640 1 -- 172.21.15.84:0/1698834532 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fdd100896a0 con 0x7fdd280ceb60 2023-10-11T06:13:28.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.151+0000 7fdd2659c640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd0806a830 0x7fdd0806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.151+0000 7fdd2659c640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd0806a830 0x7fdd0806cce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd18036000 tx=0x7fdd1803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:28.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.155+0000 7fdd1effd640 1 -- 172.21.15.84:0/1698834532 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdd10054760 con 0x7fdd280ceb60 2023-10-11T06:13:28.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.344+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd28070760 con 0x7fdd280ceb60 2023-10-11T06:13:28.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.344+0000 7fdd1effd640 1 -- 172.21.15.84:0/1698834532 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd10057d80 con 0x7fdd280ceb60 2023-10-11T06:13:28.346 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:13:28.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd0806a830 msgr2=0x7fdd0806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd0806a830 0x7fdd0806cce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd18036000 tx=0x7fdd1803d040 comp rx=0 tx=0).stop 2023-10-11T06:13:28.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd280ceb60 msgr2=0x7fdd280c37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd280ceb60 0x7fdd280c37a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000d800 tx=0x7fdd1000dcc0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 shutdown_connections 2023-10-11T06:13:28.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdd0806a830 0x7fdd0806cce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd2815a0b0 0x7fdd28137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd280ceb60 0x7fdd280c37a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 --2- 172.21.15.84:0/1698834532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd280c8d00 0x7fdd2813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 >> 172.21.15.84:0/1698834532 conn(0x7fdd28069c70 msgr2=0x7fdd280c4220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:28.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 shutdown_connections 2023-10-11T06:13:28.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.347+0000 7fdd2cc5f640 1 -- 172.21.15.84:0/1698834532 wait complete. 2023-10-11T06:13:28.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:13:28.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.585+0000 7ff3da30a640 1 Processor -- start 2023-10-11T06:13:28.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.585+0000 7ff3da30a640 1 -- start start 2023-10-11T06:13:28.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3da30a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff3d414ff30 0x7ff3d40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3da30a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d40c9b90 con 0x7ff3d4150300 2023-10-11T06:13:28.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3da30a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d415da10 con 0x7ff3d40ce510 2023-10-11T06:13:28.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3da30a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d40cc990 con 0x7ff3d414ff30 2023-10-11T06:13:28.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3d37fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff3d4150300 0x7ff3d40cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55670/0 (socket says 172.21.15.84:55670) 2023-10-11T06:13:28.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3d37fe640 1 -- 172.21.15.84:0/2715100205 learned_addr learned my addr 172.21.15.84:0/2715100205 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:28.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3d3fff640 1 --2- 172.21.15.84:0/2715100205 >> v2:172.21.15.99:3300/0 conn(0x7ff3d414ff30 0x7ff3d40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431082489 0 0) 0x7ff3d40c9b90 con 0x7ff3d4150300 2023-10-11T06:13:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.587+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3bc003520 con 0x7ff3d4150300 2023-10-11T06:13:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650857725 0 0) 0x7ff3bc003520 con 0x7ff3d4150300 2023-10-11T06:13:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 >> v2:172.21.15.99:3300/0 conn(0x7ff3d414ff30 msgr2=0x7ff3d40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:28.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 --2- 172.21.15.84:0/2715100205 >> v2:172.21.15.99:3300/0 conn(0x7ff3d414ff30 0x7ff3d40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 >> v1:172.21.15.99:6789/0 conn(0x7ff3d40ce510 legacy=0x7ff3d40ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d415ce00 con 0x7ff3d4150300 2023-10-11T06:13:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7ff3c8001670 con 0x7ff3d4150300 2023-10-11T06:13:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 >> v1:172.21.15.84:6789/0 conn(0x7ff3d4150300 legacy=0x7ff3d40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3bc0039c0 0x7ff3bc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3bc004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3bc0051c0 0x7ff3bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40cc990 con 0x7ff3bc0047c0 2023-10-11T06:13:28.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc00c0d0 con 0x7ff3bc0051c0 2023-10-11T06:13:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc00c290 con 0x7ff3bc0039c0 2023-10-11T06:13:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.588+0000 7ff3d37fe640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3bc004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d3fff640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3bc0039c0 0x7ff3bc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d37fe640 1 -- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3bc0039c0 msgr2=0x7ff3bc003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:28.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d37fe640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3bc0039c0 0x7ff3bc003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d88a7640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3bc0051c0 0x7ff3bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d37fe640 1 -- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3bc0051c0 msgr2=0x7ff3bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:28.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d37fe640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3bc0051c0 0x7ff3bc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d37fe640 1 -- 172.21.15.84:0/2715100205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc003520 con 0x7ff3bc0047c0 2023-10-11T06:13:28.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.589+0000 7ff3d88a7640 1 -- 172.21.15.84:0/2715100205 reap_dead start 2023-10-11T06:13:28.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.590+0000 7ff3d37fe640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3bc004c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8007fd0 tx=0x7ff3bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6839e4b1c0eeb15 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:28.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.590+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8005ad0 con 0x7ff3bc0047c0 2023-10-11T06:13:28.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.590+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c8005c30 con 0x7ff3bc0047c0 2023-10-11T06:13:28.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.590+0000 7ff3d2ffd640 1 -- 172.21.15.84:0/2715100205 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c802c7c0 con 0x7ff3bc0047c0 2023-10-11T06:13:28.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.591+0000 7ff3da30a640 1 -- 172.21.15.84:0/2715100205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 msgr2=0x7ff3bc004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.591+0000 7ff3da30a640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3bc004c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8007fd0 tx=0x7ff3bc0057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.591+0000 7ff3da30a640 1 -- 172.21.15.84:0/2715100205 shutdown_connections 2023-10-11T06:13:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.591+0000 7ff3da30a640 1 --2- 172.21.15.84:0/2715100205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3bc004c30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.591+0000 7ff3da30a640 1 -- 172.21.15.84:0/2715100205 >> 172.21.15.84:0/2715100205 conn(0x7ff3d4069c70 msgr2=0x7ff3d406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:28.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.591+0000 7ff3da30a640 1 -- 172.21.15.84:0/2715100205 shutdown_connections 2023-10-11T06:13:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.591+0000 7ff3da30a640 1 -- 172.21.15.84:0/2715100205 wait complete. 2023-10-11T06:13:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.593+0000 7ff3da30a640 1 Processor -- start 2023-10-11T06:13:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.594+0000 7ff3da30a640 1 -- start start 2023-10-11T06:13:28.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.594+0000 7ff3da30a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3d41478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.594+0000 7ff3da30a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d415abe0 0x7ff3d40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.594+0000 7ff3da30a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3d4143d90 0x7ff3d4144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.594+0000 7ff3da30a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4144770 con 0x7ff3bc0047c0 2023-10-11T06:13:28.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.594+0000 7ff3da30a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4154b10 con 0x7ff3d415abe0 2023-10-11T06:13:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.594+0000 7ff3da30a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d415b380 con 0x7ff3d4143d90 2023-10-11T06:13:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3d41478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3d41478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46322/0 (socket says 172.21.15.84:46322) 2023-10-11T06:13:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 -- 172.21.15.84:0/1315014306 learned_addr learned my addr 172.21.15.84:0/1315014306 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:28.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 -- 172.21.15.84:0/1315014306 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3d4143d90 msgr2=0x7ff3d4144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3d4143d90 0x7ff3d4144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 -- 172.21.15.84:0/1315014306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d415abe0 msgr2=0x7ff3d40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d415abe0 0x7ff3d40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 -- 172.21.15.84:0/1315014306 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d415b660 con 0x7ff3bc0047c0 2023-10-11T06:13:28.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.595+0000 7ff3d3fff640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3d41478d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40108e0 tx=0x7ff3c4010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.596+0000 7ff3d0ff9640 1 -- 172.21.15.84:0/1315014306 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400eb60 con 0x7ff3bc0047c0 2023-10-11T06:13:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.596+0000 7ff3d0ff9640 1 -- 172.21.15.84:0/1315014306 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c400ecc0 con 0x7ff3bc0047c0 2023-10-11T06:13:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.596+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d415b940 con 0x7ff3bc0047c0 2023-10-11T06:13:28.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.596+0000 7ff3d0ff9640 1 -- 172.21.15.84:0/1315014306 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c401c400 con 0x7ff3bc0047c0 2023-10-11T06:13:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.596+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d415b050 con 0x7ff3bc0047c0 2023-10-11T06:13:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.597+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d406ab70 con 0x7ff3bc0047c0 2023-10-11T06:13:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.598+0000 7ff3d0ff9640 1 -- 172.21.15.84:0/1315014306 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff3c401c560 con 0x7ff3bc0047c0 2023-10-11T06:13:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.599+0000 7ff3d0ff9640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3c006a860 0x7ff3c006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.599+0000 7ff3d0ff9640 1 -- 172.21.15.84:0/1315014306 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7ff3c4089850 con 0x7ff3bc0047c0 2023-10-11T06:13:28.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.599+0000 7ff3d37fe640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3c006a860 0x7ff3c006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:28.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.600+0000 7ff3d37fe640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3c006a860 0x7ff3c006cd10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8034000 tx=0x7ff3c8039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:28.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.604+0000 7ff3d0ff9640 1 -- 172.21.15.84:0/1315014306 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff3c4054910 con 0x7ff3bc0047c0 2023-10-11T06:13:28.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.792+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff3d4073ed0 con 0x7ff3bc0047c0 2023-10-11T06:13:28.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.793+0000 7ff3d0ff9640 1 -- 172.21.15.84:0/1315014306 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff3c4057f30 con 0x7ff3bc0047c0 2023-10-11T06:13:28.795 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:13:28.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.795+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3c006a860 msgr2=0x7ff3c006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3c006a860 0x7ff3c006cd10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8034000 tx=0x7ff3c8039040 comp rx=0 tx=0).stop 2023-10-11T06:13:28.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 msgr2=0x7ff3d41478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:28.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3d41478d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40108e0 tx=0x7ff3c4010da0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 shutdown_connections 2023-10-11T06:13:28.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff3c006a860 0x7ff3c006cd10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3d4143d90 0x7ff3d4144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d415abe0 0x7ff3d40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 --2- 172.21.15.84:0/1315014306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc0047c0 0x7ff3d41478d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:28.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 >> 172.21.15.84:0/1315014306 conn(0x7ff3d4069c70 msgr2=0x7ff3d406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:28.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 shutdown_connections 2023-10-11T06:13:28.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:28.796+0000 7ff3da30a640 1 -- 172.21.15.84:0/1315014306 wait complete. 2023-10-11T06:13:28.817 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:13:29.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.033+0000 7f3a30cb6640 1 Processor -- start 2023-10-11T06:13:29.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.033+0000 7f3a30cb6640 1 -- start start 2023-10-11T06:13:29.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a30cb6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a2c1591f0 0x7f3a2c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a30cb6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c15cd40 con 0x7f3a2c158e20 2023-10-11T06:13:29.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a30cb6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c15cfe0 con 0x7f3a2c0cc890 2023-10-11T06:13:29.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a30cb6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c15d2c0 con 0x7f3a2c1591f0 2023-10-11T06:13:29.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a2b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3a2c158e20 0x7f3a2c0cae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54338/0 (socket says 172.21.15.84:54338) 2023-10-11T06:13:29.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a2b7fe640 1 -- 172.21.15.84:0/3071639402 learned_addr learned my addr 172.21.15.84:0/3071639402 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:29.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373256656 0 0) 0x7f3a2c15cd40 con 0x7f3a2c158e20 2023-10-11T06:13:29.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.035+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a14005730 con 0x7f3a2c158e20 2023-10-11T06:13:29.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3071639402 >> v2:172.21.15.99:3300/0 conn(0x7f3a2c1591f0 0x7f3a2c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2402723127 0 0) 0x7f3a14005730 con 0x7f3a2c158e20 2023-10-11T06:13:29.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 >> v2:172.21.15.99:3300/0 conn(0x7f3a2c1591f0 msgr2=0x7f3a2c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 --2- 172.21.15.84:0/3071639402 >> v2:172.21.15.99:3300/0 conn(0x7f3a2c1591f0 0x7f3a2c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 >> v1:172.21.15.99:6789/0 conn(0x7f3a2c0cc890 legacy=0x7f3a2c0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c15d5a0 con 0x7f3a2c158e20 2023-10-11T06:13:29.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f3a1c001670 con 0x7f3a2c158e20 2023-10-11T06:13:29.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 >> v1:172.21.15.84:6789/0 conn(0x7f3a2c158e20 legacy=0x7f3a2c0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a14005bd0 0x7f3a14005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140069d0 0x7f3a14006e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a140074a0 0x7f3a1400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15d2c0 con 0x7f3a140074a0 2023-10-11T06:13:29.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1400e380 con 0x7f3a14005bd0 2023-10-11T06:13:29.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1400e540 con 0x7f3a140069d0 2023-10-11T06:13:29.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a140074a0 0x7f3a1400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.036+0000 7f3a2b7fe640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a14005bd0 0x7f3a14005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.037+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140069d0 msgr2=0x7f3a14006e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.037+0000 7f3a2affd640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140069d0 0x7f3a14006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.037+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140069d0 0x7f3a14006e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.037+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a14005bd0 msgr2=0x7f3a14005fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.037+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a14005bd0 0x7f3a14005fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.037+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3071639402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14005730 con 0x7f3a140074a0 2023-10-11T06:13:29.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.038+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3071639402 reap_dead start 2023-10-11T06:13:29.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.038+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a140074a0 0x7f3a1400db90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3a24007360 tx=0x7f3a14007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6a3bf83f674525 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.038+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2402ccf0 con 0x7f3a140074a0 2023-10-11T06:13:29.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.038+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2402ce50 con 0x7f3a140074a0 2023-10-11T06:13:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.038+0000 7f3a2a7fc640 1 -- 172.21.15.84:0/3071639402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24003bd0 con 0x7f3a140074a0 2023-10-11T06:13:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.039+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3071639402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a140074a0 msgr2=0x7f3a1400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.039+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a140074a0 0x7f3a1400db90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3a24007360 tx=0x7f3a14007a00 comp rx=0 tx=0).stop 2023-10-11T06:13:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.039+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3071639402 shutdown_connections 2023-10-11T06:13:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.039+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3071639402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a140074a0 0x7f3a1400db90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.039+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3071639402 >> 172.21.15.84:0/3071639402 conn(0x7f3a2c06bfb0 msgr2=0x7f3a2c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.039+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3071639402 shutdown_connections 2023-10-11T06:13:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.039+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3071639402 wait complete. 2023-10-11T06:13:29.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.041+0000 7f3a30cb6640 1 Processor -- start 2023-10-11T06:13:29.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.041+0000 7f3a30cb6640 1 -- start start 2023-10-11T06:13:29.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a30cb6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140074a0 0x7f3a2c0c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a30cb6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2c0c4470 0x7f3a2c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a30cb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a2c157620 0x7f3a2c157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a30cb6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c157fd0 con 0x7f3a2c157620 2023-10-11T06:13:29.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a30cb6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0ced40 con 0x7f3a2c0c4470 2023-10-11T06:13:29.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a30cb6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0c3460 con 0x7f3a140074a0 2023-10-11T06:13:29.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a2c157620 0x7f3a2c157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a2c157620 0x7f3a2c157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33858/0 (socket says 172.21.15.84:33858) 2023-10-11T06:13:29.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3658261260 learned_addr learned my addr 172.21.15.84:0/3658261260 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:29.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2affd640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2c0c4470 0x7f3a2c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2b7fe640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140074a0 0x7f3a2c0c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140074a0 msgr2=0x7f3a2c0c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140074a0 0x7f3a2c0c2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2c0c4470 msgr2=0x7f3a2c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2c0c4470 0x7f3a2c0ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.042+0000 7f3a2bfff640 1 -- 172.21.15.84:0/3658261260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0b2d70 con 0x7f3a2c157620 2023-10-11T06:13:29.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.043+0000 7f3a2bfff640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a2c157620 0x7f3a2c157a90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402dfd0 tx=0x7f3a24004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.043+0000 7f3a13fff640 1 -- 172.21.15.84:0/3658261260 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24039070 con 0x7f3a2c157620 2023-10-11T06:13:29.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.043+0000 7f3a13fff640 1 -- 172.21.15.84:0/3658261260 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a24003c80 con 0x7f3a2c157620 2023-10-11T06:13:29.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.043+0000 7f3a13fff640 1 -- 172.21.15.84:0/3658261260 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24004040 con 0x7f3a2c157620 2023-10-11T06:13:29.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.043+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c0c36b0 con 0x7f3a2c157620 2023-10-11T06:13:29.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.043+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c0c3c70 con 0x7f3a2c157620 2023-10-11T06:13:29.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.044+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c072310 con 0x7f3a2c157620 2023-10-11T06:13:29.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.045+0000 7f3a13fff640 1 -- 172.21.15.84:0/3658261260 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3a2403d070 con 0x7f3a2c157620 2023-10-11T06:13:29.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.046+0000 7f3a13fff640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a1806a860 0x7f3a1806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.046+0000 7f3a13fff640 1 -- 172.21.15.84:0/3658261260 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f3a240aa130 con 0x7f3a2c157620 2023-10-11T06:13:29.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.046+0000 7f3a2b7fe640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a1806a860 0x7f3a1806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.047+0000 7f3a2b7fe640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a1806a860 0x7f3a1806cd10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c001050 tx=0x7f3a1c000f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.050+0000 7f3a13fff640 1 -- 172.21.15.84:0/3658261260 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3a240751f0 con 0x7f3a2c157620 2023-10-11T06:13:29.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.239+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a2c06e240 con 0x7f3a2c157620 2023-10-11T06:13:29.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.243+0000 7f3a13fff640 1 -- 172.21.15.84:0/3658261260 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a24044020 con 0x7f3a2c157620 2023-10-11T06:13:29.244 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:29.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a1806a860 msgr2=0x7f3a1806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a1806a860 0x7f3a1806cd10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c001050 tx=0x7f3a1c000f40 comp rx=0 tx=0).stop 2023-10-11T06:13:29.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a2c157620 msgr2=0x7f3a2c157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a2c157620 0x7f3a2c157a90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402dfd0 tx=0x7f3a24004ab0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 shutdown_connections 2023-10-11T06:13:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a1806a860 0x7f3a1806cd10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a2c157620 0x7f3a2c157a90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a2c0c4470 0x7f3a2c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 --2- 172.21.15.84:0/3658261260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a140074a0 0x7f3a2c0c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 >> 172.21.15.84:0/3658261260 conn(0x7f3a2c06bfb0 msgr2=0x7f3a2c0ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 shutdown_connections 2023-10-11T06:13:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.246+0000 7f3a30cb6640 1 -- 172.21.15.84:0/3658261260 wait complete. 2023-10-11T06:13:29.266 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:13:29.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.483+0000 7f001ec10640 1 Processor -- start 2023-10-11T06:13:29.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.483+0000 7f001ec10640 1 -- start start 2023-10-11T06:13:29.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001ec10640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f001806c880 0x7f001806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001ec10640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0018156be0 0x7f001815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001ec10640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00180cde80 con 0x7f0018156810 2023-10-11T06:13:29.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001ec10640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00180ce140 con 0x7f0018156be0 2023-10-11T06:13:29.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001ec10640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001815da30 con 0x7f001806c880 2023-10-11T06:13:29.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001d1ad640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0018156be0 0x7f001815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001c9ac640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0018156810 0x7f001806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54340/0 (socket says 172.21.15.84:54340) 2023-10-11T06:13:29.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.485+0000 7f001c9ac640 1 -- 172.21.15.84:0/1575277947 learned_addr learned my addr 172.21.15.84:0/1575277947 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:29.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.486+0000 7f0017fff640 1 --2- 172.21.15.84:0/1575277947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f001806c880 0x7f001806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.486+0000 7f00177fe640 1 -- 172.21.15.84:0/1575277947 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680854869 0 0) 0x7f00180cde80 con 0x7f0018156810 2023-10-11T06:13:29.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.486+0000 7f001d1ad640 1 -- 172.21.15.84:0/1575277947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f001806c880 msgr2=0x7f001806d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.486+0000 7f001d1ad640 1 --2- 172.21.15.84:0/1575277947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f001806c880 0x7f001806d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.486+0000 7f001d1ad640 1 -- 172.21.15.84:0/1575277947 >> v1:172.21.15.84:6789/0 conn(0x7f0018156810 legacy=0x7f001806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.486+0000 7f001d1ad640 1 -- 172.21.15.84:0/1575277947 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001806df80 con 0x7f0018156be0 2023-10-11T06:13:29.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.487+0000 7f001d1ad640 1 --2- 172.21.15.84:0/1575277947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156be0 0x7f001815d4f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0010009d90 tx=0x7f001002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56f99598dbde7ab server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f00177fe640 1 -- 172.21.15.84:0/1575277947 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001002cbc0 con 0x7f0018156be0 2023-10-11T06:13:29.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f00177fe640 1 -- 172.21.15.84:0/1575277947 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001002cd20 con 0x7f0018156be0 2023-10-11T06:13:29.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f00177fe640 1 -- 172.21.15.84:0/1575277947 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010034660 con 0x7f0018156be0 2023-10-11T06:13:29.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 -- 172.21.15.84:0/1575277947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156be0 msgr2=0x7f001815d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 --2- 172.21.15.84:0/1575277947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156be0 0x7f001815d4f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0010009d90 tx=0x7f001002be10 comp rx=0 tx=0).stop 2023-10-11T06:13:29.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 -- 172.21.15.84:0/1575277947 shutdown_connections 2023-10-11T06:13:29.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 --2- 172.21.15.84:0/1575277947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156be0 0x7f001815d4f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 --2- 172.21.15.84:0/1575277947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f001806c880 0x7f001806d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 -- 172.21.15.84:0/1575277947 >> 172.21.15.84:0/1575277947 conn(0x7f0018069c70 msgr2=0x7f001806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:29.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 -- 172.21.15.84:0/1575277947 shutdown_connections 2023-10-11T06:13:29.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.488+0000 7f001ec10640 1 -- 172.21.15.84:0/1575277947 wait complete. 2023-10-11T06:13:29.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.490+0000 7f001ec10640 1 Processor -- start 2023-10-11T06:13:29.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.490+0000 7f001ec10640 1 -- start start 2023-10-11T06:13:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.491+0000 7f001ec10640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f001806c880 0x7f00180ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.491+0000 7f001ec10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156810 0x7f00180ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.491+0000 7f001ec10640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0018156be0 0x7f001814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.491+0000 7f001ec10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b1120 con 0x7f0018156810 2023-10-11T06:13:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.491+0000 7f001ec10640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180abe10 con 0x7f001806c880 2023-10-11T06:13:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.491+0000 7f001ec10640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180af6d0 con 0x7f0018156be0 2023-10-11T06:13:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156810 0x7f00180ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156810 0x7f00180ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33888/0 (socket says 172.21.15.84:33888) 2023-10-11T06:13:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 -- 172.21.15.84:0/1861430862 learned_addr learned my addr 172.21.15.84:0/1861430862 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f001c9ac640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f001806c880 0x7f00180ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f001d1ad640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0018156be0 0x7f001814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 -- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0018156be0 msgr2=0x7f001814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0018156be0 0x7f001814fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 -- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f001806c880 msgr2=0x7f00180ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f001806c880 0x7f00180ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f0017fff640 1 -- 172.21.15.84:0/1861430862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010009a70 con 0x7f0018156810 2023-10-11T06:13:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.492+0000 7f001d1ad640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0018156be0 0x7f001814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.493+0000 7f0017fff640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156810 0x7f00180ac3c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f000400d7d0 tx=0x7f000400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.493+0000 7f00157fa640 1 -- 172.21.15.84:0/1861430862 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400b660 con 0x7f0018156810 2023-10-11T06:13:29.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.493+0000 7f00157fa640 1 -- 172.21.15.84:0/1861430862 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000400b7c0 con 0x7f0018156810 2023-10-11T06:13:29.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.493+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00180afc90 con 0x7f0018156810 2023-10-11T06:13:29.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.493+0000 7f00157fa640 1 -- 172.21.15.84:0/1861430862 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400bb50 con 0x7f0018156810 2023-10-11T06:13:29.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.493+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00180b0250 con 0x7f0018156810 2023-10-11T06:13:29.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.494+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00180b03e0 con 0x7f0018156810 2023-10-11T06:13:29.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.495+0000 7f00157fa640 1 -- 172.21.15.84:0/1861430862 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f000400bcb0 con 0x7f0018156810 2023-10-11T06:13:29.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.496+0000 7f00157fa640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efffc06a830 0x7efffc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.496+0000 7f00157fa640 1 -- 172.21.15.84:0/1861430862 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f00040896f0 con 0x7f0018156810 2023-10-11T06:13:29.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.496+0000 7f001c9ac640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efffc06a830 0x7efffc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.497+0000 7f001c9ac640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efffc06a830 0x7efffc06cce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f000c000f60 tx=0x7f000c007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.500+0000 7f00157fa640 1 -- 172.21.15.84:0/1861430862 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0004054720 con 0x7f0018156810 2023-10-11T06:13:29.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.689+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0018073ed0 con 0x7f0018156810 2023-10-11T06:13:29.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.690+0000 7f00157fa640 1 -- 172.21.15.84:0/1861430862 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0004057d40 con 0x7f0018156810 2023-10-11T06:13:29.691 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:13:29.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efffc06a830 msgr2=0x7efffc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efffc06a830 0x7efffc06cce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f000c000f60 tx=0x7f000c007b50 comp rx=0 tx=0).stop 2023-10-11T06:13:29.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156810 msgr2=0x7f00180ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156810 0x7f00180ac3c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f000400d7d0 tx=0x7f000400dc90 comp rx=0 tx=0).stop 2023-10-11T06:13:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 shutdown_connections 2023-10-11T06:13:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efffc06a830 0x7efffc06cce0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0018156be0 0x7f001814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0018156810 0x7f00180ac3c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 --2- 172.21.15.84:0/1861430862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f001806c880 0x7f00180ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 >> 172.21.15.84:0/1861430862 conn(0x7f0018069c70 msgr2=0x7f00180c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:29.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 shutdown_connections 2023-10-11T06:13:29.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.693+0000 7f001ec10640 1 -- 172.21.15.84:0/1861430862 wait complete. 2023-10-11T06:13:29.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-10-11T06:13:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.931+0000 7f2ea441a640 1 Processor -- start 2023-10-11T06:13:29.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.931+0000 7f2ea441a640 1 -- start start 2023-10-11T06:13:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.933+0000 7f2ea441a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e9c0ced30 0x7f2e9c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.933+0000 7f2ea441a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c071380 con 0x7f2e9c0c8d00 2023-10-11T06:13:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.933+0000 7f2ea441a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c06de10 con 0x7f2e9c15a0b0 2023-10-11T06:13:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.933+0000 7f2ea441a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c06e0f0 con 0x7f2e9c0ced30 2023-10-11T06:13:29.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.933+0000 7f2ea29b7640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2e9c0c8d00 0x7f2e9c06c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54354/0 (socket says 172.21.15.84:54354) 2023-10-11T06:13:29.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.933+0000 7f2ea29b7640 1 -- 172.21.15.84:0/4060184810 learned_addr learned my addr 172.21.15.84:0/4060184810 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.933+0000 7f2ea19b5640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e9c0ced30 0x7f2e9c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587084570 0 0) 0x7f2e9c071380 con 0x7f2e9c0c8d00 2023-10-11T06:13:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e88003520 con 0x7f2e9c0c8d00 2023-10-11T06:13:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116388888 0 0) 0x7f2e9c06de10 con 0x7f2e9c15a0b0 2023-10-11T06:13:29.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9c071380 con 0x7f2e9c15a0b0 2023-10-11T06:13:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105396182 0 0) 0x7f2e88003520 con 0x7f2e9c0c8d00 2023-10-11T06:13:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e9c0ced30 msgr2=0x7f2e9c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e9c0ced30 0x7f2e9c0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 >> v1:172.21.15.99:6789/0 conn(0x7f2e9c15a0b0 legacy=0x7f2e9c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c070dc0 con 0x7f2e9c0c8d00 2023-10-11T06:13:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f2e940019f0 con 0x7f2e9c0c8d00 2023-10-11T06:13:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 >> v1:172.21.15.84:6789/0 conn(0x7f2e9c0c8d00 legacy=0x7f2e9c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e88003b30 0x7f2e88003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e880047c0 0x7f2e88004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e88005170 0x7f2e8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c06e0f0 con 0x7f2e88005170 2023-10-11T06:13:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8800c0d0 con 0x7f2e88003b30 2023-10-11T06:13:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8800c290 con 0x7f2e880047c0 2023-10-11T06:13:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.934+0000 7f2ea19b5640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e9c0ced30 0x7f2e9c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:13:29.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea29b7640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e88005170 0x7f2e8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea21b6640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e88003b30 0x7f2e88003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea29b7640 1 -- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e880047c0 msgr2=0x7f2e88004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea29b7640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e880047c0 0x7f2e88004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea29b7640 1 -- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e88003b30 msgr2=0x7f2e88003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:29.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea29b7640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e88003b30 0x7f2e88003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea29b7640 1 -- 172.21.15.84:0/4060184810 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88003520 con 0x7f2e88005170 2023-10-11T06:13:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.935+0000 7f2ea21b6640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e88003b30 0x7f2e88003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.936+0000 7f2ea29b7640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e88005170 0x7f2e8800b980 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400a7c0 tx=0x7f2e880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a00f51341b8841 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.936+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94034630 con 0x7f2e88005170 2023-10-11T06:13:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.936+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9402f070 con 0x7f2e88005170 2023-10-11T06:13:29.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.936+0000 7f2ea11b4640 1 -- 172.21.15.84:0/4060184810 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e940349f0 con 0x7f2e88005170 2023-10-11T06:13:29.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.937+0000 7f2ea441a640 1 -- 172.21.15.84:0/4060184810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e88005170 msgr2=0x7f2e8800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.937+0000 7f2ea441a640 1 --2- 172.21.15.84:0/4060184810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e88005170 0x7f2e8800b980 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400a7c0 tx=0x7f2e880057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.937+0000 7f2ea29b7640 1 -- 172.21.15.84:0/4060184810 reap_dead start 2023-10-11T06:13:29.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.937+0000 7f2ea441a640 1 -- 172.21.15.84:0/4060184810 shutdown_connections 2023-10-11T06:13:29.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.937+0000 7f2ea441a640 1 -- 172.21.15.84:0/4060184810 >> 172.21.15.84:0/4060184810 conn(0x7f2e9c069c70 msgr2=0x7f2e9c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:29.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.937+0000 7f2ea441a640 1 -- 172.21.15.84:0/4060184810 shutdown_connections 2023-10-11T06:13:29.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.937+0000 7f2ea441a640 1 -- 172.21.15.84:0/4060184810 wait complete. 2023-10-11T06:13:29.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.939+0000 7f2ea441a640 1 Processor -- start 2023-10-11T06:13:29.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.939+0000 7f2ea441a640 1 -- start start 2023-10-11T06:13:29.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea441a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e88005170 0x7f2e9c13f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea441a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e9c142f10 0x7f2e9c0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea441a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15a480 0x7f2e9c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea441a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c15ae30 con 0x7f2e9c142f10 2023-10-11T06:13:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea441a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b3b70 con 0x7f2e9c15a480 2023-10-11T06:13:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea441a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b2f60 con 0x7f2e88005170 2023-10-11T06:13:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea19b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e9c142f10 0x7f2e9c0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea19b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e9c142f10 0x7f2e9c0b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33910/0 (socket says 172.21.15.84:33910) 2023-10-11T06:13:29.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.940+0000 7f2ea19b5640 1 -- 172.21.15.84:0/3240571033 learned_addr learned my addr 172.21.15.84:0/3240571033 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea21b6640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e88005170 0x7f2e9c13f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea29b7640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15a480 0x7f2e9c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea19b5640 1 -- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e88005170 msgr2=0x7f2e9c13f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea19b5640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e88005170 0x7f2e9c13f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea19b5640 1 -- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15a480 msgr2=0x7f2e9c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:29.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea19b5640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15a480 0x7f2e9c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:29.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea19b5640 1 -- 172.21.15.84:0/3240571033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0b3240 con 0x7f2e9c142f10 2023-10-11T06:13:29.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.941+0000 7f2ea19b5640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e9c142f10 0x7f2e9c0b2610 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2e90004790 tx=0x7f2e9000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.942+0000 7f2e8effd640 1 -- 172.21.15.84:0/3240571033 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90004030 con 0x7f2e9c142f10 2023-10-11T06:13:29.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.942+0000 7f2e8effd640 1 -- 172.21.15.84:0/3240571033 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e900041b0 con 0x7f2e9c142f10 2023-10-11T06:13:29.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.942+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c0b3520 con 0x7f2e9c142f10 2023-10-11T06:13:29.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.942+0000 7f2e8effd640 1 -- 172.21.15.84:0/3240571033 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90013680 con 0x7f2e9c142f10 2023-10-11T06:13:29.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.942+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c0b4000 con 0x7f2e9c142f10 2023-10-11T06:13:29.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.943+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c0b3ce0 con 0x7f2e9c142f10 2023-10-11T06:13:29.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.944+0000 7f2e8effd640 1 -- 172.21.15.84:0/3240571033 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2e900137e0 con 0x7f2e9c142f10 2023-10-11T06:13:29.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.945+0000 7f2e8effd640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2e8006a860 0x7f2e8006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:29.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.945+0000 7f2e8effd640 1 -- 172.21.15.84:0/3240571033 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f2e90088d20 con 0x7f2e9c142f10 2023-10-11T06:13:29.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.945+0000 7f2ea21b6640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2e8006a860 0x7f2e8006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:29.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.946+0000 7f2ea21b6640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2e8006a860 0x7f2e8006cd10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000f60 tx=0x7f2e98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:29.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:29.949+0000 7f2e8effd640 1 -- 172.21.15.84:0/3240571033 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2e90053d50 con 0x7f2e9c142f10 2023-10-11T06:13:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.137+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f2e9c073ed0 con 0x7f2e9c142f10 2023-10-11T06:13:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.138+0000 7f2e8effd640 1 -- 172.21.15.84:0/3240571033 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v41) v1 ==== 97+0+11 (secure 0 0 0) 0x7f2e900200a0 con 0x7f2e9c142f10 2023-10-11T06:13:30.139 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2e8006a860 msgr2=0x7f2e8006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2e8006a860 0x7f2e8006cd10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000f60 tx=0x7f2e98009040 comp rx=0 tx=0).stop 2023-10-11T06:13:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e9c142f10 msgr2=0x7f2e9c0b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e9c142f10 0x7f2e9c0b2610 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2e90004790 tx=0x7f2e9000e970 comp rx=0 tx=0).stop 2023-10-11T06:13:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 shutdown_connections 2023-10-11T06:13:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2e8006a860 0x7f2e8006cd10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e9c15a480 0x7f2e9c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e9c142f10 0x7f2e9c0b2610 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 --2- 172.21.15.84:0/3240571033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e88005170 0x7f2e9c13f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 >> 172.21.15.84:0/3240571033 conn(0x7f2e9c069c70 msgr2=0x7f2e9c06c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 shutdown_connections 2023-10-11T06:13:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.141+0000 7f2ea441a640 1 -- 172.21.15.84:0/3240571033 wait complete. 2023-10-11T06:13:30.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:13:30.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.376+0000 7f0a3d0f7640 1 Processor -- start 2023-10-11T06:13:30.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.377+0000 7f0a3d0f7640 1 -- start start 2023-10-11T06:13:30.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.378+0000 7f0a3d0f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a380cb540 0x7f0a38164f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.378+0000 7f0a3d0f7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a38071a90 con 0x7f0a380ca300 2023-10-11T06:13:30.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.378+0000 7f0a3d0f7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a380cb970 con 0x7f0a3815d4f0 2023-10-11T06:13:30.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.378+0000 7f0a3d0f7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a38072310 con 0x7f0a380cb540 2023-10-11T06:13:30.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.378+0000 7f0a3759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a380cb540 0x7f0a38164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.379+0000 7f0a3759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a380cb540 0x7f0a38164f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33916/0 (socket says 172.21.15.84:33916) 2023-10-11T06:13:30.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.379+0000 7f0a3759e640 1 -- 172.21.15.84:0/1241697437 learned_addr learned my addr 172.21.15.84:0/1241697437 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:30.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.378+0000 7f0a2ffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0a380ca300 0x7f0a380c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54364/0 (socket says 172.21.15.84:54364) 2023-10-11T06:13:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.379+0000 7f0a3759e640 1 -- 172.21.15.84:0/1241697437 >> v1:172.21.15.99:6789/0 conn(0x7f0a3815d4f0 legacy=0x7f0a380c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.379+0000 7f0a3759e640 1 -- 172.21.15.84:0/1241697437 >> v1:172.21.15.84:6789/0 conn(0x7f0a380ca300 legacy=0x7f0a380c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.379+0000 7f0a3759e640 1 -- 172.21.15.84:0/1241697437 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38070280 con 0x7f0a380cb540 2023-10-11T06:13:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.379+0000 7f0a3659c640 1 -- 172.21.15.84:0/1241697437 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613754476 0 0) 0x7f0a38071a90 con 0x7f0a380ca300 2023-10-11T06:13:30.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.380+0000 7f0a3759e640 1 --2- 172.21.15.84:0/1241697437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380cb540 0x7f0a38164f50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009e50 tx=0x7f0a3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b533f9a9fc0857 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3659c640 1 -- 172.21.15.84:0/1241697437 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3002ca70 con 0x7f0a380cb540 2023-10-11T06:13:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3659c640 1 -- 172.21.15.84:0/1241697437 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3002cbd0 con 0x7f0a380cb540 2023-10-11T06:13:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3659c640 1 -- 172.21.15.84:0/1241697437 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300343f0 con 0x7f0a380cb540 2023-10-11T06:13:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/1241697437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380cb540 msgr2=0x7f0a38164f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/1241697437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380cb540 0x7f0a38164f50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009e50 tx=0x7f0a3002bed0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/1241697437 shutdown_connections 2023-10-11T06:13:30.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/1241697437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380cb540 0x7f0a38164f50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/1241697437 >> 172.21.15.84:0/1241697437 conn(0x7f0a38069c70 msgr2=0x7f0a3806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:30.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.381+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/1241697437 shutdown_connections 2023-10-11T06:13:30.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.382+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/1241697437 wait complete. 2023-10-11T06:13:30.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.383+0000 7f0a3d0f7640 1 Processor -- start 2023-10-11T06:13:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.383+0000 7f0a3d0f7640 1 -- start start 2023-10-11T06:13:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.384+0000 7f0a3d0f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca300 0x7f0a380aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.384+0000 7f0a3d0f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a380cb540 0x7f0a380ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.384+0000 7f0a3d0f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3815d4f0 0x7f0a38156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.384+0000 7f0a3d0f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38148530 con 0x7f0a380ca300 2023-10-11T06:13:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.384+0000 7f0a3d0f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38147490 con 0x7f0a3815d4f0 2023-10-11T06:13:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.384+0000 7f0a3d0f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38147770 con 0x7f0a380cb540 2023-10-11T06:13:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca300 0x7f0a380aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca300 0x7f0a380aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33930/0 (socket says 172.21.15.84:33930) 2023-10-11T06:13:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 -- 172.21.15.84:0/424185253 learned_addr learned my addr 172.21.15.84:0/424185253 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a3759e640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3815d4f0 0x7f0a38156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 -- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a380cb540 msgr2=0x7f0a380ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a2ffff640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a380cb540 0x7f0a380ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a380cb540 0x7f0a380ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 -- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3815d4f0 msgr2=0x7f0a38156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3815d4f0 0x7f0a38156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 -- 172.21.15.84:0/424185253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a30009b30 con 0x7f0a380ca300 2023-10-11T06:13:30.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a3759e640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3815d4f0 0x7f0a38156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:30.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.385+0000 7f0a36d9d640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca300 0x7f0a380aed80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800b3e0 tx=0x7f0a2800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:30.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.386+0000 7f0a2f7fe640 1 -- 172.21.15.84:0/424185253 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28011830 con 0x7f0a380ca300 2023-10-11T06:13:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.386+0000 7f0a2f7fe640 1 -- 172.21.15.84:0/424185253 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a280119b0 con 0x7f0a380ca300 2023-10-11T06:13:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.386+0000 7f0a2f7fe640 1 -- 172.21.15.84:0/424185253 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28011da0 con 0x7f0a380ca300 2023-10-11T06:13:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.386+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a38147a50 con 0x7f0a380ca300 2023-10-11T06:13:30.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.386+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a38148010 con 0x7f0a380ca300 2023-10-11T06:13:30.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.387+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38073ed0 con 0x7f0a380ca300 2023-10-11T06:13:30.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.388+0000 7f0a2f7fe640 1 -- 172.21.15.84:0/424185253 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0a2801a400 con 0x7f0a380ca300 2023-10-11T06:13:30.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.389+0000 7f0a2f7fe640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a1806a830 0x7f0a1806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.389+0000 7f0a2f7fe640 1 -- 172.21.15.84:0/424185253 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f0a28089330 con 0x7f0a380ca300 2023-10-11T06:13:30.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.389+0000 7f0a2ffff640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a1806a830 0x7f0a1806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.390+0000 7f0a2ffff640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a1806a830 0x7f0a1806cce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0a381577f0 tx=0x7f0a20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:30.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.393+0000 7f0a2f7fe640 1 -- 172.21.15.84:0/424185253 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0a280543f0 con 0x7f0a380ca300 2023-10-11T06:13:30.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.619+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a380b09b0 con 0x7f0a380ca300 2023-10-11T06:13:30.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.621+0000 7f0a2f7fe640 1 -- 172.21.15.84:0/424185253 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4644 (secure 0 0 0) 0x7f0a28057a10 con 0x7f0a380ca300 2023-10-11T06:13:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:30.623 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:12:40.002715+0000","modified":"2023-10-11T06:12:51.393935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5334":{"gid":5334,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/2172719017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2172719017},{"type":"v1","addr":"172.21.15.99:6835","nonce":2172719017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T06:13:30.623 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-10-11T06:13:30.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.623+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a1806a830 msgr2=0x7f0a1806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.623+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a1806a830 0x7f0a1806cce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0a381577f0 tx=0x7f0a20009040 comp rx=0 tx=0).stop 2023-10-11T06:13:30.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.623+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca300 msgr2=0x7f0a380aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.623+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca300 0x7f0a380aed80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800b3e0 tx=0x7f0a2800b8a0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 shutdown_connections 2023-10-11T06:13:30.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a1806a830 0x7f0a1806cce0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3815d4f0 0x7f0a38156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a380cb540 0x7f0a380ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 --2- 172.21.15.84:0/424185253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a380ca300 0x7f0a380aed80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 >> 172.21.15.84:0/424185253 conn(0x7f0a38069c70 msgr2=0x7f0a380b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:30.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 shutdown_connections 2023-10-11T06:13:30.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.624+0000 7f0a3d0f7640 1 -- 172.21.15.84:0/424185253 wait complete. 2023-10-11T06:13:30.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:30.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.854+0000 7f203d533640 1 Processor -- start 2023-10-11T06:13:30.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.855+0000 7f203d533640 1 -- start start 2023-10-11T06:13:30.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.856+0000 7f203d533640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20380ceae0 0x7f203815a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.856+0000 7f203d533640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203815da50 con 0x7f20380b3e70 2023-10-11T06:13:30.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.856+0000 7f203d533640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20380cad10 con 0x7f203806fd80 2023-10-11T06:13:30.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.856+0000 7f203d533640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203815dca0 con 0x7f20380ceae0 2023-10-11T06:13:30.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.856+0000 7f2037fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f20380b3e70 0x7f20380ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54374/0 (socket says 172.21.15.84:54374) 2023-10-11T06:13:30.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.856+0000 7f2037fff640 1 -- 172.21.15.84:0/1011618883 learned_addr learned my addr 172.21.15.84:0/1011618883 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:30.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f202ffff640 1 --2- 172.21.15.84:0/1011618883 >> v2:172.21.15.99:3300/0 conn(0x7f20380ceae0 0x7f203815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770115038 0 0) 0x7f203815da50 con 0x7f20380b3e70 2023-10-11T06:13:30.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2018003520 con 0x7f20380b3e70 2023-10-11T06:13:30.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3477873191 0 0) 0x7f2018003520 con 0x7f20380b3e70 2023-10-11T06:13:30.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 >> v2:172.21.15.99:3300/0 conn(0x7f20380ceae0 msgr2=0x7f203815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:30.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 --2- 172.21.15.84:0/1011618883 >> v2:172.21.15.99:3300/0 conn(0x7f20380ceae0 0x7f203815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 >> v1:172.21.15.99:6789/0 conn(0x7f203806fd80 legacy=0x7f203815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:30.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203815df80 con 0x7f20380b3e70 2023-10-11T06:13:30.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f2028001670 con 0x7f20380b3e70 2023-10-11T06:13:30.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 >> v1:172.21.15.84:6789/0 conn(0x7f20380b3e70 legacy=0x7f20380ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2018003970 0x7f2018003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.857+0000 7f20377fe640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20180047c0 0x7f2018004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f20377fe640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2018005170 0x7f201800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203815dca0 con 0x7f20180047c0 2023-10-11T06:13:30.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201800bfd0 con 0x7f2018003970 2023-10-11T06:13:30.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201800c190 con 0x7f2018005170 2023-10-11T06:13:30.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f202ffff640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20180047c0 0x7f2018004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f2037fff640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2018003970 0x7f2018003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f203cd32640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2018005170 0x7f201800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f202ffff640 1 -- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2018005170 msgr2=0x7f201800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:30.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f202ffff640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2018005170 0x7f201800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f202ffff640 1 -- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2018003970 msgr2=0x7f2018003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:30.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f202ffff640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2018003970 0x7f2018003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f202ffff640 1 -- 172.21.15.84:0/1011618883 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018003520 con 0x7f20180047c0 2023-10-11T06:13:30.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.858+0000 7f203cd32640 1 -- 172.21.15.84:0/1011618883 reap_dead start 2023-10-11T06:13:30.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.859+0000 7f202ffff640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20180047c0 0x7f2018004c30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2020009a90 tx=0x7f20180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15b0c1b5badd37 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:30.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.860+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202002c6b0 con 0x7f20180047c0 2023-10-11T06:13:30.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.860+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202002c810 con 0x7f20180047c0 2023-10-11T06:13:30.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.860+0000 7f20377fe640 1 -- 172.21.15.84:0/1011618883 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202002cba0 con 0x7f20180047c0 2023-10-11T06:13:30.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.860+0000 7f203d533640 1 -- 172.21.15.84:0/1011618883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20180047c0 msgr2=0x7f2018004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.860+0000 7f203d533640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20180047c0 0x7f2018004c30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2020009a90 tx=0x7f20180057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.860+0000 7f203d533640 1 -- 172.21.15.84:0/1011618883 shutdown_connections 2023-10-11T06:13:30.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.861+0000 7f203d533640 1 --2- 172.21.15.84:0/1011618883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20180047c0 0x7f2018004c30 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.861+0000 7f203d533640 1 -- 172.21.15.84:0/1011618883 >> 172.21.15.84:0/1011618883 conn(0x7f2038069c70 msgr2=0x7f203806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:30.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.861+0000 7f203d533640 1 -- 172.21.15.84:0/1011618883 shutdown_connections 2023-10-11T06:13:30.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.861+0000 7f203d533640 1 -- 172.21.15.84:0/1011618883 wait complete. 2023-10-11T06:13:30.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.862+0000 7f203d533640 1 Processor -- start 2023-10-11T06:13:30.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.863+0000 7f203d533640 1 -- start start 2023-10-11T06:13:30.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203d533640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20180047c0 0x7f20380c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203d533640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20381f4ce0 0x7f20380aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203d533640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20380aeca0 0x7f20380af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203d533640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20381f5c80 con 0x7f20380aeca0 2023-10-11T06:13:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203d533640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20381f5150 con 0x7f20381f4ce0 2023-10-11T06:13:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203d533640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20381f53e0 con 0x7f20180047c0 2023-10-11T06:13:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20380aeca0 0x7f20380af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20380aeca0 0x7f20380af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33944/0 (socket says 172.21.15.84:33944) 2023-10-11T06:13:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 -- 172.21.15.84:0/3256839297 learned_addr learned my addr 172.21.15.84:0/3256839297 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f202ffff640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20381f4ce0 0x7f20380aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f2037fff640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20180047c0 0x7f20380c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 -- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20180047c0 msgr2=0x7f20380c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20180047c0 0x7f20380c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 -- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20381f4ce0 msgr2=0x7f20380aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20381f4ce0 0x7f20380aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f203cd32640 1 -- 172.21.15.84:0/3256839297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20381f56c0 con 0x7f20380aeca0 2023-10-11T06:13:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.864+0000 7f2037fff640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20180047c0 0x7f20380c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.865+0000 7f203cd32640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20380aeca0 0x7f20380af110 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f20380cbc50 tx=0x7f203000d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.865+0000 7f20357fa640 1 -- 172.21.15.84:0/3256839297 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20300048f0 con 0x7f20380aeca0 2023-10-11T06:13:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.865+0000 7f20357fa640 1 -- 172.21.15.84:0/3256839297 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2030004a50 con 0x7f20380aeca0 2023-10-11T06:13:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.865+0000 7f20357fa640 1 -- 172.21.15.84:0/3256839297 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030004e00 con 0x7f20380aeca0 2023-10-11T06:13:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.865+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20381f59a0 con 0x7f20380aeca0 2023-10-11T06:13:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.865+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20380ad410 con 0x7f20380aeca0 2023-10-11T06:13:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.866+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20380c5510 con 0x7f20380aeca0 2023-10-11T06:13:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.867+0000 7f20357fa640 1 -- 172.21.15.84:0/3256839297 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2030005030 con 0x7f20380aeca0 2023-10-11T06:13:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.868+0000 7f20357fa640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f201c06a860 0x7f201c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.868+0000 7f20357fa640 1 -- 172.21.15.84:0/3256839297 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f20300880a0 con 0x7f20380aeca0 2023-10-11T06:13:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.868+0000 7f2037fff640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f201c06a860 0x7f201c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.869+0000 7f2037fff640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f201c06a860 0x7f201c06cd10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2028001cb0 tx=0x7f2028001540 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:30.872+0000 7f20357fa640 1 -- 172.21.15.84:0/3256839297 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f20380c5510 con 0x7f20380aeca0 2023-10-11T06:13:31.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.061+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20380c58f0 con 0x7f20380aeca0 2023-10-11T06:13:31.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.063+0000 7f20357fa640 1 -- 172.21.15.84:0/3256839297 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7f20380c58f0 con 0x7f20380aeca0 2023-10-11T06:13:31.065 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:31.065 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:31.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f201c06a860 msgr2=0x7f201c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f201c06a860 0x7f201c06cd10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2028001cb0 tx=0x7f2028001540 comp rx=0 tx=0).stop 2023-10-11T06:13:31.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20380aeca0 msgr2=0x7f20380af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20380aeca0 0x7f20380af110 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f20380cbc50 tx=0x7f203000d830 comp rx=0 tx=0).stop 2023-10-11T06:13:31.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 shutdown_connections 2023-10-11T06:13:31.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f201c06a860 0x7f201c06cd10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20380aeca0 0x7f20380af110 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20381f4ce0 0x7f20380aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 --2- 172.21.15.84:0/3256839297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20180047c0 0x7f20380c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 >> 172.21.15.84:0/3256839297 conn(0x7f2038069c70 msgr2=0x7f20380ab990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:31.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 shutdown_connections 2023-10-11T06:13:31.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.066+0000 7f203d533640 1 -- 172.21.15.84:0/3256839297 wait complete. 2023-10-11T06:13:31.087 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T06:13:31.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:13:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.299+0000 7fb59bfff640 1 Processor -- start 2023-10-11T06:13:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.300+0000 7fb59bfff640 1 -- start start 2023-10-11T06:13:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.301+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 0x7fb59c164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.301+0000 7fb59bfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c0b3440 con 0x7fb59c0ce1c0 2023-10-11T06:13:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.301+0000 7fb59bfff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c0ad400 con 0x7fb59c0b1230 2023-10-11T06:13:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59bfff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0b1f60 con 0x7fb59c162ad0 2023-10-11T06:13:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb59c0ce1c0 0x7fb59c0ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54384/0 (socket says 172.21.15.84:54384) 2023-10-11T06:13:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59affd640 1 -- 172.21.15.84:0/1403098822 learned_addr learned my addr 172.21.15.84:0/1403098822 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:31.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936968477 0 0) 0x7fb59c0b3440 con 0x7fb59c0ce1c0 2023-10-11T06:13:31.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb588003520 con 0x7fb59c0ce1c0 2023-10-11T06:13:31.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59b7fe640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 0x7fb59c164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309668973 0 0) 0x7fb588003520 con 0x7fb59c0ce1c0 2023-10-11T06:13:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 msgr2=0x7fb59c164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.302+0000 7fb59a7fc640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 0x7fb59c164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 >> v1:172.21.15.99:6789/0 conn(0x7fb59c0b1230 legacy=0x7fb59c0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0ad6a0 con 0x7fb59c0ce1c0 2023-10-11T06:13:31.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fb58c001670 con 0x7fb59c0ce1c0 2023-10-11T06:13:31.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 >> v1:172.21.15.84:6789/0 conn(0x7fb59c0ce1c0 legacy=0x7fb59c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb588003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 0x7fb5880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588004d00 0x7fb588005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0b1f60 con 0x7fb5880039a0 2023-10-11T06:13:31.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58800bb10 con 0x7fb588004d00 2023-10-11T06:13:31.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58800bcd0 con 0x7fb59c162ad0 2023-10-11T06:13:31.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59affd640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb588003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb593fff640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 0x7fb5880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59b7fe640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588004d00 0x7fb588005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59affd640 1 -- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 msgr2=0x7fb5880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.303+0000 7fb59affd640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c162ad0 0x7fb5880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.304+0000 7fb59affd640 1 -- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588004d00 msgr2=0x7fb588005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.304+0000 7fb59affd640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb588004d00 0x7fb588005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.304+0000 7fb59affd640 1 -- 172.21.15.84:0/1403098822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588003520 con 0x7fb5880039a0 2023-10-11T06:13:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.305+0000 7fb59affd640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb588003db0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c0e0 tx=0x7fb5880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a122536327dbf server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.305+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c005ad0 con 0x7fb5880039a0 2023-10-11T06:13:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.305+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c005c30 con 0x7fb5880039a0 2023-10-11T06:13:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.305+0000 7fb59a7fc640 1 -- 172.21.15.84:0/1403098822 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c02c810 con 0x7fb5880039a0 2023-10-11T06:13:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.306+0000 7fb59bfff640 1 -- 172.21.15.84:0/1403098822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 msgr2=0x7fb588003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.306+0000 7fb59bfff640 1 --2- 172.21.15.84:0/1403098822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb588003db0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c0e0 tx=0x7fb5880057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.306+0000 7fb59b7fe640 1 -- 172.21.15.84:0/1403098822 reap_dead start 2023-10-11T06:13:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.306+0000 7fb59bfff640 1 -- 172.21.15.84:0/1403098822 shutdown_connections 2023-10-11T06:13:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.306+0000 7fb59bfff640 1 -- 172.21.15.84:0/1403098822 >> 172.21.15.84:0/1403098822 conn(0x7fb59c06a590 msgr2=0x7fb59c099970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:31.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.306+0000 7fb59bfff640 1 -- 172.21.15.84:0/1403098822 shutdown_connections 2023-10-11T06:13:31.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.306+0000 7fb59bfff640 1 -- 172.21.15.84:0/1403098822 wait complete. 2023-10-11T06:13:31.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.308+0000 7fb59bfff640 1 Processor -- start 2023-10-11T06:13:31.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.308+0000 7fb59bfff640 1 -- start start 2023-10-11T06:13:31.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb59c148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59c06c5d0 0x7fb59c06c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c1f0c40 0x7fb59c1f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c145740 con 0x7fb5880039a0 2023-10-11T06:13:31.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59bfff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c06b9c0 con 0x7fb59c06c5d0 2023-10-11T06:13:31.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59bfff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0b35a0 con 0x7fb59c1f0c40 2023-10-11T06:13:31.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb59c148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb59c148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33958/0 (socket says 172.21.15.84:33958) 2023-10-11T06:13:31.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 -- 172.21.15.84:0/2484902797 learned_addr learned my addr 172.21.15.84:0/2484902797 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:31.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 -- 172.21.15.84:0/2484902797 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c1f0c40 msgr2=0x7fb59c1f10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c1f0c40 0x7fb59c1f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 -- 172.21.15.84:0/2484902797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59c06c5d0 msgr2=0x7fb59c06c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb593fff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59c06c5d0 0x7fb59c06c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59c06c5d0 0x7fb59c06c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.309+0000 7fb59affd640 1 -- 172.21.15.84:0/2484902797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0b3880 con 0x7fb5880039a0 2023-10-11T06:13:31.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.310+0000 7fb59affd640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb59c148ee0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb58c02c3b0 tx=0x7fb58c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:31.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.310+0000 7fb5937fe640 1 -- 172.21.15.84:0/2484902797 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c02cc20 con 0x7fb5880039a0 2023-10-11T06:13:31.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.310+0000 7fb5937fe640 1 -- 172.21.15.84:0/2484902797 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c02cd80 con 0x7fb5880039a0 2023-10-11T06:13:31.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.310+0000 7fb5937fe640 1 -- 172.21.15.84:0/2484902797 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c007290 con 0x7fb5880039a0 2023-10-11T06:13:31.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.310+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c0b3b60 con 0x7fb5880039a0 2023-10-11T06:13:31.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.310+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c0b4120 con 0x7fb5880039a0 2023-10-11T06:13:31.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.311+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c0b42b0 con 0x7fb5880039a0 2023-10-11T06:13:31.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.312+0000 7fb5937fe640 1 -- 172.21.15.84:0/2484902797 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb58c032070 con 0x7fb5880039a0 2023-10-11T06:13:31.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.313+0000 7fb5937fe640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb57c06a860 0x7fb57c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.313+0000 7fb5937fe640 1 -- 172.21.15.84:0/2484902797 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fb58c0aa1d0 con 0x7fb5880039a0 2023-10-11T06:13:31.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.313+0000 7fb593fff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb57c06a860 0x7fb57c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.314+0000 7fb593fff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb57c06a860 0x7fb57c06cd10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb584007fd0 tx=0x7fb584009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:31.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.318+0000 7fb5937fe640 1 -- 172.21.15.84:0/2484902797 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb58c075330 con 0x7fb5880039a0 2023-10-11T06:13:31.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.548+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb59c04e890 con 0x7fb5880039a0 2023-10-11T06:13:31.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.549+0000 7fb5937fe640 1 -- 172.21.15.84:0/2484902797 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4644 (secure 0 0 0) 0x7fb58c078950 con 0x7fb5880039a0 2023-10-11T06:13:31.551 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:31.551 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:12:40.002715+0000","modified":"2023-10-11T06:12:51.393935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5334":{"gid":5334,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/2172719017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2172719017},{"type":"v1","addr":"172.21.15.99:6835","nonce":2172719017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T06:13:31.551 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-10-11T06:13:31.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb57c06a860 msgr2=0x7fb57c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb57c06a860 0x7fb57c06cd10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb584007fd0 tx=0x7fb584009040 comp rx=0 tx=0).stop 2023-10-11T06:13:31.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 msgr2=0x7fb59c148ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb59c148ee0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb58c02c3b0 tx=0x7fb58c001930 comp rx=0 tx=0).stop 2023-10-11T06:13:31.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 shutdown_connections 2023-10-11T06:13:31.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb57c06a860 0x7fb57c06cd10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb59c1f0c40 0x7fb59c1f10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb59c06c5d0 0x7fb59c06c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 --2- 172.21.15.84:0/2484902797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5880039a0 0x7fb59c148ee0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 >> 172.21.15.84:0/2484902797 conn(0x7fb59c06a590 msgr2=0x7fb59c06e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:31.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 shutdown_connections 2023-10-11T06:13:31.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.552+0000 7fb59bfff640 1 -- 172.21.15.84:0/2484902797 wait complete. 2023-10-11T06:13:31.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:13:31.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.786+0000 7fbe69739640 1 Processor -- start 2023-10-11T06:13:31.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.786+0000 7fbe69739640 1 -- start start 2023-10-11T06:13:31.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.787+0000 7fbe69739640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe640adb90 0x7fbe640caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.787+0000 7fbe69739640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe640cd300 0x7fbe640cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.787+0000 7fbe69739640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe640cb960 con 0x7fbe640ce0d0 2023-10-11T06:13:31.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe69739640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6415a340 con 0x7fbe640adb90 2023-10-11T06:13:31.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe69739640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe640ce4a0 con 0x7fbe640cd300 2023-10-11T06:13:31.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe627fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe640adb90 0x7fbe640caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe627fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe640adb90 0x7fbe640caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33964/0 (socket says 172.21.15.84:33964) 2023-10-11T06:13:31.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe627fc640 1 -- 172.21.15.84:0/1479183319 learned_addr learned my addr 172.21.15.84:0/1479183319 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe637fe640 1 --2- 172.21.15.84:0/1479183319 >> v2:172.21.15.99:3300/0 conn(0x7fbe640cd300 0x7fbe640cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe627fc640 1 -- 172.21.15.84:0/1479183319 >> v2:172.21.15.99:3300/0 conn(0x7fbe640cd300 msgr2=0x7fbe640cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe627fc640 1 --2- 172.21.15.84:0/1479183319 >> v2:172.21.15.99:3300/0 conn(0x7fbe640cd300 0x7fbe640cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe627fc640 1 -- 172.21.15.84:0/1479183319 >> v1:172.21.15.99:6789/0 conn(0x7fbe640ce0d0 legacy=0x7fbe640c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.788+0000 7fbe627fc640 1 -- 172.21.15.84:0/1479183319 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe641593f0 con 0x7fbe640adb90 2023-10-11T06:13:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.789+0000 7fbe627fc640 1 --2- 172.21.15.84:0/1479183319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe640caee0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbe6406fd30 tx=0x7fbe5802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9a4aa182cdaafb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.790+0000 7fbe61ffb640 1 -- 172.21.15.84:0/1479183319 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5802ee90 con 0x7fbe640adb90 2023-10-11T06:13:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.790+0000 7fbe61ffb640 1 -- 172.21.15.84:0/1479183319 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe58036440 con 0x7fbe640adb90 2023-10-11T06:13:31.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.790+0000 7fbe61ffb640 1 -- 172.21.15.84:0/1479183319 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580367d0 con 0x7fbe640adb90 2023-10-11T06:13:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.790+0000 7fbe69739640 1 -- 172.21.15.84:0/1479183319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 msgr2=0x7fbe640caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.790+0000 7fbe69739640 1 --2- 172.21.15.84:0/1479183319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe640caee0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbe6406fd30 tx=0x7fbe5802e0e0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.791+0000 7fbe69739640 1 -- 172.21.15.84:0/1479183319 shutdown_connections 2023-10-11T06:13:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.791+0000 7fbe69739640 1 --2- 172.21.15.84:0/1479183319 >> v2:172.21.15.99:3300/0 conn(0x7fbe640cd300 0x7fbe640cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.791+0000 7fbe69739640 1 --2- 172.21.15.84:0/1479183319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe640caee0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.791+0000 7fbe69739640 1 -- 172.21.15.84:0/1479183319 >> 172.21.15.84:0/1479183319 conn(0x7fbe64069c70 msgr2=0x7fbe6406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.791+0000 7fbe69739640 1 -- 172.21.15.84:0/1479183319 shutdown_connections 2023-10-11T06:13:31.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.791+0000 7fbe69739640 1 -- 172.21.15.84:0/1479183319 wait complete. 2023-10-11T06:13:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.792+0000 7fbe69739640 1 Processor -- start 2023-10-11T06:13:31.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.793+0000 7fbe69739640 1 -- start start 2023-10-11T06:13:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.793+0000 7fbe69739640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe6415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe69739640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe640cd300 0x7fbe6414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe69739640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe640ce0d0 0x7fbe6414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe69739640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b3100 con 0x7fbe640adb90 2023-10-11T06:13:31.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe69739640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b33a0 con 0x7fbe640ce0d0 2023-10-11T06:13:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe69739640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640b3680 con 0x7fbe640cd300 2023-10-11T06:13:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe6415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe6415ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33978/0 (socket says 172.21.15.84:33978) 2023-10-11T06:13:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2301453189 learned_addr learned my addr 172.21.15.84:0/2301453189 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:31.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe637fe640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe640ce0d0 0x7fbe6414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe627fc640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe640cd300 0x7fbe6414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe640cd300 msgr2=0x7fbe6414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe640cd300 0x7fbe6414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe640ce0d0 msgr2=0x7fbe6414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe640ce0d0 0x7fbe6414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe62ffd640 1 -- 172.21.15.84:0/2301453189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5800b770 con 0x7fbe640adb90 2023-10-11T06:13:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe627fc640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe640cd300 0x7fbe6414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.794+0000 7fbe637fe640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe640ce0d0 0x7fbe6414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:31.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.795+0000 7fbe62ffd640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe6415ac60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbe54004640 tx=0x7fbe5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.795+0000 7fbe4b7fe640 1 -- 172.21.15.84:0/2301453189 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54011790 con 0x7fbe640adb90 2023-10-11T06:13:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.795+0000 7fbe4b7fe640 1 -- 172.21.15.84:0/2301453189 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe540118f0 con 0x7fbe640adb90 2023-10-11T06:13:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.795+0000 7fbe4b7fe640 1 -- 172.21.15.84:0/2301453189 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54011ce0 con 0x7fbe640adb90 2023-10-11T06:13:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.795+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe640b3960 con 0x7fbe640adb90 2023-10-11T06:13:31.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.795+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe640b4160 con 0x7fbe640adb90 2023-10-11T06:13:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.796+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe64073ed0 con 0x7fbe640adb90 2023-10-11T06:13:31.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.797+0000 7fbe4b7fe640 1 -- 172.21.15.84:0/2301453189 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbe54012420 con 0x7fbe640adb90 2023-10-11T06:13:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.798+0000 7fbe4b7fe640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe5006a830 0x7fbe5006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.798+0000 7fbe4b7fe640 1 -- 172.21.15.84:0/2301453189 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fbe54088e70 con 0x7fbe640adb90 2023-10-11T06:13:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.798+0000 7fbe627fc640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe5006a830 0x7fbe5006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.799+0000 7fbe627fc640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe5006a830 0x7fbe5006cce0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe58038000 tx=0x7fbe58032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:31.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:31.802+0000 7fbe4b7fe640 1 -- 172.21.15.84:0/2301453189 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fbe54053ea0 con 0x7fbe640adb90 2023-10-11T06:13:32.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.028+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe640c9180 con 0x7fbe640adb90 2023-10-11T06:13:32.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.029+0000 7fbe4b7fe640 1 -- 172.21.15.84:0/2301453189 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4644 (secure 0 0 0) 0x7fbe540574c0 con 0x7fbe640adb90 2023-10-11T06:13:32.030 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:32.030 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:12:40.002715+0000","modified":"2023-10-11T06:12:51.393935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5334":{"gid":5334,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/2172719017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2172719017},{"type":"v1","addr":"172.21.15.99:6835","nonce":2172719017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-11T06:13:32.031 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-10-11T06:13:32.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.031+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe5006a830 msgr2=0x7fbe5006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.031+0000 7fbe69739640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe5006a830 0x7fbe5006cce0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe58038000 tx=0x7fbe58032000 comp rx=0 tx=0).stop 2023-10-11T06:13:32.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.031+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 msgr2=0x7fbe6415ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.031+0000 7fbe69739640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe6415ac60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbe54004640 tx=0x7fbe5400c8e0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 shutdown_connections 2023-10-11T06:13:32.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbe5006a830 0x7fbe5006cce0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe640ce0d0 0x7fbe6414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe640cd300 0x7fbe6414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 --2- 172.21.15.84:0/2301453189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe640adb90 0x7fbe6415ac60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 >> 172.21.15.84:0/2301453189 conn(0x7fbe64069c70 msgr2=0x7fbe6406c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:32.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 shutdown_connections 2023-10-11T06:13:32.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.032+0000 7fbe69739640 1 -- 172.21.15.84:0/2301453189 wait complete. 2023-10-11T06:13:32.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:32.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.256+0000 7f7e3a4f7640 1 Processor -- start 2023-10-11T06:13:32.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.256+0000 7f7e3a4f7640 1 -- start start 2023-10-11T06:13:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.258+0000 7f7e3a4f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e3406d270 0x7f7e34070860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.258+0000 7f7e3a4f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34070da0 0x7f7e3415a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.258+0000 7f7e3a4f7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e340cb4b0 con 0x7f7e34157e60 2023-10-11T06:13:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.258+0000 7f7e3a4f7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e340c9b90 con 0x7f7e3406d270 2023-10-11T06:13:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.258+0000 7f7e3a4f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3406c860 con 0x7f7e34070da0 2023-10-11T06:13:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e33fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7e34157e60 0x7f7e340c9480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54400/0 (socket says 172.21.15.84:54400) 2023-10-11T06:13:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e33fff640 1 -- 172.21.15.84:0/445491206 learned_addr learned my addr 172.21.15.84:0/445491206 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e337fe640 1 --2- 172.21.15.84:0/445491206 >> v2:172.21.15.84:3300/0 conn(0x7f7e3406d270 0x7f7e34070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e38a94640 1 --2- 172.21.15.84:0/445491206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34070da0 0x7f7e3415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e32ffd640 1 -- 172.21.15.84:0/445491206 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509180238 0 0) 0x7f7e340cb4b0 con 0x7f7e34157e60 2023-10-11T06:13:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e337fe640 1 -- 172.21.15.84:0/445491206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34070da0 msgr2=0x7f7e3415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e337fe640 1 --2- 172.21.15.84:0/445491206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34070da0 0x7f7e3415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e337fe640 1 -- 172.21.15.84:0/445491206 >> v1:172.21.15.84:6789/0 conn(0x7f7e34157e60 legacy=0x7f7e340c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.259+0000 7f7e337fe640 1 -- 172.21.15.84:0/445491206 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34071190 con 0x7f7e3406d270 2023-10-11T06:13:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.260+0000 7f7e337fe640 1 --2- 172.21.15.84:0/445491206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e34070860 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7e280097a0 tx=0x7f7e2802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fd6b54b12de1e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.260+0000 7f7e32ffd640 1 -- 172.21.15.84:0/445491206 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2802d040 con 0x7f7e3406d270 2023-10-11T06:13:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e32ffd640 1 -- 172.21.15.84:0/445491206 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28031810 con 0x7f7e3406d270 2023-10-11T06:13:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e32ffd640 1 -- 172.21.15.84:0/445491206 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28031ba0 con 0x7f7e3406d270 2023-10-11T06:13:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/445491206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 msgr2=0x7f7e34070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/445491206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e34070860 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7e280097a0 tx=0x7f7e2802b8b0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/445491206 shutdown_connections 2023-10-11T06:13:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/445491206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34070da0 0x7f7e3415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/445491206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e34070860 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/445491206 >> 172.21.15.84:0/445491206 conn(0x7f7e340b1630 msgr2=0x7f7e340cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/445491206 shutdown_connections 2023-10-11T06:13:32.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.261+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/445491206 wait complete. 2023-10-11T06:13:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.263+0000 7f7e3a4f7640 1 Processor -- start 2023-10-11T06:13:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.263+0000 7f7e3a4f7640 1 -- start start 2023-10-11T06:13:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e3a4f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e3414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e3a4f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e34070da0 0x7f7e340c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e3a4f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34157e60 0x7f7e340afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e3a4f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340b3930 con 0x7f7e3406d270 2023-10-11T06:13:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e3a4f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340b3c10 con 0x7f7e34070da0 2023-10-11T06:13:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e3a4f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340b3ef0 con 0x7f7e34157e60 2023-10-11T06:13:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e3414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.264+0000 7f7e33fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e3414fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34000/0 (socket says 172.21.15.84:34000) 2023-10-11T06:13:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e33fff640 1 -- 172.21.15.84:0/3831890831 learned_addr learned my addr 172.21.15.84:0/3831890831 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e33fff640 1 -- 172.21.15.84:0/3831890831 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34157e60 msgr2=0x7f7e340afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e33fff640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34157e60 0x7f7e340afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e33fff640 1 -- 172.21.15.84:0/3831890831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e34070da0 msgr2=0x7f7e340c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e33fff640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e34070da0 0x7f7e340c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e33fff640 1 -- 172.21.15.84:0/3831890831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009400 con 0x7f7e3406d270 2023-10-11T06:13:32.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e33fff640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e3414fe90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c4a0 tx=0x7f7e2400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:32.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e30ff9640 1 -- 172.21.15.84:0/3831890831 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24005da0 con 0x7f7e3406d270 2023-10-11T06:13:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.265+0000 7f7e30ff9640 1 -- 172.21.15.84:0/3831890831 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24007070 con 0x7f7e3406d270 2023-10-11T06:13:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.266+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e340b41d0 con 0x7f7e3406d270 2023-10-11T06:13:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.266+0000 7f7e30ff9640 1 -- 172.21.15.84:0/3831890831 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24007420 con 0x7f7e3406d270 2023-10-11T06:13:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.266+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e340c7ea0 con 0x7f7e3406d270 2023-10-11T06:13:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.267+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e340b46f0 con 0x7f7e3406d270 2023-10-11T06:13:32.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.268+0000 7f7e30ff9640 1 -- 172.21.15.84:0/3831890831 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f7e24007580 con 0x7f7e3406d270 2023-10-11T06:13:32.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.268+0000 7f7e30ff9640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7e2006a860 0x7f7e2006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.268+0000 7f7e30ff9640 1 -- 172.21.15.84:0/3831890831 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7f7e240886a0 con 0x7f7e3406d270 2023-10-11T06:13:32.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.269+0000 7f7e337fe640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7e2006a860 0x7f7e2006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.269+0000 7f7e337fe640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7e2006a860 0x7f7e2006cd10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7e28036000 tx=0x7f7e28030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:32.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.273+0000 7f7e30ff9640 1 -- 172.21.15.84:0/3831890831 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f7e340b46f0 con 0x7f7e3406d270 2023-10-11T06:13:32.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.464+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e340b46f0 con 0x7f7e3406d270 2023-10-11T06:13:32.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.468+0000 7f7e30ff9640 1 -- 172.21.15.84:0/3831890831 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+19663 (secure 0 0 0) 0x7f7e340b46f0 con 0x7f7e3406d270 2023-10-11T06:13:32.470 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:32.470 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":41,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:12:52.397006+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:12:38.260176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-11T06:12:39.011036+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-10-11T06:12:50.853138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:32.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7e2006a860 msgr2=0x7f7e2006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7e2006a860 0x7f7e2006cd10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7e28036000 tx=0x7f7e28030000 comp rx=0 tx=0).stop 2023-10-11T06:13:32.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 msgr2=0x7f7e3414fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e3414fe90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c4a0 tx=0x7f7e2400c960 comp rx=0 tx=0).stop 2023-10-11T06:13:32.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 shutdown_connections 2023-10-11T06:13:32.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7e2006a860 0x7f7e2006cd10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e34157e60 0x7f7e340afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e34070da0 0x7f7e340c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 --2- 172.21.15.84:0/3831890831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e3406d270 0x7f7e3414fe90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 >> 172.21.15.84:0/3831890831 conn(0x7f7e340b1630 msgr2=0x7f7e34069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 shutdown_connections 2023-10-11T06:13:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.471+0000 7f7e3a4f7640 1 -- 172.21.15.84:0/3831890831 wait complete. 2023-10-11T06:13:32.492 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T06:13:32.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.705+0000 7fe80e8ca640 1 Processor -- start 2023-10-11T06:13:32.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.706+0000 7fe80e8ca640 1 -- start start 2023-10-11T06:13:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80e8ca640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080ce0d0 0x7fe8080c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80e8ca640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe8080adb90 0x7fe8080caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80e8ca640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8080cd300 0x7fe8080cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80e8ca640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80815a340 con 0x7fe8080cd300 2023-10-11T06:13:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80e8ca640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8080ce4a0 con 0x7fe8080adb90 2023-10-11T06:13:32.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80e8ca640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080cb960 con 0x7fe8080ce0d0 2023-10-11T06:13:32.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80ce67640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8080cd300 0x7fe8080cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80ce67640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8080cd300 0x7fe8080cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34008/0 (socket says 172.21.15.84:34008) 2023-10-11T06:13:32.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.707+0000 7fe80ce67640 1 -- 172.21.15.84:0/2280015817 learned_addr learned my addr 172.21.15.84:0/2280015817 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:32.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.708+0000 7fe7fffff640 1 --2- 172.21.15.84:0/2280015817 >> v2:172.21.15.99:3300/0 conn(0x7fe8080adb90 0x7fe8080caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.708+0000 7fe807fff640 1 --2- 172.21.15.84:0/2280015817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080ce0d0 0x7fe8080c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.708+0000 7fe80ce67640 1 -- 172.21.15.84:0/2280015817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080ce0d0 msgr2=0x7fe8080c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:32.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.708+0000 7fe80ce67640 1 --2- 172.21.15.84:0/2280015817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080ce0d0 0x7fe8080c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.708+0000 7fe80ce67640 1 -- 172.21.15.84:0/2280015817 >> v2:172.21.15.99:3300/0 conn(0x7fe8080adb90 msgr2=0x7fe8080caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:32.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.708+0000 7fe80ce67640 1 --2- 172.21.15.84:0/2280015817 >> v2:172.21.15.99:3300/0 conn(0x7fe8080adb90 0x7fe8080caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.708+0000 7fe80ce67640 1 -- 172.21.15.84:0/2280015817 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8081593f0 con 0x7fe8080cd300 2023-10-11T06:13:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.709+0000 7fe80ce67640 1 --2- 172.21.15.84:0/2280015817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080cd300 0x7fe8080cb420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe80806f0a0 tx=0x7fe80002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d063891c2bf25c56 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.709+0000 7fe8077fe640 1 -- 172.21.15.84:0/2280015817 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80002ec80 con 0x7fe8080cd300 2023-10-11T06:13:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.709+0000 7fe8077fe640 1 -- 172.21.15.84:0/2280015817 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80002ede0 con 0x7fe8080cd300 2023-10-11T06:13:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.709+0000 7fe8077fe640 1 -- 172.21.15.84:0/2280015817 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800036610 con 0x7fe8080cd300 2023-10-11T06:13:32.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 -- 172.21.15.84:0/2280015817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080cd300 msgr2=0x7fe8080cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/2280015817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080cd300 0x7fe8080cb420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe80806f0a0 tx=0x7fe80002e0e0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 -- 172.21.15.84:0/2280015817 shutdown_connections 2023-10-11T06:13:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/2280015817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080cd300 0x7fe8080cb420 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/2280015817 >> v2:172.21.15.99:3300/0 conn(0x7fe8080adb90 0x7fe8080caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/2280015817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080ce0d0 0x7fe8080c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 -- 172.21.15.84:0/2280015817 >> 172.21.15.84:0/2280015817 conn(0x7fe808069c70 msgr2=0x7fe80806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 -- 172.21.15.84:0/2280015817 shutdown_connections 2023-10-11T06:13:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.710+0000 7fe80e8ca640 1 -- 172.21.15.84:0/2280015817 wait complete. 2023-10-11T06:13:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.712+0000 7fe80e8ca640 1 Processor -- start 2023-10-11T06:13:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.712+0000 7fe80e8ca640 1 -- start start 2023-10-11T06:13:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80e8ca640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080adb90 0x7fe808156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80e8ca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8080cd300 0x7fe8081545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80e8ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080ce0d0 0x7fe8080af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80e8ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c6ae0 con 0x7fe8080ce0d0 2023-10-11T06:13:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80e8ca640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c6d80 con 0x7fe8080cd300 2023-10-11T06:13:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80e8ca640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080c7060 con 0x7fe8080adb90 2023-10-11T06:13:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080ce0d0 0x7fe8080af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080ce0d0 0x7fe8080af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34014/0 (socket says 172.21.15.84:34014) 2023-10-11T06:13:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 -- 172.21.15.84:0/188187029 learned_addr learned my addr 172.21.15.84:0/188187029 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe807fff640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080adb90 0x7fe808156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 -- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080adb90 msgr2=0x7fe808156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080adb90 0x7fe808156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe7fffff640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8080cd300 0x7fe8081545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 -- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8080cd300 msgr2=0x7fe8081545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8080cd300 0x7fe8081545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.713+0000 7fe80ce67640 1 -- 172.21.15.84:0/188187029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009b30 con 0x7fe8080ce0d0 2023-10-11T06:13:32.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe807fff640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080adb90 0x7fe808156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe7fffff640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8080cd300 0x7fe8081545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe80ce67640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080ce0d0 0x7fe8080af810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe800038040 tx=0x7fe800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe8057fa640 1 -- 172.21.15.84:0/188187029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80002ec80 con 0x7fe8080ce0d0 2023-10-11T06:13:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe8057fa640 1 -- 172.21.15.84:0/188187029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe800009510 con 0x7fe8080ce0d0 2023-10-11T06:13:32.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe8057fa640 1 -- 172.21.15.84:0/188187029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80003f690 con 0x7fe8080ce0d0 2023-10-11T06:13:32.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8080c7340 con 0x7fe8080ce0d0 2023-10-11T06:13:32.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.714+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe808154b10 con 0x7fe8080ce0d0 2023-10-11T06:13:32.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.715+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe808073ed0 con 0x7fe8080ce0d0 2023-10-11T06:13:32.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.716+0000 7fe8057fa640 1 -- 172.21.15.84:0/188187029 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe80003f7f0 con 0x7fe8080ce0d0 2023-10-11T06:13:32.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.717+0000 7fe8057fa640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe7e806a830 0x7fe7e806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:32.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.717+0000 7fe8057fa640 1 -- 172.21.15.84:0/188187029 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6486+0+0 (secure 0 0 0) 0x7fe8000ab9c0 con 0x7fe8080ce0d0 2023-10-11T06:13:32.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.717+0000 7fe807fff640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe7e806a830 0x7fe7e806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:32.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.718+0000 7fe807fff640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe7e806a830 0x7fe7e806cce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8006fd0 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:32.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.721+0000 7fe8057fa640 1 -- 172.21.15.84:0/188187029 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe800076a90 con 0x7fe8080ce0d0 2023-10-11T06:13:32.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.948+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe8080ca380 con 0x7fe8080ce0d0 2023-10-11T06:13:32.950 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-11T06:13:32.949+0000 7feb66d4a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T06:13:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.966+0000 7fe8057fa640 1 -- 172.21.15.84:0/188187029 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe80007a0b0 con 0x7fe8080ce0d0 2023-10-11T06:13:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T06:13:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe7e806a830 msgr2=0x7fe7e806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe7e806a830 0x7fe7e806cce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8006fd0 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2023-10-11T06:13:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080ce0d0 msgr2=0x7fe8080af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080ce0d0 0x7fe8080af810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe800038040 tx=0x7fe800004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 shutdown_connections 2023-10-11T06:13:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe7e806a830 0x7fe7e806cce0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8080ce0d0 0x7fe8080af810 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8080cd300 0x7fe8081545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 --2- 172.21.15.84:0/188187029 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8080adb90 0x7fe808156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 >> 172.21.15.84:0/188187029 conn(0x7fe808069c70 msgr2=0x7fe8080b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 shutdown_connections 2023-10-11T06:13:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:32.969+0000 7fe80e8ca640 1 -- 172.21.15.84:0/188187029 wait complete. 2023-10-11T06:13:32.990 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T06:13:32.991 INFO:tasks.ceph.mds.b.smithi099.stderr: -3836> 2023-10-11T06:12:42.156+0000 7f782a45d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T06:13:33.063 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2023-10-11T06:13:33.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.208+0000 7f46711ee640 1 Processor -- start 2023-10-11T06:13:33.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.208+0000 7f46711ee640 1 -- start start 2023-10-11T06:13:33.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.209+0000 7f46711ee640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f466c15c290 0x7f466c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.209+0000 7f46711ee640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 0x7f466c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.209+0000 7f46711ee640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0d2a60 con 0x7f466c0d25e0 2023-10-11T06:13:33.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.209+0000 7f46711ee640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466c06cd10 con 0x7f466c15c290 2023-10-11T06:13:33.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.209+0000 7f46711ee640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0afb60 con 0x7f466c0d0c20 2023-10-11T06:13:33.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f466c0d25e0 0x7f466c0d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54414/0 (socket says 172.21.15.84:54414) 2023-10-11T06:13:33.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466259c640 1 -- 172.21.15.84:0/2693669521 learned_addr learned my addr 172.21.15.84:0/2693669521 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:33.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105429665 0 0) 0x7f466c0d2a60 con 0x7f466c0d25e0 2023-10-11T06:13:33.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4658003520 con 0x7f466c0d25e0 2023-10-11T06:13:33.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466b59e640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 0x7f466c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:33.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466ad9d640 1 --2- 172.21.15.84:0/2693669521 >> v2:172.21.15.99:3300/0 conn(0x7f466c15c290 0x7f466c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:33.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1880471160 0 0) 0x7f4658003520 con 0x7f466c0d25e0 2023-10-11T06:13:33.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 msgr2=0x7f466c0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:33.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 0x7f466c0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 >> v2:172.21.15.99:3300/0 conn(0x7f466c15c290 msgr2=0x7f466c0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:33.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 --2- 172.21.15.84:0/2693669521 >> v2:172.21.15.99:3300/0 conn(0x7f466c15c290 0x7f466c0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0afe40 con 0x7f466c0d25e0 2023-10-11T06:13:33.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.210+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f4654001670 con 0x7f466c0d25e0 2023-10-11T06:13:33.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466b59e640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 0x7f466c0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:13:33.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 >> v1:172.21.15.84:6789/0 conn(0x7f466c0d25e0 legacy=0x7f466c0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:33.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466a59c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4658003970 0x7f4658003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466a59c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46580047c0 0x7f4658004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466a59c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 0x7f4658005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c06cd10 con 0x7f46580047c0 2023-10-11T06:13:33.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0afb60 con 0x7f4658003970 2023-10-11T06:13:33.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465800bae0 con 0x7f466c0d0c20 2023-10-11T06:13:33.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466259c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46580047c0 0x7f4658004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:33.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466b59e640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 0x7f4658005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:33.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466259c640 1 -- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 msgr2=0x7f4658005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:33.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466259c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f466c0d0c20 0x7f4658005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466259c640 1 -- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4658003970 msgr2=0x7f4658003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:33.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466259c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4658003970 0x7f4658003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.211+0000 7f466259c640 1 -- 172.21.15.84:0/2693669521 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658003520 con 0x7f46580047c0 2023-10-11T06:13:33.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.212+0000 7f466259c640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46580047c0 0x7f4658004c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f465400c110 tx=0x7f46580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce94a8fbc975313c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:33.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654005ad0 con 0x7f46580047c0 2023-10-11T06:13:33.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4654005c50 con 0x7f46580047c0 2023-10-11T06:13:33.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f466a59c640 1 -- 172.21.15.84:0/2693669521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465402c830 con 0x7f46580047c0 2023-10-11T06:13:33.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f46711ee640 1 -- 172.21.15.84:0/2693669521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46580047c0 msgr2=0x7f4658004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:33.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f46711ee640 1 --2- 172.21.15.84:0/2693669521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46580047c0 0x7f4658004c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f465400c110 tx=0x7f46580057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f466b59e640 1 -- 172.21.15.84:0/2693669521 reap_dead start 2023-10-11T06:13:33.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f46711ee640 1 -- 172.21.15.84:0/2693669521 shutdown_connections 2023-10-11T06:13:33.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f46711ee640 1 -- 172.21.15.84:0/2693669521 >> 172.21.15.84:0/2693669521 conn(0x7f466c071cc0 msgr2=0x7f466c09bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:33.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f46711ee640 1 -- 172.21.15.84:0/2693669521 shutdown_connections 2023-10-11T06:13:33.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.213+0000 7f46711ee640 1 -- 172.21.15.84:0/2693669521 wait complete. 2023-10-11T06:13:33.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.215+0000 7f46711ee640 1 Processor -- start 2023-10-11T06:13:33.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.215+0000 7f46711ee640 1 -- start start 2023-10-11T06:13:33.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f46711ee640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46580047c0 0x7f466c09c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f46711ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c0cf3a0 0x7f466c151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f46711ee640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f466c0c87d0 0x7f466c0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f46711ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0c9180 con 0x7f466c0cf3a0 2023-10-11T06:13:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f46711ee640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c152530 con 0x7f466c0c87d0 2023-10-11T06:13:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f46711ee640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c09cc50 con 0x7f46580047c0 2023-10-11T06:13:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f466259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c0cf3a0 0x7f466c151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:33.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f466259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c0cf3a0 0x7f466c151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34042/0 (socket says 172.21.15.84:34042) 2023-10-11T06:13:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.216+0000 7f466259c640 1 -- 172.21.15.84:0/2127055237 learned_addr learned my addr 172.21.15.84:0/2127055237 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466ad9d640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46580047c0 0x7f466c09c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466259c640 1 -- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46580047c0 msgr2=0x7f466c09c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:33.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466259c640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46580047c0 0x7f466c09c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466259c640 1 -- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f466c0c87d0 msgr2=0x7f466c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:33.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466259c640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f466c0c87d0 0x7f466c0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466259c640 1 -- 172.21.15.84:0/2127055237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c09cf30 con 0x7f466c0cf3a0 2023-10-11T06:13:33.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466ad9d640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46580047c0 0x7f466c09c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:33.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.217+0000 7f466259c640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c0cf3a0 0x7f466c151ff0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f465402c3d0 tx=0x7f4654001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:33.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.218+0000 7f4663fff640 1 -- 172.21.15.84:0/2127055237 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465403c040 con 0x7f466c0cf3a0 2023-10-11T06:13:33.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.218+0000 7f4663fff640 1 -- 172.21.15.84:0/2127055237 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465402dcd0 con 0x7f466c0cf3a0 2023-10-11T06:13:33.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.218+0000 7f4663fff640 1 -- 172.21.15.84:0/2127055237 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540072b0 con 0x7f466c0cf3a0 2023-10-11T06:13:33.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.218+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c09d210 con 0x7f466c0cf3a0 2023-10-11T06:13:33.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.218+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c09d7d0 con 0x7f466c0cf3a0 2023-10-11T06:13:33.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.219+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c073ed0 con 0x7f466c0cf3a0 2023-10-11T06:13:33.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.220+0000 7f4663fff640 1 -- 172.21.15.84:0/2127055237 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4654032070 con 0x7f466c0cf3a0 2023-10-11T06:13:33.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.221+0000 7f4663fff640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f464c06a860 0x7f464c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:33.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.221+0000 7f4663fff640 1 -- 172.21.15.84:0/2127055237 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f46540aabc0 con 0x7f466c0cf3a0 2023-10-11T06:13:33.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.221+0000 7f466ad9d640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f464c06a860 0x7f464c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:33.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.222+0000 7f466ad9d640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f464c06a860 0x7f464c06cd10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f465c006a90 tx=0x7f465c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:33.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.225+0000 7f4663fff640 1 -- 172.21.15.84:0/2127055237 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4654075cd0 con 0x7f466c0cf3a0 2023-10-11T06:13:33.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.454+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f466c15ba40 con 0x7f466c0cf3a0 2023-10-11T06:13:33.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.970+0000 7f4663fff640 1 -- 172.21.15.84:0/2127055237 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f46540792f0 con 0x7f466c0cf3a0 2023-10-11T06:13:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f464c06a860 msgr2=0x7f464c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f464c06a860 0x7f464c06cd10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f465c006a90 tx=0x7f465c008040 comp rx=0 tx=0).stop 2023-10-11T06:13:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c0cf3a0 msgr2=0x7f466c151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c0cf3a0 0x7f466c151ff0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f465402c3d0 tx=0x7f4654001930 comp rx=0 tx=0).stop 2023-10-11T06:13:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 shutdown_connections 2023-10-11T06:13:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f464c06a860 0x7f464c06cd10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f466c0c87d0 0x7f466c0c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c0cf3a0 0x7f466c151ff0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 --2- 172.21.15.84:0/2127055237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46580047c0 0x7f466c09c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:33.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 >> 172.21.15.84:0/2127055237 conn(0x7f466c071cc0 msgr2=0x7f466c0d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:33.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 shutdown_connections 2023-10-11T06:13:33.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:33.974+0000 7f46711ee640 1 -- 172.21.15.84:0/2127055237 wait complete. 2023-10-11T06:13:33.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T06:13:34.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.211+0000 7effc4026640 1 Processor -- start 2023-10-11T06:13:34.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.211+0000 7effc4026640 1 -- start start 2023-10-11T06:13:34.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc4026640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7effbc156810 0x7effbc06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:34.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc4026640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156be0 0x7effbc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:34.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc4026640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effbc06c170 0x7effbc06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:34.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc4026640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0ce140 con 0x7effbc156810 2023-10-11T06:13:34.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc4026640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0cde80 con 0x7effbc06c170 2023-10-11T06:13:34.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc4026640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc15da30 con 0x7effbc156be0 2023-10-11T06:13:34.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc1dc2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7effbc156810 0x7effbc06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:34.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc1dc2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7effbc156810 0x7effbc06d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34056/0 (socket says 172.21.15.84:34056) 2023-10-11T06:13:34.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.213+0000 7effc1dc2640 1 -- 172.21.15.84:0/1570146119 learned_addr learned my addr 172.21.15.84:0/1570146119 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:34.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.214+0000 7effc1dc2640 1 -- 172.21.15.84:0/1570146119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156be0 msgr2=0x7effbc15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:34.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.214+0000 7effc1dc2640 1 --2- 172.21.15.84:0/1570146119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156be0 0x7effbc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:34.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.214+0000 7effc1dc2640 1 -- 172.21.15.84:0/1570146119 >> v2:172.21.15.99:3300/0 conn(0x7effbc06c170 msgr2=0x7effbc06c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:34.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.214+0000 7effc1dc2640 1 --2- 172.21.15.84:0/1570146119 >> v2:172.21.15.99:3300/0 conn(0x7effbc06c170 0x7effbc06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:34.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.214+0000 7effc1dc2640 1 -- 172.21.15.84:0/1570146119 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc06df80 con 0x7effbc156810 2023-10-11T06:13:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.215+0000 7effc1dc2640 1 --2- 172.21.15.84:0/1570146119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc156810 0x7effbc06d8e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7effbc0affd0 tx=0x7effb802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a875e59d06b6c02 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.215+0000 7effc0dc0640 1 -- 172.21.15.84:0/1570146119 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb80317e0 con 0x7effbc156810 2023-10-11T06:13:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.215+0000 7effc0dc0640 1 -- 172.21.15.84:0/1570146119 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8031940 con 0x7effbc156810 2023-10-11T06:13:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.215+0000 7effc0dc0640 1 -- 172.21.15.84:0/1570146119 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8031cd0 con 0x7effbc156810 2023-10-11T06:13:34.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 -- 172.21.15.84:0/1570146119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc156810 msgr2=0x7effbc06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 --2- 172.21.15.84:0/1570146119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc156810 0x7effbc06d8e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7effbc0affd0 tx=0x7effb802b880 comp rx=0 tx=0).stop 2023-10-11T06:13:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 -- 172.21.15.84:0/1570146119 shutdown_connections 2023-10-11T06:13:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 --2- 172.21.15.84:0/1570146119 >> v2:172.21.15.99:3300/0 conn(0x7effbc06c170 0x7effbc06c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 --2- 172.21.15.84:0/1570146119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156be0 0x7effbc15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:34.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 --2- 172.21.15.84:0/1570146119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc156810 0x7effbc06d8e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:34.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 -- 172.21.15.84:0/1570146119 >> 172.21.15.84:0/1570146119 conn(0x7effbc069c70 msgr2=0x7effbc06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:34.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 -- 172.21.15.84:0/1570146119 shutdown_connections 2023-10-11T06:13:34.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.216+0000 7effc4026640 1 -- 172.21.15.84:0/1570146119 wait complete. 2023-10-11T06:13:34.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.218+0000 7effc4026640 1 Processor -- start 2023-10-11T06:13:34.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.218+0000 7effc4026640 1 -- start start 2023-10-11T06:13:34.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc4026640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc06c170 0x7effbc1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:34.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc4026640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156810 0x7effbc1f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc4026640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc156be0 0x7effbc1f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc4026640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc15ff60 con 0x7effbc06c170 2023-10-11T06:13:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc4026640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0ad870 con 0x7effbc156be0 2023-10-11T06:13:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc4026640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc15f630 con 0x7effbc156810 2023-10-11T06:13:34.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc1dc2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc06c170 0x7effbc1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc1dc2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc06c170 0x7effbc1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34060/0 (socket says 172.21.15.84:34060) 2023-10-11T06:13:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.219+0000 7effc1dc2640 1 -- 172.21.15.84:0/826073960 learned_addr learned my addr 172.21.15.84:0/826073960 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc1dc2640 1 -- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156810 msgr2=0x7effbc1f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc15c1640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156810 0x7effbc1f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc1dc2640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156810 0x7effbc1f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:34.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc25c3640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc156be0 0x7effbc1f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc1dc2640 1 -- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc156be0 msgr2=0x7effbc1f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc1dc2640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc156be0 0x7effbc1f2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc1dc2640 1 -- 172.21.15.84:0/826073960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80094d0 con 0x7effbc06c170 2023-10-11T06:13:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effc1dc2640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc06c170 0x7effbc1512e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7effb802b8c0 tx=0x7effb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:34.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.220+0000 7effb27fc640 1 -- 172.21.15.84:0/826073960 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8032940 con 0x7effbc06c170 2023-10-11T06:13:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.221+0000 7effb27fc640 1 -- 172.21.15.84:0/826073960 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8032aa0 con 0x7effbc06c170 2023-10-11T06:13:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.221+0000 7effb27fc640 1 -- 172.21.15.84:0/826073960 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8032e50 con 0x7effbc06c170 2023-10-11T06:13:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.221+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc15f910 con 0x7effbc06c170 2023-10-11T06:13:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.221+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc151820 con 0x7effbc06c170 2023-10-11T06:13:34.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.222+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc0cbd30 con 0x7effbc06c170 2023-10-11T06:13:34.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.223+0000 7effb27fc640 1 -- 172.21.15.84:0/826073960 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7effb804e450 con 0x7effbc06c170 2023-10-11T06:13:34.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.224+0000 7effb27fc640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7effa006a860 0x7effa006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.224+0000 7effb27fc640 1 -- 172.21.15.84:0/826073960 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6572+0+0 (secure 0 0 0) 0x7effb80ab3a0 con 0x7effbc06c170 2023-10-11T06:13:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.224+0000 7effc15c1640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7effa006a860 0x7effa006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.225+0000 7effc15c1640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7effa006a860 0x7effa006cd10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7effbc15d4a0 tx=0x7effac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:34.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.228+0000 7effb27fc640 1 -- 172.21.15.84:0/826073960 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7effb8076410 con 0x7effbc06c170 2023-10-11T06:13:34.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:34.418+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7effbc073ed0 con 0x7effbc06c170 2023-10-11T06:13:35.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.286+0000 7effb27fc640 1 -- 172.21.15.84:0/826073960 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7effb8079a30 con 0x7effbc06c170 2023-10-11T06:13:35.288 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-11T06:13:35.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.291+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7effa006a860 msgr2=0x7effa006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:35.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.291+0000 7effc4026640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7effa006a860 0x7effa006cd10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7effbc15d4a0 tx=0x7effac008040 comp rx=0 tx=0).stop 2023-10-11T06:13:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.291+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc06c170 msgr2=0x7effbc1512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.291+0000 7effc4026640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc06c170 0x7effbc1512e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7effb802b8c0 tx=0x7effb80046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.294+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 shutdown_connections 2023-10-11T06:13:35.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.294+0000 7effc4026640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7effa006a860 0x7effa006cd10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.294+0000 7effc4026640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc156be0 0x7effbc1f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.294+0000 7effc4026640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effbc156810 0x7effbc1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.294+0000 7effc4026640 1 --2- 172.21.15.84:0/826073960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effbc06c170 0x7effbc1512e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.294+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 >> 172.21.15.84:0/826073960 conn(0x7effbc069c70 msgr2=0x7effbc0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:35.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.294+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 shutdown_connections 2023-10-11T06:13:35.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.295+0000 7effc4026640 1 -- 172.21.15.84:0/826073960 wait complete. 2023-10-11T06:13:35.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T06:13:35.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.526+0000 7f53a9567640 1 Processor -- start 2023-10-11T06:13:35.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.527+0000 7f53a9567640 1 -- start start 2023-10-11T06:13:35.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.528+0000 7f53a9567640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 0x7f53a406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:35.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a9567640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f53a4156be0 0x7f53a415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:35.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a9567640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a40cde80 con 0x7f53a4156810 2023-10-11T06:13:35.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a9567640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a40ce140 con 0x7f53a4156be0 2023-10-11T06:13:35.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a9567640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a415da30 con 0x7f53a406c880 2023-10-11T06:13:35.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a2ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f53a4156810 0x7f53a406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54422/0 (socket says 172.21.15.84:54422) 2023-10-11T06:13:35.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a2ffd640 1 -- 172.21.15.84:0/3628262305 learned_addr learned my addr 172.21.15.84:0/3628262305 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:35.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a37fe640 1 --2- 172.21.15.84:0/3628262305 >> v2:172.21.15.84:3300/0 conn(0x7f53a4156be0 0x7f53a415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:35.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a37fe640 1 -- 172.21.15.84:0/3628262305 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 msgr2=0x7f53a406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:35.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a37fe640 1 --2- 172.21.15.84:0/3628262305 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 0x7f53a406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a37fe640 1 -- 172.21.15.84:0/3628262305 >> v1:172.21.15.84:6789/0 conn(0x7f53a4156810 legacy=0x7f53a406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:35.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.529+0000 7f53a37fe640 1 -- 172.21.15.84:0/3628262305 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a406df80 con 0x7f53a4156be0 2023-10-11T06:13:35.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.531+0000 7f53a37fe640 1 --2- 172.21.15.84:0/3628262305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156be0 0x7f53a415d4f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f539c009d90 tx=0x7f539c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ea1feeae08577f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:35.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.531+0000 7f53a27fc640 1 -- 172.21.15.84:0/3628262305 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c02cbc0 con 0x7f53a4156be0 2023-10-11T06:13:35.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.531+0000 7f53a27fc640 1 -- 172.21.15.84:0/3628262305 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c02cd20 con 0x7f53a4156be0 2023-10-11T06:13:35.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.531+0000 7f53a27fc640 1 -- 172.21.15.84:0/3628262305 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c034660 con 0x7f53a4156be0 2023-10-11T06:13:35.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 -- 172.21.15.84:0/3628262305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156be0 msgr2=0x7f53a415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:35.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 --2- 172.21.15.84:0/3628262305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156be0 0x7f53a415d4f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f539c009d90 tx=0x7f539c02be10 comp rx=0 tx=0).stop 2023-10-11T06:13:35.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 -- 172.21.15.84:0/3628262305 shutdown_connections 2023-10-11T06:13:35.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 --2- 172.21.15.84:0/3628262305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156be0 0x7f53a415d4f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 --2- 172.21.15.84:0/3628262305 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 0x7f53a406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 -- 172.21.15.84:0/3628262305 >> 172.21.15.84:0/3628262305 conn(0x7f53a4069c70 msgr2=0x7f53a406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:35.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 -- 172.21.15.84:0/3628262305 shutdown_connections 2023-10-11T06:13:35.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.532+0000 7f53a9567640 1 -- 172.21.15.84:0/3628262305 wait complete. 2023-10-11T06:13:35.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.534+0000 7f53a9567640 1 Processor -- start 2023-10-11T06:13:35.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.534+0000 7f53a9567640 1 -- start start 2023-10-11T06:13:35.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.535+0000 7f53a9567640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 0x7f53a414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:35.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.535+0000 7f53a9567640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156810 0x7f53a415f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:35.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.535+0000 7f53a9567640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a4156be0 0x7f53a415f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:35.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f53a9567640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4153de0 con 0x7f53a4156810 2023-10-11T06:13:35.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f53a9567640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41540c0 con 0x7f53a4156be0 2023-10-11T06:13:35.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f53a9567640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41543a0 con 0x7f53a406c880 2023-10-11T06:13:35.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156810 0x7f53a415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:35.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156810 0x7f53a415f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34084/0 (socket says 172.21.15.84:34084) 2023-10-11T06:13:35.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 -- 172.21.15.84:0/1244298057 learned_addr learned my addr 172.21.15.84:0/1244298057 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:35.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f53a2ffd640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 0x7f53a414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:35.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f53a37fe640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a4156be0 0x7f53a415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:35.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 -- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 msgr2=0x7f53a414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:35.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 0x7f53a414e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 -- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a4156be0 msgr2=0x7f53a415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:35.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a4156be0 0x7f53a415f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:35.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f539bfff640 1 -- 172.21.15.84:0/1244298057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c009a70 con 0x7f53a4156810 2023-10-11T06:13:35.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.536+0000 7f53a37fe640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a4156be0 0x7f53a415f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:35.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.537+0000 7f539bfff640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156810 0x7f53a415f410 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f538c00b450 tx=0x7f538c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:35.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.537+0000 7f539b7fe640 1 -- 172.21.15.84:0/1244298057 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c011890 con 0x7f53a4156810 2023-10-11T06:13:35.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.537+0000 7f539b7fe640 1 -- 172.21.15.84:0/1244298057 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c0119f0 con 0x7f53a4156810 2023-10-11T06:13:35.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.537+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a4154680 con 0x7f53a4156810 2023-10-11T06:13:35.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.537+0000 7f539b7fe640 1 -- 172.21.15.84:0/1244298057 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c011da0 con 0x7f53a4156810 2023-10-11T06:13:35.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.537+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a4153970 con 0x7f53a4156810 2023-10-11T06:13:35.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.538+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a4153b90 con 0x7f53a4156810 2023-10-11T06:13:35.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.539+0000 7f539b7fe640 1 -- 172.21.15.84:0/1244298057 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f538c01a400 con 0x7f53a4156810 2023-10-11T06:13:35.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.540+0000 7f539b7fe640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f538406a860 0x7f538406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:35.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.540+0000 7f539b7fe640 1 -- 172.21.15.84:0/1244298057 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6134+0+0 (secure 0 0 0) 0x7f538c089430 con 0x7f53a4156810 2023-10-11T06:13:35.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.540+0000 7f53a2ffd640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f538406a860 0x7f538406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:35.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.541+0000 7f53a2ffd640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f538406a860 0x7f538406cd10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5394005880 tx=0x7f5394009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:35.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.545+0000 7f539b7fe640 1 -- 172.21.15.84:0/1244298057 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f538c054650 con 0x7f53a4156810 2023-10-11T06:13:35.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:35.736+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53a40cbd30 con 0x7f53a4156810 2023-10-11T06:13:36.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.289+0000 7f539b7fe640 1 -- 172.21.15.84:0/1244298057 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) v1 ==== 169+0+0 (secure 0 0 0) 0x7f538c009280 con 0x7f53a4156810 2023-10-11T06:13:36.292 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-11T06:13:36.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.297+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f538406a860 msgr2=0x7f538406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.297+0000 7f53a9567640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f538406a860 0x7f538406cd10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5394005880 tx=0x7f5394009040 comp rx=0 tx=0).stop 2023-10-11T06:13:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.297+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156810 msgr2=0x7f53a415f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:36.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.297+0000 7f53a9567640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156810 0x7f53a415f410 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f538c00b450 tx=0x7f538c00b910 comp rx=0 tx=0).stop 2023-10-11T06:13:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.298+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 shutdown_connections 2023-10-11T06:13:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.298+0000 7f53a9567640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f538406a860 0x7f538406cd10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.298+0000 7f53a9567640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a4156be0 0x7f53a415f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.298+0000 7f53a9567640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53a4156810 0x7f53a415f410 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.298+0000 7f53a9567640 1 --2- 172.21.15.84:0/1244298057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53a406c880 0x7f53a414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.298+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 >> 172.21.15.84:0/1244298057 conn(0x7f53a4069c70 msgr2=0x7f53a40b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:36.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.299+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 shutdown_connections 2023-10-11T06:13:36.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.299+0000 7f53a9567640 1 -- 172.21.15.84:0/1244298057 wait complete. 2023-10-11T06:13:36.325 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2023-10-11T06:13:36.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.528+0000 7f3ef62f7640 1 Processor -- start 2023-10-11T06:13:36.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.528+0000 7f3ef62f7640 1 -- start start 2023-10-11T06:13:36.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3ef62f7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ef015a0b0 0x7f3ef00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:36.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3ef62f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ef00ced30 0x7f3ef006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3ef62f7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef0071380 con 0x7f3ef00c8d00 2023-10-11T06:13:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3ef62f7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef006e0f0 con 0x7f3ef00ced30 2023-10-11T06:13:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3ef62f7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00cbae0 con 0x7f3ef015a0b0 2023-10-11T06:13:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3ef4894640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ef00ced30 0x7f3ef006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:36.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3eef7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3ef00c8d00 0x7f3ef00ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54432/0 (socket says 172.21.15.84:54432) 2023-10-11T06:13:36.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3eef7fe640 1 -- 172.21.15.84:0/1962715842 learned_addr learned my addr 172.21.15.84:0/1962715842 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:36.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.530+0000 7f3eeffff640 1 --2- 172.21.15.84:0/1962715842 >> v2:172.21.15.99:3300/0 conn(0x7f3ef015a0b0 0x7f3ef00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.531+0000 7f3ef4894640 1 -- 172.21.15.84:0/1962715842 >> v2:172.21.15.99:3300/0 conn(0x7f3ef015a0b0 msgr2=0x7f3ef00cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.531+0000 7f3ef4894640 1 --2- 172.21.15.84:0/1962715842 >> v2:172.21.15.99:3300/0 conn(0x7f3ef015a0b0 0x7f3ef00cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.531+0000 7f3ef4894640 1 -- 172.21.15.84:0/1962715842 >> v1:172.21.15.84:6789/0 conn(0x7f3ef00c8d00 legacy=0x7f3ef00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.531+0000 7f3ef4894640 1 -- 172.21.15.84:0/1962715842 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef006de10 con 0x7f3ef00ced30 2023-10-11T06:13:36.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.532+0000 7f3ef4894640 1 --2- 172.21.15.84:0/1962715842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef00ced30 0x7f3ef006c170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006ad30 tx=0x7f3ee802bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14303b13842c647f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.532+0000 7f3eeeffd640 1 -- 172.21.15.84:0/1962715842 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee802d910 con 0x7f3ef00ced30 2023-10-11T06:13:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.532+0000 7f3eeeffd640 1 -- 172.21.15.84:0/1962715842 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee802da70 con 0x7f3ef00ced30 2023-10-11T06:13:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.532+0000 7f3eeeffd640 1 -- 172.21.15.84:0/1962715842 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee802de00 con 0x7f3ef00ced30 2023-10-11T06:13:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 -- 172.21.15.84:0/1962715842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef00ced30 msgr2=0x7f3ef006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:36.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/1962715842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef00ced30 0x7f3ef006c170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006ad30 tx=0x7f3ee802bbd0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 -- 172.21.15.84:0/1962715842 shutdown_connections 2023-10-11T06:13:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/1962715842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef00ced30 0x7f3ef006c170 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/1962715842 >> v2:172.21.15.99:3300/0 conn(0x7f3ef015a0b0 0x7f3ef00cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 -- 172.21.15.84:0/1962715842 >> 172.21.15.84:0/1962715842 conn(0x7f3ef0069c70 msgr2=0x7f3ef006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 -- 172.21.15.84:0/1962715842 shutdown_connections 2023-10-11T06:13:36.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.533+0000 7f3ef62f7640 1 -- 172.21.15.84:0/1962715842 wait complete. 2023-10-11T06:13:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.535+0000 7f3ef62f7640 1 Processor -- start 2023-10-11T06:13:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.535+0000 7f3ef62f7640 1 -- start start 2023-10-11T06:13:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef62f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ef00c8d00 0x7f3ef00b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef62f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00ced30 0x7f3ef014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:36.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef62f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef015a0b0 0x7f3ef01f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef62f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef01f1d60 con 0x7f3ef015a0b0 2023-10-11T06:13:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef62f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef014eb30 con 0x7f3ef00ced30 2023-10-11T06:13:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef62f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0154640 con 0x7f3ef00c8d00 2023-10-11T06:13:36.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef015a0b0 0x7f3ef01f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef015a0b0 0x7f3ef01f2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34106/0 (socket says 172.21.15.84:34106) 2023-10-11T06:13:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 -- 172.21.15.84:0/3565886477 learned_addr learned my addr 172.21.15.84:0/3565886477 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3eef7fe640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00ced30 0x7f3ef014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3eeffff640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ef00c8d00 0x7f3ef00b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:36.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 -- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ef00c8d00 msgr2=0x7f3ef00b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ef00c8d00 0x7f3ef00b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 -- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00ced30 msgr2=0x7f3ef014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00ced30 0x7f3ef014e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3ef4894640 1 -- 172.21.15.84:0/3565886477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8009a70 con 0x7f3ef015a0b0 2023-10-11T06:13:36.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.536+0000 7f3eeffff640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ef00c8d00 0x7f3ef00b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.537+0000 7f3ef4894640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef015a0b0 0x7f3ef01f2e20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8037040 tx=0x7f3ee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.537+0000 7f3eecff9640 1 -- 172.21.15.84:0/3565886477 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8035d70 con 0x7f3ef015a0b0 2023-10-11T06:13:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.537+0000 7f3eecff9640 1 -- 172.21.15.84:0/3565886477 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee803f430 con 0x7f3ef015a0b0 2023-10-11T06:13:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.537+0000 7f3eecff9640 1 -- 172.21.15.84:0/3565886477 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee803f7e0 con 0x7f3ef015a0b0 2023-10-11T06:13:36.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.537+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef0154920 con 0x7f3ef015a0b0 2023-10-11T06:13:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.537+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef0154ee0 con 0x7f3ef015a0b0 2023-10-11T06:13:36.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.538+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef01f1ed0 con 0x7f3ef015a0b0 2023-10-11T06:13:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.539+0000 7f3eecff9640 1 -- 172.21.15.84:0/3565886477 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3ee803f940 con 0x7f3ef015a0b0 2023-10-11T06:13:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.540+0000 7f3eecff9640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3edc06a830 0x7f3edc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:36.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.540+0000 7f3eecff9640 1 -- 172.21.15.84:0/3565886477 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5691+0+0 (secure 0 0 0) 0x7f3ee8031070 con 0x7f3ef015a0b0 2023-10-11T06:13:36.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.540+0000 7f3eeffff640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3edc06a830 0x7f3edc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:36.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.541+0000 7f3eeffff640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3edc06a830 0x7f3edc06cce0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0006fd0 tx=0x7f3ee0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:36.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.545+0000 7f3eecff9640 1 -- 172.21.15.84:0/3565886477 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3ee8077230 con 0x7f3ef015a0b0 2023-10-11T06:13:36.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:36.733+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ef0073ed0 con 0x7f3ef015a0b0 2023-10-11T06:13:37.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.314+0000 7f3eecff9640 1 -- 172.21.15.84:0/3565886477 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v45) v1 ==== 160+0+0 (secure 0 0 0) 0x7f3ee804f020 con 0x7f3ef015a0b0 2023-10-11T06:13:37.316 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'data_new' removed 2023-10-11T06:13:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3edc06a830 msgr2=0x7f3edc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3edc06a830 0x7f3edc06cce0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0006fd0 tx=0x7f3ee0008040 comp rx=0 tx=0).stop 2023-10-11T06:13:37.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef015a0b0 msgr2=0x7f3ef01f2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:37.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef015a0b0 0x7f3ef01f2e20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8037040 tx=0x7f3ee8004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:37.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 shutdown_connections 2023-10-11T06:13:37.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3edc06a830 0x7f3edc06cce0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef015a0b0 0x7f3ef01f2e20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef00ced30 0x7f3ef014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 --2- 172.21.15.84:0/3565886477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ef00c8d00 0x7f3ef00b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.321+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 >> 172.21.15.84:0/3565886477 conn(0x7f3ef0069c70 msgr2=0x7f3ef00c3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:37.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.322+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 shutdown_connections 2023-10-11T06:13:37.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.322+0000 7f3ef62f7640 1 -- 172.21.15.84:0/3565886477 wait complete. 2023-10-11T06:13:37.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-11T06:13:37.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.555+0000 7fd528ac4640 1 Processor -- start 2023-10-11T06:13:37.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.555+0000 7fd528ac4640 1 -- start start 2023-10-11T06:13:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.556+0000 7fd528ac4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5240adb90 0x7fd5240caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.556+0000 7fd528ac4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd5240cd300 0x7fd5240cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.556+0000 7fd528ac4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240cb960 con 0x7fd5240ce0d0 2023-10-11T06:13:37.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.556+0000 7fd528ac4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52415a340 con 0x7fd5240adb90 2023-10-11T06:13:37.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.556+0000 7fd528ac4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5240ce4a0 con 0x7fd5240cd300 2023-10-11T06:13:37.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd51bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5240adb90 0x7fd5240caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:37.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd51bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5240adb90 0x7fd5240caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34110/0 (socket says 172.21.15.84:34110) 2023-10-11T06:13:37.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd51bfff640 1 -- 172.21.15.84:0/4117756006 learned_addr learned my addr 172.21.15.84:0/4117756006 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:37.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd522d9d640 1 --2- 172.21.15.84:0/4117756006 >> v2:172.21.15.99:3300/0 conn(0x7fd5240cd300 0x7fd5240cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:37.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd51bfff640 1 -- 172.21.15.84:0/4117756006 >> v2:172.21.15.99:3300/0 conn(0x7fd5240cd300 msgr2=0x7fd5240cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:37.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd51bfff640 1 --2- 172.21.15.84:0/4117756006 >> v2:172.21.15.99:3300/0 conn(0x7fd5240cd300 0x7fd5240cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd51bfff640 1 -- 172.21.15.84:0/4117756006 >> v1:172.21.15.99:6789/0 conn(0x7fd5240ce0d0 legacy=0x7fd5240c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd51bfff640 1 -- 172.21.15.84:0/4117756006 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240ce610 con 0x7fd5240adb90 2023-10-11T06:13:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.557+0000 7fd522d9d640 1 --2- 172.21.15.84:0/4117756006 >> v2:172.21.15.99:3300/0 conn(0x7fd5240cd300 0x7fd5240cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.558+0000 7fd51bfff640 1 --2- 172.21.15.84:0/4117756006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240adb90 0x7fd5240caee0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd52406fd30 tx=0x7fd50c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1afeb2ba25a570 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.559+0000 7fd521d9b640 1 -- 172.21.15.84:0/4117756006 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c031890 con 0x7fd5240adb90 2023-10-11T06:13:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.559+0000 7fd521d9b640 1 -- 172.21.15.84:0/4117756006 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c0319f0 con 0x7fd5240adb90 2023-10-11T06:13:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.559+0000 7fd521d9b640 1 -- 172.21.15.84:0/4117756006 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c031d80 con 0x7fd5240adb90 2023-10-11T06:13:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.559+0000 7fd528ac4640 1 -- 172.21.15.84:0/4117756006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240adb90 msgr2=0x7fd5240caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.559+0000 7fd528ac4640 1 --2- 172.21.15.84:0/4117756006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240adb90 0x7fd5240caee0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd52406fd30 tx=0x7fd50c02b930 comp rx=0 tx=0).stop 2023-10-11T06:13:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.560+0000 7fd528ac4640 1 -- 172.21.15.84:0/4117756006 shutdown_connections 2023-10-11T06:13:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.560+0000 7fd528ac4640 1 --2- 172.21.15.84:0/4117756006 >> v2:172.21.15.99:3300/0 conn(0x7fd5240cd300 0x7fd5240cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.560+0000 7fd528ac4640 1 --2- 172.21.15.84:0/4117756006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240adb90 0x7fd5240caee0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.560+0000 7fd528ac4640 1 -- 172.21.15.84:0/4117756006 >> 172.21.15.84:0/4117756006 conn(0x7fd524069c70 msgr2=0x7fd52406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.560+0000 7fd528ac4640 1 -- 172.21.15.84:0/4117756006 shutdown_connections 2023-10-11T06:13:37.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.560+0000 7fd528ac4640 1 -- 172.21.15.84:0/4117756006 wait complete. 2023-10-11T06:13:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.561+0000 7fd528ac4640 1 Processor -- start 2023-10-11T06:13:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.562+0000 7fd528ac4640 1 -- start start 2023-10-11T06:13:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.562+0000 7fd528ac4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5240adb90 0x7fd5240b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.562+0000 7fd528ac4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5240cd300 0x7fd5240ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.562+0000 7fd528ac4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240ce0d0 0x7fd524146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.562+0000 7fd528ac4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240af480 con 0x7fd5240ce0d0 2023-10-11T06:13:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.562+0000 7fd528ac4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd524163fe0 con 0x7fd5240cd300 2023-10-11T06:13:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.562+0000 7fd528ac4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd524164840 con 0x7fd5240adb90 2023-10-11T06:13:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240ce0d0 0x7fd524146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240ce0d0 0x7fd524146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34114/0 (socket says 172.21.15.84:34114) 2023-10-11T06:13:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 -- 172.21.15.84:0/2888206425 learned_addr learned my addr 172.21.15.84:0/2888206425 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd52259c640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5240adb90 0x7fd5240b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 -- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5240adb90 msgr2=0x7fd5240b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5240adb90 0x7fd5240b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd51bfff640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5240cd300 0x7fd5240ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 -- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5240cd300 msgr2=0x7fd5240ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5240cd300 0x7fd5240ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.563+0000 7fd522d9d640 1 -- 172.21.15.84:0/2888206425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0094d0 con 0x7fd5240ce0d0 2023-10-11T06:13:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.564+0000 7fd522d9d640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240ce0d0 0x7fd524146540 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00cef0 tx=0x7fd51c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.564+0000 7fd51affd640 1 -- 172.21.15.84:0/2888206425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00d840 con 0x7fd5240ce0d0 2023-10-11T06:13:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.564+0000 7fd51affd640 1 -- 172.21.15.84:0/2888206425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c00d9c0 con 0x7fd5240ce0d0 2023-10-11T06:13:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.564+0000 7fd51affd640 1 -- 172.21.15.84:0/2888206425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00dd70 con 0x7fd5240ce0d0 2023-10-11T06:13:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.564+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5240aeed0 con 0x7fd5240ce0d0 2023-10-11T06:13:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.564+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5241641f0 con 0x7fd5240ce0d0 2023-10-11T06:13:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.565+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd524073ed0 con 0x7fd5240ce0d0 2023-10-11T06:13:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.566+0000 7fd51affd640 1 -- 172.21.15.84:0/2888206425 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd51c01e3b0 con 0x7fd5240ce0d0 2023-10-11T06:13:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.567+0000 7fd51affd640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd50406a830 0x7fd50406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:37.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.567+0000 7fd51affd640 1 -- 172.21.15.84:0/2888206425 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd51c089820 con 0x7fd5240ce0d0 2023-10-11T06:13:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.567+0000 7fd52259c640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd50406a830 0x7fd50406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.568+0000 7fd52259c640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd50406a830 0x7fd50406cce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd5241f5660 tx=0x7fd514009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.572+0000 7fd51affd640 1 -- 172.21.15.84:0/2888206425 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd51c054de0 con 0x7fd5240ce0d0 2023-10-11T06:13:37.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.768+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd524164980 con 0x7fd5240ce0d0 2023-10-11T06:13:37.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.777+0000 7fd51affd640 1 -- 172.21.15.84:0/2888206425 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd51c058400 con 0x7fd5240ce0d0 2023-10-11T06:13:37.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd50406a830 msgr2=0x7fd50406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:37.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd50406a830 0x7fd50406cce0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd5241f5660 tx=0x7fd514009040 comp rx=0 tx=0).stop 2023-10-11T06:13:37.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240ce0d0 msgr2=0x7fd524146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:37.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240ce0d0 0x7fd524146540 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00cef0 tx=0x7fd51c00a710 comp rx=0 tx=0).stop 2023-10-11T06:13:37.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 shutdown_connections 2023-10-11T06:13:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd50406a830 0x7fd50406cce0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5240ce0d0 0x7fd524146540 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5240cd300 0x7fd5240ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 --2- 172.21.15.84:0/2888206425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd5240adb90 0x7fd5240b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:37.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.779+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 >> 172.21.15.84:0/2888206425 conn(0x7fd524069c70 msgr2=0x7fd524070910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.780+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 shutdown_connections 2023-10-11T06:13:37.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:37.780+0000 7fd528ac4640 1 -- 172.21.15.84:0/2888206425 wait complete. 2023-10-11T06:13:37.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-10-11T06:13:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.012+0000 7fd30d79f640 1 Processor -- start 2023-10-11T06:13:38.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.013+0000 7fd30d79f640 1 -- start start 2023-10-11T06:13:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.014+0000 7fd30d79f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 0x7fd3080af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.014+0000 7fd30d79f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd3080d25e0 0x7fd3080b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.014+0000 7fd30d79f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30806cd10 con 0x7fd30815c290 2023-10-11T06:13:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.014+0000 7fd30d79f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3080d2a20 con 0x7fd3080d25e0 2023-10-11T06:13:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.014+0000 7fd30d79f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080afb60 con 0x7fd3080d0c20 2023-10-11T06:13:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.014+0000 7fd306ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd30815c290 0x7fd3080d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54444/0 (socket says 172.21.15.84:54444) 2023-10-11T06:13:38.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.014+0000 7fd306ffd640 1 -- 172.21.15.84:0/3433597513 learned_addr learned my addr 172.21.15.84:0/3433597513 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210539320 0 0) 0x7fd30806cd10 con 0x7fd30815c290 2023-10-11T06:13:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f4003520 con 0x7fd30815c290 2023-10-11T06:13:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd3067fc640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 0x7fd3080af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd3077fe640 1 --2- 172.21.15.84:0/3433597513 >> v2:172.21.15.99:3300/0 conn(0x7fd3080d25e0 0x7fd3080b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2446791283 0 0) 0x7fd2f4003520 con 0x7fd30815c290 2023-10-11T06:13:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 msgr2=0x7fd3080af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 0x7fd3080af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 >> v2:172.21.15.99:3300/0 conn(0x7fd3080d25e0 msgr2=0x7fd3080b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 --2- 172.21.15.84:0/3433597513 >> v2:172.21.15.99:3300/0 conn(0x7fd3080d25e0 0x7fd3080b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3080afe40 con 0x7fd30815c290 2023-10-11T06:13:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fd2f8001670 con 0x7fd30815c290 2023-10-11T06:13:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 >> v1:172.21.15.84:6789/0 conn(0x7fd30815c290 legacy=0x7fd3080d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2f4003970 0x7fd2f4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 0x7fd2f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4004d00 0x7fd2f4005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080d2a20 con 0x7fd2f4003970 2023-10-11T06:13:38.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3080afb60 con 0x7fd2f4004d00 2023-10-11T06:13:38.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f400bae0 con 0x7fd3080d0c20 2023-10-11T06:13:38.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.015+0000 7fd306ffd640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2f4003970 0x7fd2f4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.016+0000 7fd306ffd640 1 -- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 msgr2=0x7fd2f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:38.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.016+0000 7fd3067fc640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 0x7fd2f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.016+0000 7fd306ffd640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3080d0c20 0x7fd2f40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.016+0000 7fd306ffd640 1 -- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4004d00 msgr2=0x7fd2f4005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:38.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.016+0000 7fd306ffd640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4004d00 0x7fd2f4005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.016+0000 7fd306ffd640 1 -- 172.21.15.84:0/3433597513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4003520 con 0x7fd2f4003970 2023-10-11T06:13:38.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.017+0000 7fd306ffd640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2f4003970 0x7fd2f4003d80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c210 tx=0x7fd2f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effb7acd0e8c2e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:38.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.017+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f802cae0 con 0x7fd2f4003970 2023-10-11T06:13:38.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.017+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f802cc40 con 0x7fd2f4003970 2023-10-11T06:13:38.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.017+0000 7fd305ffb640 1 -- 172.21.15.84:0/3433597513 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80058d0 con 0x7fd2f4003970 2023-10-11T06:13:38.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.018+0000 7fd30d79f640 1 -- 172.21.15.84:0/3433597513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2f4003970 msgr2=0x7fd2f4003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.018+0000 7fd30d79f640 1 --2- 172.21.15.84:0/3433597513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2f4003970 0x7fd2f4003d80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c210 tx=0x7fd2f40057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.018+0000 7fd3077fe640 1 -- 172.21.15.84:0/3433597513 reap_dead start 2023-10-11T06:13:38.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.018+0000 7fd30d79f640 1 -- 172.21.15.84:0/3433597513 shutdown_connections 2023-10-11T06:13:38.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.018+0000 7fd30d79f640 1 -- 172.21.15.84:0/3433597513 >> 172.21.15.84:0/3433597513 conn(0x7fd308071cc0 msgr2=0x7fd30809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:38.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.018+0000 7fd30d79f640 1 -- 172.21.15.84:0/3433597513 shutdown_connections 2023-10-11T06:13:38.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.018+0000 7fd30d79f640 1 -- 172.21.15.84:0/3433597513 wait complete. 2023-10-11T06:13:38.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.020+0000 7fd30d79f640 1 Processor -- start 2023-10-11T06:13:38.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.020+0000 7fd30d79f640 1 -- start start 2023-10-11T06:13:38.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd30d79f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4003970 0x7fd30815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd30d79f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3081f0d30 0x7fd308148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd30d79f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd308148cc0 0x7fd30809e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd30d79f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30809c910 con 0x7fd3081f0d30 2023-10-11T06:13:38.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd30d79f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30809cbf0 con 0x7fd2f4003970 2023-10-11T06:13:38.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd30d79f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30809ced0 con 0x7fd308148cc0 2023-10-11T06:13:38.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3081f0d30 0x7fd308148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3081f0d30 0x7fd308148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34124/0 (socket says 172.21.15.84:34124) 2023-10-11T06:13:38.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 -- 172.21.15.84:0/4090498354 learned_addr learned my addr 172.21.15.84:0/4090498354 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:38.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 -- 172.21.15.84:0/4090498354 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd308148cc0 msgr2=0x7fd30809e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd308148cc0 0x7fd30809e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 -- 172.21.15.84:0/4090498354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4003970 msgr2=0x7fd30815aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd306ffd640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4003970 0x7fd30815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4003970 0x7fd30815aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.021+0000 7fd3067fc640 1 -- 172.21.15.84:0/4090498354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30809d1b0 con 0x7fd3081f0d30 2023-10-11T06:13:38.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.022+0000 7fd3067fc640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3081f0d30 0x7fd308148780 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e5d0 tx=0x7fd2f000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:38.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.022+0000 7fd2ff7fe640 1 -- 172.21.15.84:0/4090498354 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0013610 con 0x7fd3081f0d30 2023-10-11T06:13:38.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.022+0000 7fd2ff7fe640 1 -- 172.21.15.84:0/4090498354 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0013770 con 0x7fd3081f0d30 2023-10-11T06:13:38.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.022+0000 7fd2ff7fe640 1 -- 172.21.15.84:0/4090498354 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0013b00 con 0x7fd3081f0d30 2023-10-11T06:13:38.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.022+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30809d490 con 0x7fd3081f0d30 2023-10-11T06:13:38.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.022+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30809c5e0 con 0x7fd3081f0d30 2023-10-11T06:13:38.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.023+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30809cd30 con 0x7fd3081f0d30 2023-10-11T06:13:38.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.024+0000 7fd2ff7fe640 1 -- 172.21.15.84:0/4090498354 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd2f0013c60 con 0x7fd3081f0d30 2023-10-11T06:13:38.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.025+0000 7fd2ff7fe640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2e806a830 0x7fd2e806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.025+0000 7fd2ff7fe640 1 -- 172.21.15.84:0/4090498354 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd2f0088970 con 0x7fd3081f0d30 2023-10-11T06:13:38.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.025+0000 7fd306ffd640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2e806a830 0x7fd2e806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.026+0000 7fd306ffd640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2e806a830 0x7fd2e806cce0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8034000 tx=0x7fd2f8007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:38.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.029+0000 7fd2ff7fe640 1 -- 172.21.15.84:0/4090498354 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd30809cd30 con 0x7fd3081f0d30 2023-10-11T06:13:38.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.272+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7fd30809d010 con 0x7fd3081f0d30 2023-10-11T06:13:38.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.313+0000 7fd2ff7fe640 1 -- 172.21.15.84:0/4090498354 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v156) v1 ==== 147+0+0 (secure 0 0 0) 0x7fd30809d010 con 0x7fd3081f0d30 2023-10-11T06:13:38.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.315+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2e806a830 msgr2=0x7fd2e806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.315+0000 7fd30d79f640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2e806a830 0x7fd2e806cce0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8034000 tx=0x7fd2f8007000 comp rx=0 tx=0).stop 2023-10-11T06:13:38.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.315+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3081f0d30 msgr2=0x7fd308148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.315+0000 7fd30d79f640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3081f0d30 0x7fd308148780 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e5d0 tx=0x7fd2f000ea90 comp rx=0 tx=0).stop 2023-10-11T06:13:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 shutdown_connections 2023-10-11T06:13:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd2e806a830 0x7fd2e806cce0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd308148cc0 0x7fd30809e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3081f0d30 0x7fd308148780 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 --2- 172.21.15.84:0/4090498354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f4003970 0x7fd30815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 >> 172.21.15.84:0/4090498354 conn(0x7fd308071cc0 msgr2=0x7fd308072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:38.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 shutdown_connections 2023-10-11T06:13:38.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.316+0000 7fd30d79f640 1 -- 172.21.15.84:0/4090498354 wait complete. 2023-10-11T06:13:38.335 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2023-10-11T06:13:38.336 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-11T06:13:38.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-10-11T06:13:38.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.535+0000 7fbbcd637640 1 Processor -- start 2023-10-11T06:13:38.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.535+0000 7fbbcd637640 1 -- start start 2023-10-11T06:13:38.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.536+0000 7fbbcd637640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbc80ce0d0 0x7fbbc80c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbcd637640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc81593f0 con 0x7fbbc80adb90 2023-10-11T06:13:38.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbcd637640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc81596c0 con 0x7fbbc80cd300 2023-10-11T06:13:38.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbcd637640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc815a340 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbc6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbc80ce0d0 0x7fbbc80c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbc6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbc80ce0d0 0x7fbbc80c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34130/0 (socket says 172.21.15.84:34130) 2023-10-11T06:13:38.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbc6ffd640 1 -- 172.21.15.84:0/683010152 learned_addr learned my addr 172.21.15.84:0/683010152 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:38.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbbffff640 1 --1- 172.21.15.84:0/683010152 >> v1:172.21.15.84:6789/0 conn(0x7fbbc80adb90 0x7fbbc80caee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54448/0 (socket says 172.21.15.84:54448) 2023-10-11T06:13:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbc6ffd640 1 -- 172.21.15.84:0/683010152 >> v1:172.21.15.99:6789/0 conn(0x7fbbc80cd300 legacy=0x7fbbc815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbc6ffd640 1 -- 172.21.15.84:0/683010152 >> v1:172.21.15.84:6789/0 conn(0x7fbbc80adb90 legacy=0x7fbbc80caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbc6ffd640 1 -- 172.21.15.84:0/683010152 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc81599a0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.537+0000 7fbbc67fc640 1 -- 172.21.15.84:0/683010152 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257124809 0 0) 0x7fbbc81593f0 con 0x7fbbc80adb90 2023-10-11T06:13:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.538+0000 7fbbc6ffd640 1 --2- 172.21.15.84:0/683010152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc80c8cc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009850 tx=0x7fbbb802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c188bf0aec8e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbc67fc640 1 -- 172.21.15.84:0/683010152 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8004230 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbc67fc640 1 -- 172.21.15.84:0/683010152 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8004390 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbc67fc640 1 -- 172.21.15.84:0/683010152 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8030840 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbcd637640 1 -- 172.21.15.84:0/683010152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 msgr2=0x7fbbc80c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbcd637640 1 --2- 172.21.15.84:0/683010152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc80c8cc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009850 tx=0x7fbbb802b980 comp rx=0 tx=0).stop 2023-10-11T06:13:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbcd637640 1 -- 172.21.15.84:0/683010152 shutdown_connections 2023-10-11T06:13:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbcd637640 1 --2- 172.21.15.84:0/683010152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc80c8cc0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbcd637640 1 -- 172.21.15.84:0/683010152 >> 172.21.15.84:0/683010152 conn(0x7fbbc8069c70 msgr2=0x7fbbc806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbcd637640 1 -- 172.21.15.84:0/683010152 shutdown_connections 2023-10-11T06:13:38.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.539+0000 7fbbcd637640 1 -- 172.21.15.84:0/683010152 wait complete. 2023-10-11T06:13:38.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.541+0000 7fbbcd637640 1 Processor -- start 2023-10-11T06:13:38.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.541+0000 7fbbcd637640 1 -- start start 2023-10-11T06:13:38.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbcd637640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbc80adb90 0x7fbbc80c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbcd637640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbc80cd300 0x7fbbc80c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbcd637640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc8151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbcd637640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc814ef70 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbcd637640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80c7c00 con 0x7fbbc80cd300 2023-10-11T06:13:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbcd637640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80c7ee0 con 0x7fbbc80adb90 2023-10-11T06:13:38.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbc77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc8151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbc77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc8151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34146/0 (socket says 172.21.15.84:34146) 2023-10-11T06:13:38.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.542+0000 7fbbc77fe640 1 -- 172.21.15.84:0/2547800391 learned_addr learned my addr 172.21.15.84:0/2547800391 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:38.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbc77fe640 1 -- 172.21.15.84:0/2547800391 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbc80adb90 msgr2=0x7fbbc80c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbc77fe640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbc80adb90 0x7fbbc80c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbc77fe640 1 -- 172.21.15.84:0/2547800391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbc80cd300 msgr2=0x7fbbc80c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:38.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbbffff640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbc80cd300 0x7fbbc80c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbc77fe640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbc80cd300 0x7fbbc80c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:38.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbc77fe640 1 -- 172.21.15.84:0/2547800391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8009460 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbc77fe640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc8151c70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b970 tx=0x7fbbc000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:38.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbbf7fe640 1 -- 172.21.15.84:0/2547800391 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000cb40 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbbf7fe640 1 -- 172.21.15.84:0/2547800391 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc000cca0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbbf7fe640 1 -- 172.21.15.84:0/2547800391 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0014680 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc80c81c0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.543+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc80c78f0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.544+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc80c86e0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.545+0000 7fbbbf7fe640 1 -- 172.21.15.84:0/2547800391 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbbc001d3b0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.546+0000 7fbbbf7fe640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbbac06a830 0x7fbbac06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:38.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.546+0000 7fbbbf7fe640 1 -- 172.21.15.84:0/2547800391 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbbc0088af0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.546+0000 7fbbc6ffd640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbbac06a830 0x7fbbac06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:38.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.547+0000 7fbbc6ffd640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbbac06a830 0x7fbbac06cce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8035000 tx=0x7fbbb8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:38.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.551+0000 7fbbbf7fe640 1 -- 172.21.15.84:0/2547800391 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fbbc80c86e0 con 0x7fbbc80ce0d0 2023-10-11T06:13:38.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:38.787+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fbbc80c86e0 con 0x7fbbc80ce0d0 2023-10-11T06:13:39.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.323+0000 7fbbbf7fe640 1 -- 172.21.15.84:0/2547800391 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v157) v1 ==== 137+0+0 (secure 0 0 0) 0x7fbbc80c86e0 con 0x7fbbc80ce0d0 2023-10-11T06:13:39.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbbac06a830 msgr2=0x7fbbac06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbbac06a830 0x7fbbac06cce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8035000 tx=0x7fbbb8030000 comp rx=0 tx=0).stop 2023-10-11T06:13:39.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 msgr2=0x7fbbc8151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc8151c70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b970 tx=0x7fbbc000be30 comp rx=0 tx=0).stop 2023-10-11T06:13:39.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 shutdown_connections 2023-10-11T06:13:39.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbbac06a830 0x7fbbac06cce0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbc80ce0d0 0x7fbbc8151c70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbc80cd300 0x7fbbc80c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 --2- 172.21.15.84:0/2547800391 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbc80adb90 0x7fbbc80c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.327+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 >> 172.21.15.84:0/2547800391 conn(0x7fbbc8069c70 msgr2=0x7fbbc80caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:39.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.328+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 shutdown_connections 2023-10-11T06:13:39.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.328+0000 7fbbcd637640 1 -- 172.21.15.84:0/2547800391 wait complete. 2023-10-11T06:13:39.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-11T06:13:39.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.567+0000 7f214765f640 1 Processor -- start 2023-10-11T06:13:39.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.568+0000 7f214765f640 1 -- start start 2023-10-11T06:13:39.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.569+0000 7f214765f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21400d2ac0 0x7f21400d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:39.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.569+0000 7f214765f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f21400d0940 0x7f21400d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:39.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.569+0000 7f214765f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21400d3450 con 0x7f214015f7e0 2023-10-11T06:13:39.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.569+0000 7f214765f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21400cf220 con 0x7f21400d0940 2023-10-11T06:13:39.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.569+0000 7f214765f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21400cf500 con 0x7f21400d2ac0 2023-10-11T06:13:39.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2144bfa640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21400d2ac0 0x7f21400d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:39.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2144bfa640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21400d2ac0 0x7f21400d2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:46726/0 (socket says 172.21.15.84:46726) 2023-10-11T06:13:39.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2145bfc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f21400d0940 0x7f21400d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:39.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2144bfa640 1 -- 172.21.15.84:0/1471990299 learned_addr learned my addr 172.21.15.84:0/1471990299 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:39.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2144bfa640 1 -- 172.21.15.84:0/1471990299 >> v2:172.21.15.99:3300/0 conn(0x7f21400d0940 msgr2=0x7f21400d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:39.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2144bfa640 1 --2- 172.21.15.84:0/1471990299 >> v2:172.21.15.99:3300/0 conn(0x7f21400d0940 0x7f21400d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2144bfa640 1 -- 172.21.15.84:0/1471990299 >> v1:172.21.15.99:6789/0 conn(0x7f214015f7e0 legacy=0x7f21400cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.570+0000 7f2144bfa640 1 -- 172.21.15.84:0/1471990299 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400cf7e0 con 0x7f21400d2ac0 2023-10-11T06:13:39.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.572+0000 7f2144bfa640 1 --2- 172.21.15.84:0/1471990299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21400d2ac0 0x7f21400d2f10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2134009870 tx=0x7f213402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e464ffcbb41f6d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:39.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.572+0000 7f213ffff640 1 -- 172.21.15.84:0/1471990299 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134030840 con 0x7f21400d2ac0 2023-10-11T06:13:39.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.572+0000 7f213ffff640 1 -- 172.21.15.84:0/1471990299 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21340309a0 con 0x7f21400d2ac0 2023-10-11T06:13:39.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.572+0000 7f213ffff640 1 -- 172.21.15.84:0/1471990299 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134030d30 con 0x7f21400d2ac0 2023-10-11T06:13:39.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 -- 172.21.15.84:0/1471990299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21400d2ac0 msgr2=0x7f21400d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 --2- 172.21.15.84:0/1471990299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21400d2ac0 0x7f21400d2f10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2134009870 tx=0x7f213402b980 comp rx=0 tx=0).stop 2023-10-11T06:13:39.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 -- 172.21.15.84:0/1471990299 shutdown_connections 2023-10-11T06:13:39.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 --2- 172.21.15.84:0/1471990299 >> v2:172.21.15.99:3300/0 conn(0x7f21400d0940 0x7f21400d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 --2- 172.21.15.84:0/1471990299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21400d2ac0 0x7f21400d2f10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 -- 172.21.15.84:0/1471990299 >> 172.21.15.84:0/1471990299 conn(0x7f21400713a0 msgr2=0x7f21400717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:39.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 -- 172.21.15.84:0/1471990299 shutdown_connections 2023-10-11T06:13:39.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.573+0000 7f214765f640 1 -- 172.21.15.84:0/1471990299 wait complete. 2023-10-11T06:13:39.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.575+0000 7f214765f640 1 Processor -- start 2023-10-11T06:13:39.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.575+0000 7f214765f640 1 -- start start 2023-10-11T06:13:39.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f214765f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21400d0940 0x7f21401f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:39.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f214765f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21400d2ac0 0x7f21400c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:39.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f214765f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f214015f7e0 0x7f21400b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:39.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f214765f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214014c060 con 0x7f21400d2ac0 2023-10-11T06:13:39.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f214765f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21400b5960 con 0x7f21400d0940 2023-10-11T06:13:39.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f214765f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21401f59d0 con 0x7f214015f7e0 2023-10-11T06:13:39.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f2144bfa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21400d2ac0 0x7f21400c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:39.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f2144bfa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21400d2ac0 0x7f21400c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52666/0 (socket says 172.21.15.84:52666) 2023-10-11T06:13:39.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f2144bfa640 1 -- 172.21.15.84:0/2512059535 learned_addr learned my addr 172.21.15.84:0/2512059535 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:39.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f2145bfc640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f214015f7e0 0x7f21400b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:39.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.576+0000 7f21453fb640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21400d0940 0x7f21401f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:39.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f2144bfa640 1 -- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f214015f7e0 msgr2=0x7f21400b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f2144bfa640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f214015f7e0 0x7f21400b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f2144bfa640 1 -- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21400d0940 msgr2=0x7f21401f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f2144bfa640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21400d0940 0x7f21401f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f2144bfa640 1 -- 172.21.15.84:0/2512059535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340094d0 con 0x7f21400d2ac0 2023-10-11T06:13:39.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f2144bfa640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21400d2ac0 0x7f21400c8aa0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2134000c00 tx=0x7f2134004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:39.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f213dffb640 1 -- 172.21.15.84:0/2512059535 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134039c90 con 0x7f21400d2ac0 2023-10-11T06:13:39.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f213dffb640 1 -- 172.21.15.84:0/2512059535 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2134039df0 con 0x7f21400d2ac0 2023-10-11T06:13:39.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21401f5cb0 con 0x7f21400d2ac0 2023-10-11T06:13:39.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.577+0000 7f213dffb640 1 -- 172.21.15.84:0/2512059535 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134043640 con 0x7f21400d2ac0 2023-10-11T06:13:39.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.578+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21400b5c00 con 0x7f21400d2ac0 2023-10-11T06:13:39.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.579+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21400b5df0 con 0x7f21400d2ac0 2023-10-11T06:13:39.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.580+0000 7f213dffb640 1 -- 172.21.15.84:0/2512059535 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f21340437a0 con 0x7f21400d2ac0 2023-10-11T06:13:39.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.581+0000 7f213dffb640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f212c06a830 0x7f212c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:39.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.581+0000 7f213dffb640 1 -- 172.21.15.84:0/2512059535 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f213403e070 con 0x7f21400d2ac0 2023-10-11T06:13:39.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.581+0000 7f21453fb640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f212c06a830 0x7f212c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:39.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.582+0000 7f21453fb640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f212c06a830 0x7f212c06cce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f21300012e0 tx=0x7f2130009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:39.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.586+0000 7f213dffb640 1 -- 172.21.15.84:0/2512059535 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f21340772a0 con 0x7f21400d2ac0 2023-10-11T06:13:39.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.774+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2140073ed0 con 0x7f21400d2ac0 2023-10-11T06:13:39.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.784+0000 7f213dffb640 1 -- 172.21.15.84:0/2512059535 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f213407a8c0 con 0x7f21400d2ac0 2023-10-11T06:13:39.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f212c06a830 msgr2=0x7f212c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f212c06a830 0x7f212c06cce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f21300012e0 tx=0x7f2130009040 comp rx=0 tx=0).stop 2023-10-11T06:13:39.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21400d2ac0 msgr2=0x7f21400c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:39.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21400d2ac0 0x7f21400c8aa0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2134000c00 tx=0x7f2134004730 comp rx=0 tx=0).stop 2023-10-11T06:13:39.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 shutdown_connections 2023-10-11T06:13:39.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f212c06a830 0x7f212c06cce0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f214015f7e0 0x7f21400b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21400d2ac0 0x7f21400c8aa0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 --2- 172.21.15.84:0/2512059535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21400d0940 0x7f21401f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:39.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 >> 172.21.15.84:0/2512059535 conn(0x7f21400713a0 msgr2=0x7f2140159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:39.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 shutdown_connections 2023-10-11T06:13:39.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:39.786+0000 7f214765f640 1 -- 172.21.15.84:0/2512059535 wait complete. 2023-10-11T06:13:39.806 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T06:13:39.806 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:13:39.922 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:13:40.045 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T06:13:40.046 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:40.046 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:40.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053598,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T06:13:40.107 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:13:40.107 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:40.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:13:40.168 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:40.168 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:40.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053598,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T06:13:40.229 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:13:40.229 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:40.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:13:40.287 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-11T06:13:40.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:13:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.510+0000 7faa4bb0a640 1 Processor -- start 2023-10-11T06:13:40.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.510+0000 7faa4bb0a640 1 -- start start 2023-10-11T06:13:40.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.511+0000 7faa4bb0a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faa4415d4f0 0x7faa440cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.511+0000 7faa4bb0a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 0x7faa440ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.511+0000 7faa4bb0a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa44071a90 con 0x7faa440ca300 2023-10-11T06:13:40.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.511+0000 7faa4bb0a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa44072310 con 0x7faa4415d4f0 2023-10-11T06:13:40.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.511+0000 7faa4bb0a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa44070280 con 0x7faa440c9360 2023-10-11T06:13:40.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa490a5640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7faa440ca300 0x7faa440c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35474/0 (socket says 172.21.15.84:35474) 2023-10-11T06:13:40.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa490a5640 1 -- 172.21.15.84:0/2870782415 learned_addr learned my addr 172.21.15.84:0/2870782415 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:40.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640876323 0 0) 0x7faa44071a90 con 0x7faa440ca300 2023-10-11T06:13:40.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa30003520 con 0x7faa440ca300 2023-10-11T06:13:40.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 0x7faa440ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa498a6640 1 --2- 172.21.15.84:0/2870782415 >> v2:172.21.15.99:3300/0 conn(0x7faa4415d4f0 0x7faa440cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1971054202 0 0) 0x7faa30003520 con 0x7faa440ca300 2023-10-11T06:13:40.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 msgr2=0x7faa440ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:40.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 0x7faa440ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 >> v2:172.21.15.99:3300/0 conn(0x7faa4415d4f0 msgr2=0x7faa440cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:40.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 --2- 172.21.15.84:0/2870782415 >> v2:172.21.15.99:3300/0 conn(0x7faa4415d4f0 0x7faa440cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44071cc0 con 0x7faa440ca300 2023-10-11T06:13:40.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7faa34001690 con 0x7faa440ca300 2023-10-11T06:13:40.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 >> v1:172.21.15.84:6789/0 conn(0x7faa440ca300 legacy=0x7faa440c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:40.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.512+0000 7faa488a4640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 0x7faa30003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa488a4640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30003ef0 0x7faa300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa488a4640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa30004d00 0x7faa30005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44072310 con 0x7faa30004d00 2023-10-11T06:13:40.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44070280 con 0x7faa30003ef0 2023-10-11T06:13:40.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3000b980 con 0x7faa440c9360 2023-10-11T06:13:40.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa30004d00 0x7faa30005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa490a5640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30003ef0 0x7faa300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa498a6640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 0x7faa30003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 msgr2=0x7faa30003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:40.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440c9360 0x7faa30003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30003ef0 msgr2=0x7faa300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:40.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30003ef0 0x7faa300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2870782415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa30003520 con 0x7faa30004d00 2023-10-11T06:13:40.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.513+0000 7faa490a5640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30003ef0 0x7faa300047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:40.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.514+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa30004d00 0x7faa30005170 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009fc0 tx=0x7faa300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e100201f9a7d2089 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:40.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c02cc20 con 0x7faa30004d00 2023-10-11T06:13:40.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c00f040 con 0x7faa30004d00 2023-10-11T06:13:40.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa488a4640 1 -- 172.21.15.84:0/2870782415 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c034440 con 0x7faa30004d00 2023-10-11T06:13:40.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2870782415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa30004d00 msgr2=0x7faa30005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:40.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa4bb0a640 1 --2- 172.21.15.84:0/2870782415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa30004d00 0x7faa30005170 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009fc0 tx=0x7faa300057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2870782415 reap_dead start 2023-10-11T06:13:40.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2870782415 shutdown_connections 2023-10-11T06:13:40.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2870782415 >> 172.21.15.84:0/2870782415 conn(0x7faa44069c70 msgr2=0x7faa4406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:40.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.515+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2870782415 shutdown_connections 2023-10-11T06:13:40.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.516+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2870782415 wait complete. 2023-10-11T06:13:40.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.517+0000 7faa4bb0a640 1 Processor -- start 2023-10-11T06:13:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.518+0000 7faa4bb0a640 1 -- start start 2023-10-11T06:13:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4bb0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30004d00 0x7faa441580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4bb0a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b3420 0x7faa441f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4bb0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa441ed5a0 0x7faa4415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4bb0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44158630 con 0x7faa441ed5a0 2023-10-11T06:13:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4bb0a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441f1050 con 0x7faa30004d00 2023-10-11T06:13:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4bb0a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa441f12f0 con 0x7faa440b3420 2023-10-11T06:13:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa441ed5a0 0x7faa4415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa441ed5a0 0x7faa4415da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52692/0 (socket says 172.21.15.84:52692) 2023-10-11T06:13:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2944403294 learned_addr learned my addr 172.21.15.84:0/2944403294 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:40.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa490a5640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b3420 0x7faa441f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa498a6640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30004d00 0x7faa441580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b3420 msgr2=0x7faa441f0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b3420 0x7faa441f0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30004d00 msgr2=0x7faa441580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:40.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30004d00 0x7faa441580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa4a0a7640 1 -- 172.21.15.84:0/2944403294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa441f15d0 con 0x7faa441ed5a0 2023-10-11T06:13:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa490a5640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b3420 0x7faa441f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.519+0000 7faa498a6640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30004d00 0x7faa441580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.520+0000 7faa4a0a7640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa441ed5a0 0x7faa4415da40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7faa3c002230 tx=0x7faa3c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:40.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.520+0000 7faa3a7fc640 1 -- 172.21.15.84:0/2944403294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c043070 con 0x7faa441ed5a0 2023-10-11T06:13:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.520+0000 7faa3a7fc640 1 -- 172.21.15.84:0/2944403294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c034ad0 con 0x7faa441ed5a0 2023-10-11T06:13:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.520+0000 7faa3a7fc640 1 -- 172.21.15.84:0/2944403294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c034e80 con 0x7faa441ed5a0 2023-10-11T06:13:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.520+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa441f18b0 con 0x7faa441ed5a0 2023-10-11T06:13:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.520+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa441596d0 con 0x7faa441ed5a0 2023-10-11T06:13:40.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.521+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa44073ed0 con 0x7faa441ed5a0 2023-10-11T06:13:40.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.522+0000 7faa3a7fc640 1 -- 172.21.15.84:0/2944403294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7faa3c03e440 con 0x7faa441ed5a0 2023-10-11T06:13:40.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.523+0000 7faa3a7fc640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7faa2806a820 0x7faa2806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.523+0000 7faa3a7fc640 1 -- 172.21.15.84:0/2944403294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7faa3c0aa860 con 0x7faa441ed5a0 2023-10-11T06:13:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.524+0000 7faa498a6640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7faa2806a820 0x7faa2806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.524+0000 7faa498a6640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7faa2806a820 0x7faa2806ccd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faa40009a50 tx=0x7faa40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:40.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.528+0000 7faa3a7fc640 1 -- 172.21.15.84:0/2944403294 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7faa3c075df0 con 0x7faa441ed5a0 2023-10-11T06:13:40.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.752+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa440b09b0 con 0x7faa441ed5a0 2023-10-11T06:13:40.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.753+0000 7faa3a7fc640 1 -- 172.21.15.84:0/2944403294 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3520 (secure 0 0 0) 0x7faa3c039dd0 con 0x7faa441ed5a0 2023-10-11T06:13:40.755 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:40.756 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/659642378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":659642378},{"type":"v1","addr":"172.21.15.99:6835","nonce":659642378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5325,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1799986655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1799986655},{"type":"v1","addr":"172.21.15.84:6835","nonce":1799986655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5328,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2890561635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2890561635},{"type":"v1","addr":"172.21.15.84:6837","nonce":2890561635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/44897153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":44897153},{"type":"v1","addr":"172.21.15.99:6833","nonce":44897153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2023-10-11T06:13:40.756 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2023-10-11T06:13:40.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7faa2806a820 msgr2=0x7faa2806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:40.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7faa2806a820 0x7faa2806ccd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faa40009a50 tx=0x7faa40008040 comp rx=0 tx=0).stop 2023-10-11T06:13:40.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa441ed5a0 msgr2=0x7faa4415da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:40.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa441ed5a0 0x7faa4415da40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7faa3c002230 tx=0x7faa3c009750 comp rx=0 tx=0).stop 2023-10-11T06:13:40.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 shutdown_connections 2023-10-11T06:13:40.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7faa2806a820 0x7faa2806ccd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa441ed5a0 0x7faa4415da40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b3420 0x7faa441f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.755+0000 7faa4bb0a640 1 --2- 172.21.15.84:0/2944403294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa30004d00 0x7faa441580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:40.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.756+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 >> 172.21.15.84:0/2944403294 conn(0x7faa44069c70 msgr2=0x7faa440cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:40.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.756+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 shutdown_connections 2023-10-11T06:13:40.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:40.756+0000 7faa4bb0a640 1 -- 172.21.15.84:0/2944403294 wait complete. 2023-10-11T06:13:40.775 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-11T06:13:40.775 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-11T06:13:40.776 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T06:13:40.776 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:13:40.777 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-11T06:13:40.777 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-11T06:13:40.777 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T06:13:40.778 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:13:40.778 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-11T06:13:40.779 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-11T06:13:40.779 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T06:13:40.780 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:13:40.780 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-11T06:13:40.781 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-11T06:13:40.781 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T06:13:40.782 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:13:40.792 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T06:13:40.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-11T06:13:40.795 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T06:13:40.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-11T06:13:40.801 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T06:13:40.801 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-11T06:13:40.807 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T06:13:40.808 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-11T06:13:40.844 INFO:tasks.ceph.mds.a:Started 2023-10-11T06:13:40.845 DEBUG:teuthology.parallel:result is None 2023-10-11T06:13:40.852 INFO:tasks.ceph.mds.c:Started 2023-10-11T06:13:40.852 DEBUG:teuthology.parallel:result is None 2023-10-11T06:13:40.856 INFO:tasks.ceph.mds.b:Started 2023-10-11T06:13:40.856 DEBUG:teuthology.parallel:result is None 2023-10-11T06:13:40.863 INFO:tasks.ceph.mds.d:Started 2023-10-11T06:13:40.864 DEBUG:teuthology.parallel:result is None 2023-10-11T06:13:40.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-11T06:13:41.007 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-10-11T06:13:41.008 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-11T06:13:41.018 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2023-10-11T06:13:41.024 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2023-10-11T06:13:41.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.065+0000 7f0fe0cb9640 1 Processor -- start 2023-10-11T06:13:41.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.065+0000 7f0fe0cb9640 1 -- start start 2023-10-11T06:13:41.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.066+0000 7f0fe0cb9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 0x7f0fdc160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.066+0000 7f0fe0cb9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fdc15fc20 con 0x7f0fdc0d0300 2023-10-11T06:13:41.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.066+0000 7f0fe0cb9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fdc1601e0 con 0x7f0fdc0cf750 2023-10-11T06:13:41.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.066+0000 7f0fe0cb9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc15c510 con 0x7f0fdc0d2d40 2023-10-11T06:13:41.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.066+0000 7f0fdbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0fdc0d0300 0x7f0fdc0ccd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35498/0 (socket says 172.21.15.84:35498) 2023-10-11T06:13:41.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.066+0000 7f0fdbfff640 1 -- 172.21.15.84:0/3899983254 learned_addr learned my addr 172.21.15.84:0/3899983254 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:41.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048422151 0 0) 0x7f0fdc15fc20 con 0x7f0fdc0d0300 2023-10-11T06:13:41.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fc4003520 con 0x7f0fdc0d0300 2023-10-11T06:13:41.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fdaffd640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 0x7f0fdc160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762471993 0 0) 0x7f0fdc1601e0 con 0x7f0fdc0cf750 2023-10-11T06:13:41.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fdc15fc20 con 0x7f0fdc0cf750 2023-10-11T06:13:41.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558247307 0 0) 0x7f0fc4003520 con 0x7f0fdc0d0300 2023-10-11T06:13:41.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 msgr2=0x7f0fdc160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:41.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 0x7f0fdc160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:41.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 >> v1:172.21.15.99:6789/0 conn(0x7f0fdc0cf750 legacy=0x7f0fdc0d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:41.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc0d0a30 con 0x7f0fdc0d0300 2023-10-11T06:13:41.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f0fd40019f0 con 0x7f0fdc0d0300 2023-10-11T06:13:41.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 >> v1:172.21.15.84:6789/0 conn(0x7f0fdc0d0300 legacy=0x7f0fdc0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:41.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc4003b30 0x7f0fc4003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 0x7f0fc40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fc4004d00 0x7f0fc4005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.067+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc15c510 con 0x7f0fc4004d00 2023-10-11T06:13:41.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc400bb10 con 0x7f0fc4003b30 2023-10-11T06:13:41.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc400bcd0 con 0x7f0fdc0d2d40 2023-10-11T06:13:41.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdbfff640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fc4004d00 0x7f0fc4005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdb7fe640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc4003b30 0x7f0fc4003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdaffd640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 0x7f0fc40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdbfff640 1 -- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 msgr2=0x7f0fc40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:41.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdbfff640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc0d2d40 0x7f0fc40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:41.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdbfff640 1 -- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc4003b30 msgr2=0x7f0fc4003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:41.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdbfff640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fc4003b30 0x7f0fc4003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:41.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.068+0000 7f0fdbfff640 1 -- 172.21.15.84:0/3899983254 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4003520 con 0x7f0fc4004d00 2023-10-11T06:13:41.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.069+0000 7f0fdbfff640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fc4004d00 0x7f0fc4005170 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400a710 tx=0x7f0fc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8af904fd9b4207 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:41.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.069+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4034630 con 0x7f0fc4004d00 2023-10-11T06:13:41.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd402f070 con 0x7f0fc4004d00 2023-10-11T06:13:41.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fda7fc640 1 -- 172.21.15.84:0/3899983254 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4034a10 con 0x7f0fc4004d00 2023-10-11T06:13:41.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/3899983254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fc4004d00 msgr2=0x7f0fc4005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:41.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fe0cb9640 1 --2- 172.21.15.84:0/3899983254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fc4004d00 0x7f0fc4005170 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400a710 tx=0x7f0fc40057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:41.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fdbfff640 1 -- 172.21.15.84:0/3899983254 reap_dead start 2023-10-11T06:13:41.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/3899983254 shutdown_connections 2023-10-11T06:13:41.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/3899983254 >> 172.21.15.84:0/3899983254 conn(0x7f0fdc071cc0 msgr2=0x7f0fdc09bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:41.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/3899983254 shutdown_connections 2023-10-11T06:13:41.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.070+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/3899983254 wait complete. 2023-10-11T06:13:41.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.072+0000 7f0fe0cb9640 1 Processor -- start 2023-10-11T06:13:41.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.072+0000 7f0fe0cb9640 1 -- start start 2023-10-11T06:13:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.073+0000 7f0fe0cb9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fc4004d00 0x7f0fdc0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.073+0000 7f0fe0cb9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fdc0c6260 0x7f0fdc153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.073+0000 7f0fe0cb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fdc06b160 0x7f0fdc06b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.073+0000 7f0fe0cb9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0b18f0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.073+0000 7f0fe0cb9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc06bb10 con 0x7f0fdc06b160 2023-10-11T06:13:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.073+0000 7f0fe0cb9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc153aa0 con 0x7f0fc4004d00 2023-10-11T06:13:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.073+0000 7f0fdaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fdc0c6260 0x7f0fdc153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fdc0c6260 0x7f0fdc153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52750/0 (socket says 172.21.15.84:52750) 2023-10-11T06:13:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 -- 172.21.15.84:0/4090240929 learned_addr learned my addr 172.21.15.84:0/4090240929 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:41.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdb7fe640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fc4004d00 0x7f0fdc0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdbfff640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fdc06b160 0x7f0fdc06b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 -- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fc4004d00 msgr2=0x7f0fdc0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fc4004d00 0x7f0fdc0b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 -- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fdc06b160 msgr2=0x7f0fdc06b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:41.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fdc06b160 0x7f0fdc06b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 -- 172.21.15.84:0/4090240929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc1f5db0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdbfff640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fdc06b160 0x7f0fdc06b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fdaffd640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fdc0c6260 0x7f0fdc153560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b820 tx=0x7f0fd000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.074+0000 7f0fc3fff640 1 -- 172.21.15.84:0/4090240929 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0004130 con 0x7f0fdc0c6260 2023-10-11T06:13:41.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.075+0000 7f0fc3fff640 1 -- 172.21.15.84:0/4090240929 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd00107d0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.075+0000 7f0fc3fff640 1 -- 172.21.15.84:0/4090240929 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0010b60 con 0x7f0fdc0c6260 2023-10-11T06:13:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.075+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc1f17a0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.075+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc1f5510 con 0x7f0fdc0c6260 2023-10-11T06:13:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.076+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc15f9b0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.077+0000 7f0fc3fff640 1 -- 172.21.15.84:0/4090240929 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0fd0010cc0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.078+0000 7f0fc3fff640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0fc806a830 0x7f0fc806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.078+0000 7f0fc3fff640 1 -- 172.21.15.84:0/4090240929 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0fd00892d0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.078+0000 7f0fdb7fe640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0fc806a830 0x7f0fc806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.079+0000 7f0fdb7fe640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0fc806a830 0x7f0fc806cce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc000c00 tx=0x7f0fcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:41.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.083+0000 7f0fc3fff640 1 -- 172.21.15.84:0/4090240929 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0fd00547d0 con 0x7f0fdc0c6260 2023-10-11T06:13:41.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:41.280+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0fdc0d2b60 con 0x7f0fdc0c6260 2023-10-11T06:13:42.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.053+0000 7f0fc3fff640 1 -- 172.21.15.84:0/4090240929 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0fd0057df0 con 0x7f0fdc0c6260 2023-10-11T06:13:42.055 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-10-11T06:13:42.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0fc806a830 msgr2=0x7f0fc806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0fc806a830 0x7f0fc806cce0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc000c00 tx=0x7f0fcc009040 comp rx=0 tx=0).stop 2023-10-11T06:13:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fdc0c6260 msgr2=0x7f0fdc153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fdc0c6260 0x7f0fdc153560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b820 tx=0x7f0fd000bc20 comp rx=0 tx=0).stop 2023-10-11T06:13:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 shutdown_connections 2023-10-11T06:13:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0fc806a830 0x7f0fc806cce0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fdc06b160 0x7f0fdc06b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fdc0c6260 0x7f0fdc153560 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 --2- 172.21.15.84:0/4090240929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fc4004d00 0x7f0fdc0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.056+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 >> 172.21.15.84:0/4090240929 conn(0x7f0fdc071cc0 msgr2=0x7f0fdc06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.057+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 shutdown_connections 2023-10-11T06:13:42.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.057+0000 7f0fe0cb9640 1 -- 172.21.15.84:0/4090240929 wait complete. 2023-10-11T06:13:42.077 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-11T06:13:42.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.288+0000 7f0460350640 1 Processor -- start 2023-10-11T06:13:42.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.288+0000 7f0460350640 1 -- start start 2023-10-11T06:13:42.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f0460350640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f045815d4f0 0x7f04580cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f0460350640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04580ca300 0x7f04580c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f0460350640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0458071a90 con 0x7f04580c9190 2023-10-11T06:13:42.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f0460350640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0458072310 con 0x7f04580ca300 2023-10-11T06:13:42.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f0460350640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0458070280 con 0x7f045815d4f0 2023-10-11T06:13:42.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f045d8eb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04580ca300 0x7f04580c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f045d8eb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f04580ca300 0x7f04580c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52752/0 (socket says 172.21.15.84:52752) 2023-10-11T06:13:42.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f045d8eb640 1 -- 172.21.15.84:0/2910788152 learned_addr learned my addr 172.21.15.84:0/2910788152 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:42.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.290+0000 7f045e0ec640 1 --2- 172.21.15.84:0/2910788152 >> v2:172.21.15.99:3300/0 conn(0x7f045815d4f0 0x7f04580cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.291+0000 7f045d8eb640 1 -- 172.21.15.84:0/2910788152 >> v2:172.21.15.99:3300/0 conn(0x7f045815d4f0 msgr2=0x7f04580cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:42.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.291+0000 7f045d8eb640 1 --2- 172.21.15.84:0/2910788152 >> v2:172.21.15.99:3300/0 conn(0x7f045815d4f0 0x7f04580cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.291+0000 7f045d8eb640 1 -- 172.21.15.84:0/2910788152 >> v1:172.21.15.99:6789/0 conn(0x7f04580c9190 legacy=0x7f045815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:42.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.291+0000 7f045d8eb640 1 -- 172.21.15.84:0/2910788152 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458071cc0 con 0x7f04580ca300 2023-10-11T06:13:42.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.292+0000 7f045d8eb640 1 --2- 172.21.15.84:0/2910788152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04580ca300 0x7f04580c8c50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f04580cf6a0 tx=0x7f044802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da025a5bb8eea2b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:42.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.292+0000 7f045d0ea640 1 -- 172.21.15.84:0/2910788152 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448031870 con 0x7f04580ca300 2023-10-11T06:13:42.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.292+0000 7f045d0ea640 1 -- 172.21.15.84:0/2910788152 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04480319d0 con 0x7f04580ca300 2023-10-11T06:13:42.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.292+0000 7f045d0ea640 1 -- 172.21.15.84:0/2910788152 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448031d60 con 0x7f04580ca300 2023-10-11T06:13:42.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 -- 172.21.15.84:0/2910788152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04580ca300 msgr2=0x7f04580c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 --2- 172.21.15.84:0/2910788152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04580ca300 0x7f04580c8c50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f04580cf6a0 tx=0x7f044802b910 comp rx=0 tx=0).stop 2023-10-11T06:13:42.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 -- 172.21.15.84:0/2910788152 shutdown_connections 2023-10-11T06:13:42.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 --2- 172.21.15.84:0/2910788152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04580ca300 0x7f04580c8c50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 --2- 172.21.15.84:0/2910788152 >> v2:172.21.15.99:3300/0 conn(0x7f045815d4f0 0x7f04580cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 -- 172.21.15.84:0/2910788152 >> 172.21.15.84:0/2910788152 conn(0x7f0458069c70 msgr2=0x7f045806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:42.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 -- 172.21.15.84:0/2910788152 shutdown_connections 2023-10-11T06:13:42.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.293+0000 7f0460350640 1 -- 172.21.15.84:0/2910788152 wait complete. 2023-10-11T06:13:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.294+0000 7f0460350640 1 Processor -- start 2023-10-11T06:13:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.295+0000 7f0460350640 1 -- start start 2023-10-11T06:13:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.295+0000 7f0460350640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04580c9190 0x7f0458156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.295+0000 7f0460350640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04580ca300 0x7f04580c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.295+0000 7f0460350640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f045815d4f0 0x7f045814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.295+0000 7f0460350640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04580a74d0 con 0x7f045815d4f0 2023-10-11T06:13:42.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.295+0000 7f0460350640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04580c6190 con 0x7f04580c9190 2023-10-11T06:13:42.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f0460350640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04580c6470 con 0x7f04580ca300 2023-10-11T06:13:42.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f045815d4f0 0x7f045814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f045815d4f0 0x7f045814a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52754/0 (socket says 172.21.15.84:52754) 2023-10-11T06:13:42.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 -- 172.21.15.84:0/309684541 learned_addr learned my addr 172.21.15.84:0/309684541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:42.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e0ec640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04580c9190 0x7f0458156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 -- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04580ca300 msgr2=0x7f04580c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045d8eb640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04580ca300 0x7f04580c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04580ca300 0x7f04580c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 -- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04580c9190 msgr2=0x7f0458156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04580c9190 0x7f0458156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 -- 172.21.15.84:0/309684541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04480094d0 con 0x7f045815d4f0 2023-10-11T06:13:42.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.296+0000 7f045e8ed640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f045815d4f0 0x7f045814a170 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f045000b970 tx=0x7f045000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:42.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.297+0000 7f044effd640 1 -- 172.21.15.84:0/309684541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045000cb40 con 0x7f045815d4f0 2023-10-11T06:13:42.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.297+0000 7f044effd640 1 -- 172.21.15.84:0/309684541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045000cca0 con 0x7f045815d4f0 2023-10-11T06:13:42.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.297+0000 7f044effd640 1 -- 172.21.15.84:0/309684541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450014680 con 0x7f045815d4f0 2023-10-11T06:13:42.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.297+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04580c6750 con 0x7f045815d4f0 2023-10-11T06:13:42.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.297+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0458159390 con 0x7f045815d4f0 2023-10-11T06:13:42.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.298+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0458073ed0 con 0x7f045815d4f0 2023-10-11T06:13:42.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.299+0000 7f044effd640 1 -- 172.21.15.84:0/309684541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f045001d3b0 con 0x7f045815d4f0 2023-10-11T06:13:42.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.300+0000 7f044effd640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f043c06a830 0x7f043c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.300+0000 7f044effd640 1 -- 172.21.15.84:0/309684541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0450088ab0 con 0x7f045815d4f0 2023-10-11T06:13:42.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.300+0000 7f045e0ec640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f043c06a830 0x7f043c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.301+0000 7f045e0ec640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f043c06a830 0x7f043c06cce0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0454009770 tx=0x7f0454008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:42.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.305+0000 7f044effd640 1 -- 172.21.15.84:0/309684541 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0450054080 con 0x7f045815d4f0 2023-10-11T06:13:42.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.559+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0458159580 con 0x7f045815d4f0 2023-10-11T06:13:42.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.560+0000 7f044effd640 1 -- 172.21.15.84:0/309684541 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f04500576a0 con 0x7f045815d4f0 2023-10-11T06:13:42.562 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:42.562 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:13:42.562 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-10-11T06:13:42.562 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.563 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-10-11T06:13:42.563 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZl2TIZBBAAHTDKquuoZy+2HB2lk815Fg==", 2023-10-11T06:13:42.563 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.563 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.563 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:13:42.564 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:13:42.564 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.564 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.565 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-10-11T06:13:42.565 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZlaZYAJhAAzmJNe7HYcmHJu+7Fbfv6dw==", 2023-10-11T06:13:42.565 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.565 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.565 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:13:42.565 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:13:42.566 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.566 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.566 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.566 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-10-11T06:13:42.567 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZlj+QWFBAAw6XDEihH6iXYbQsLht/VsQ==", 2023-10-11T06:13:42.567 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.567 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.567 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:13:42.567 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:13:42.567 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.568 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.568 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.568 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-10-11T06:13:42.568 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA4PCZlgvsCNxAAJL7HKUMYQA6zG6useZDoQw==", 2023-10-11T06:13:42.568 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.569 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.569 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:13:42.569 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:13:42.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.569 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.570 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.570 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-10-11T06:13:42.570 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA9PCZljp29IBAAFJgwhWwcw3OWMIDeeXR2sw==", 2023-10-11T06:13:42.570 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.570 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.571 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.571 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.571 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.572 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-10-11T06:13:42.572 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBAPCZlBCAQLBAAeWQVnN/FYimDHxX1nYzVKA==", 2023-10-11T06:13:42.572 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.572 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.572 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.572 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.573 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.573 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.573 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-10-11T06:13:42.573 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBDPCZlu1o+NRAAjAauNPRb88x0p37gT88vBw==", 2023-10-11T06:13:42.573 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.574 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.574 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.574 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.574 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.574 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.575 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-10-11T06:13:42.575 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBHPCZlcy98BBAALniVjO3ojVM/+q9fw6tNsw==", 2023-10-11T06:13:42.575 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.575 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.575 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.575 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.576 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.576 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.576 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-10-11T06:13:42.576 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBOPCZlmds+CBAAb0nHqugOzMsqLZCUabaHGQ==", 2023-10-11T06:13:42.576 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.577 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.577 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.577 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.577 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.577 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.578 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-10-11T06:13:42.578 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBRPCZlPObDEhAA1yMeENf3iV3lQf6f6WM8UQ==", 2023-10-11T06:13:42.578 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.578 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.578 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.579 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.579 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.579 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.579 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-10-11T06:13:42.579 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBUPCZlZQvrHBAAc/hx1KwJuGfHv5M+Tobkjg==", 2023-10-11T06:13:42.580 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.580 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.580 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.580 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.580 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.581 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-10-11T06:13:42.581 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBXPCZlKKt5NRAAkM8aLRyZPYaG0n1AH2tDIw==", 2023-10-11T06:13:42.581 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.581 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-10-11T06:13:42.581 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-10-11T06:13:42.582 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.582 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.582 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.582 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-10-11T06:13:42.583 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZlP/K+CRAAYT5I5tAyH8nHrsBReUAL9A==", 2023-10-11T06:13:42.583 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.583 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.583 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-10-11T06:13:42.583 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw pool=cephfs_data" 2023-10-11T06:13:42.583 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.584 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-10-11T06:13:42.584 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZlzhkWHxAAMV2BVukY7a/l7AXFNPCMhg==", 2023-10-11T06:13:42.584 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.585 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.585 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-11T06:13:42.585 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-11T06:13:42.585 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-11T06:13:42.585 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.586 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-10-11T06:13:42.586 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZli1j2ExAA77o/7VayFJb+C/TfdZrxRg==", 2023-10-11T06:13:42.586 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.586 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.587 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-11T06:13:42.587 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-11T06:13:42.587 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-11T06:13:42.587 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.588 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-10-11T06:13:42.588 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA5PCZljnzfKRAA2s7Yg5NkKNp/LP1eWIBEVA==", 2023-10-11T06:13:42.588 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.588 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-10-11T06:13:42.589 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-10-11T06:13:42.589 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-10-11T06:13:42.589 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-10-11T06:13:42.589 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.590 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-10-11T06:13:42.590 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA2PCZlczZbMRAAMctxZazuzrHiDNfSBIb+sA==", 2023-10-11T06:13:42.590 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.590 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-11T06:13:42.590 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-10-11T06:13:42.591 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-10-11T06:13:42.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.591 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.591 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-10-11T06:13:42.592 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA3PCZlsakeMxAA52n80muv9FKTIvML6txp5w==", 2023-10-11T06:13:42.592 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.592 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-11T06:13:42.592 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-11T06:13:42.592 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.592 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:42.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:42.593 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-10-11T06:13:42.593 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQA3PCZlcQtlJhAAyb6WWJKJgbNLkVkNL36ttg==", 2023-10-11T06:13:42.593 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-10-11T06:13:42.593 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-10-11T06:13:42.594 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-10-11T06:13:42.594 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-10-11T06:13:42.594 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.594 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:42.594 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-10-11T06:13:42.595 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:13:42.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f043c06a830 msgr2=0x7f043c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f043c06a830 0x7f043c06cce0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0454009770 tx=0x7f0454008040 comp rx=0 tx=0).stop 2023-10-11T06:13:42.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f045815d4f0 msgr2=0x7f045814a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f045815d4f0 0x7f045814a170 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f045000b970 tx=0x7f045000be30 comp rx=0 tx=0).stop 2023-10-11T06:13:42.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 shutdown_connections 2023-10-11T06:13:42.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f043c06a830 0x7f043c06cce0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f045815d4f0 0x7f045814a170 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04580ca300 0x7f04580c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 --2- 172.21.15.84:0/309684541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04580c9190 0x7f0458156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 >> 172.21.15.84:0/309684541 conn(0x7f0458069c70 msgr2=0x7f04580af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:42.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 shutdown_connections 2023-10-11T06:13:42.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.563+0000 7f0460350640 1 -- 172.21.15.84:0/309684541 wait complete. 2023-10-11T06:13:42.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:42.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.811+0000 7f1853fff640 1 Processor -- start 2023-10-11T06:13:42.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.811+0000 7f1853fff640 1 -- start start 2023-10-11T06:13:42.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.812+0000 7f1853fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1854159d00 0x7f185415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.812+0000 7f1853fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18540b3f90 0x7f1854071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.812+0000 7f1853fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f18540719c0 0x7f185415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.812+0000 7f1853fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185406c320 con 0x7f18540b3f90 2023-10-11T06:13:42.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.812+0000 7f1853fff640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185415aa00 con 0x7f18540719c0 2023-10-11T06:13:42.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.812+0000 7f1853fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185415ace0 con 0x7f1854159d00 2023-10-11T06:13:42.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18540b3f90 0x7f1854071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18540b3f90 0x7f1854071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52766/0 (socket says 172.21.15.84:52766) 2023-10-11T06:13:42.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 -- 172.21.15.84:0/1419410332 learned_addr learned my addr 172.21.15.84:0/1419410332 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:42.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18537fe640 1 --2- 172.21.15.84:0/1419410332 >> v2:172.21.15.99:3300/0 conn(0x7f18540719c0 0x7f185415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f1852ffd640 1 --2- 172.21.15.84:0/1419410332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1854159d00 0x7f185415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 -- 172.21.15.84:0/1419410332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1854159d00 msgr2=0x7f185415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:42.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 --2- 172.21.15.84:0/1419410332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1854159d00 0x7f185415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 -- 172.21.15.84:0/1419410332 >> v2:172.21.15.99:3300/0 conn(0x7f18540719c0 msgr2=0x7f185415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:42.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 --2- 172.21.15.84:0/1419410332 >> v2:172.21.15.99:3300/0 conn(0x7f18540719c0 0x7f185415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.813+0000 7f18527fc640 1 -- 172.21.15.84:0/1419410332 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185415afc0 con 0x7f18540b3f90 2023-10-11T06:13:42.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.814+0000 7f18527fc640 1 --2- 172.21.15.84:0/1419410332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18540b3f90 0x7f1854071480 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f185406d7a0 tx=0x7f184802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e8ebd23e02f180 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:42.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1851ffb640 1 -- 172.21.15.84:0/1419410332 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18480317d0 con 0x7f18540b3f90 2023-10-11T06:13:42.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1851ffb640 1 -- 172.21.15.84:0/1419410332 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1848031930 con 0x7f18540b3f90 2023-10-11T06:13:42.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1851ffb640 1 -- 172.21.15.84:0/1419410332 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848031cc0 con 0x7f18540b3f90 2023-10-11T06:13:42.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1853fff640 1 -- 172.21.15.84:0/1419410332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18540b3f90 msgr2=0x7f1854071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1853fff640 1 --2- 172.21.15.84:0/1419410332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18540b3f90 0x7f1854071480 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f185406d7a0 tx=0x7f184802b870 comp rx=0 tx=0).stop 2023-10-11T06:13:42.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1853fff640 1 -- 172.21.15.84:0/1419410332 shutdown_connections 2023-10-11T06:13:42.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1853fff640 1 --2- 172.21.15.84:0/1419410332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18540b3f90 0x7f1854071480 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1853fff640 1 --2- 172.21.15.84:0/1419410332 >> v2:172.21.15.99:3300/0 conn(0x7f18540719c0 0x7f185415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1853fff640 1 --2- 172.21.15.84:0/1419410332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1854159d00 0x7f185415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.815+0000 7f1853fff640 1 -- 172.21.15.84:0/1419410332 >> 172.21.15.84:0/1419410332 conn(0x7f1854069c70 msgr2=0x7f185406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:42.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.816+0000 7f1853fff640 1 -- 172.21.15.84:0/1419410332 shutdown_connections 2023-10-11T06:13:42.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.816+0000 7f1853fff640 1 -- 172.21.15.84:0/1419410332 wait complete. 2023-10-11T06:13:42.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.817+0000 7f1853fff640 1 Processor -- start 2023-10-11T06:13:42.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.818+0000 7f1853fff640 1 -- start start 2023-10-11T06:13:42.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.818+0000 7f1853fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18540719c0 0x7f185415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f1853fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18540b3f90 0x7f18540b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f1853fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1854159d00 0x7f18541f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f1853fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185415fdd0 con 0x7f1854159d00 2023-10-11T06:13:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f1853fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185415ef20 con 0x7f18540719c0 2023-10-11T06:13:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f1853fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185415f1c0 con 0x7f18540b3f90 2023-10-11T06:13:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1854159d00 0x7f18541f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1854159d00 0x7f18541f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52776/0 (socket says 172.21.15.84:52776) 2023-10-11T06:13:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 -- 172.21.15.84:0/13438022 learned_addr learned my addr 172.21.15.84:0/13438022 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f1852ffd640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18540719c0 0x7f185415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18527fc640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18540b3f90 0x7f18540b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 -- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18540b3f90 msgr2=0x7f18540b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18540b3f90 0x7f18540b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 -- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18540719c0 msgr2=0x7f185415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18540719c0 0x7f185415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.819+0000 7f18537fe640 1 -- 172.21.15.84:0/13438022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18480094d0 con 0x7f1854159d00 2023-10-11T06:13:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.820+0000 7f18537fe640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1854159d00 0x7f18541f4ca0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f184c007e40 tx=0x7f184c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.820+0000 7f183bfff640 1 -- 172.21.15.84:0/13438022 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c014620 con 0x7f1854159d00 2023-10-11T06:13:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.820+0000 7f183bfff640 1 -- 172.21.15.84:0/13438022 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c00ee90 con 0x7f1854159d00 2023-10-11T06:13:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.820+0000 7f183bfff640 1 -- 172.21.15.84:0/13438022 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c0149e0 con 0x7f1854159d00 2023-10-11T06:13:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.820+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185415f4a0 con 0x7f1854159d00 2023-10-11T06:13:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.820+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185415fa60 con 0x7f1854159d00 2023-10-11T06:13:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.821+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1854156d50 con 0x7f1854159d00 2023-10-11T06:13:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.822+0000 7f183bfff640 1 -- 172.21.15.84:0/13438022 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f184c014b40 con 0x7f1854159d00 2023-10-11T06:13:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.824+0000 7f183bfff640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f184006a830 0x7f184006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.824+0000 7f183bfff640 1 -- 172.21.15.84:0/13438022 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f184c089630 con 0x7f1854159d00 2023-10-11T06:13:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.824+0000 7f1852ffd640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f184006a830 0x7f184006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:42.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.825+0000 7f1852ffd640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f184006a830 0x7f184006cce0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1844006fd0 tx=0x7f1844008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:42.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:42.828+0000 7f183bfff640 1 -- 172.21.15.84:0/13438022 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f184c054c00 con 0x7f1854159d00 2023-10-11T06:13:43.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.022+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f185404e890 con 0x7f1854159d00 2023-10-11T06:13:43.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.023+0000 7f183bfff640 1 -- 172.21.15.84:0/13438022 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+14123 (secure 0 0 0) 0x7f184c089070 con 0x7f1854159d00 2023-10-11T06:13:43.024 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:43.025 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":50,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:42.043346+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:43.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.025+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f184006a830 msgr2=0x7f184006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.025+0000 7f1853fff640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f184006a830 0x7f184006cce0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1844006fd0 tx=0x7f1844008040 comp rx=0 tx=0).stop 2023-10-11T06:13:43.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1854159d00 msgr2=0x7f18541f4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1854159d00 0x7f18541f4ca0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f184c007e40 tx=0x7f184c00e4b0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 shutdown_connections 2023-10-11T06:13:43.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f184006a830 0x7f184006cce0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1854159d00 0x7f18541f4ca0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18540b3f90 0x7f18540b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 --2- 172.21.15.84:0/13438022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18540719c0 0x7f185415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 >> 172.21.15.84:0/13438022 conn(0x7f1854069c70 msgr2=0x7f18540b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:43.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 shutdown_connections 2023-10-11T06:13:43.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.026+0000 7f1853fff640 1 -- 172.21.15.84:0/13438022 wait complete. 2023-10-11T06:13:43.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:13:43.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:13:43.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.260+0000 7f19ff2b4640 1 Processor -- start 2023-10-11T06:13:43.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.261+0000 7f19ff2b4640 1 -- start start 2023-10-11T06:13:43.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.262+0000 7f19ff2b4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f815f7e0 0x7f19f80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.262+0000 7f19ff2b4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f80cf220 con 0x7f19f80d31d0 2023-10-11T06:13:43.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.262+0000 7f19ff2b4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f80cf500 con 0x7f19f80cd6c0 2023-10-11T06:13:43.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.262+0000 7f19ff2b4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80cf7e0 con 0x7f19f815f7e0 2023-10-11T06:13:43.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.262+0000 7f19fd851640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f19f80d31d0 0x7f19f80d0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35504/0 (socket says 172.21.15.84:35504) 2023-10-11T06:13:43.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.262+0000 7f19fd851640 1 -- 172.21.15.84:0/1562039260 learned_addr learned my addr 172.21.15.84:0/1562039260 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:43.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993717834 0 0) 0x7f19f80cf220 con 0x7f19f80d31d0 2023-10-11T06:13:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e4003520 con 0x7f19f80d31d0 2023-10-11T06:13:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19fd050640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f815f7e0 0x7f19f80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3637760951 0 0) 0x7f19e4003520 con 0x7f19f80d31d0 2023-10-11T06:13:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f815f7e0 msgr2=0x7f19f80cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:43.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f815f7e0 0x7f19f80cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 >> v1:172.21.15.99:6789/0 conn(0x7f19f80cd6c0 legacy=0x7f19f80d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:43.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80cfac0 con 0x7f19f80d31d0 2023-10-11T06:13:43.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f19f0001a10 con 0x7f19f80d31d0 2023-10-11T06:13:43.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 >> v1:172.21.15.84:6789/0 conn(0x7f19f80d31d0 legacy=0x7f19f80d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19e40039a0 0x7f19e4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f815f7e0 0x7f19e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e4004d00 0x7f19e4005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80cf7e0 con 0x7f19e40039a0 2023-10-11T06:13:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e400ba90 con 0x7f19e4004d00 2023-10-11T06:13:43.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.263+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e400bc50 con 0x7f19f815f7e0 2023-10-11T06:13:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.264+0000 7f19fd050640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19e40039a0 0x7f19e4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.264+0000 7f19fd050640 1 -- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f815f7e0 msgr2=0x7f19e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.264+0000 7f19fd050640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f815f7e0 0x7f19e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.264+0000 7f19fd050640 1 -- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e4004d00 msgr2=0x7f19e4005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:43.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.264+0000 7f19fd050640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e4004d00 0x7f19e4005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.264+0000 7f19fd050640 1 -- 172.21.15.84:0/1562039260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4003520 con 0x7f19e40039a0 2023-10-11T06:13:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.265+0000 7f19fd050640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19e40039a0 0x7f19e4003db0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f19f400ba30 tx=0x7f19e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2eaca005d5808a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.265+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4032680 con 0x7f19e40039a0 2023-10-11T06:13:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.265+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f40327e0 con 0x7f19e40039a0 2023-10-11T06:13:43.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.265+0000 7f19effff640 1 -- 172.21.15.84:0/1562039260 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4032b70 con 0x7f19e40039a0 2023-10-11T06:13:43.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.266+0000 7f19ff2b4640 1 -- 172.21.15.84:0/1562039260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19e40039a0 msgr2=0x7f19e4003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.266+0000 7f19ff2b4640 1 --2- 172.21.15.84:0/1562039260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19e40039a0 0x7f19e4003db0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f19f400ba30 tx=0x7f19e40057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.266+0000 7f19fd851640 1 -- 172.21.15.84:0/1562039260 reap_dead start 2023-10-11T06:13:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.266+0000 7f19ff2b4640 1 -- 172.21.15.84:0/1562039260 shutdown_connections 2023-10-11T06:13:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.266+0000 7f19ff2b4640 1 -- 172.21.15.84:0/1562039260 >> 172.21.15.84:0/1562039260 conn(0x7f19f80713a0 msgr2=0x7f19f80717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.266+0000 7f19ff2b4640 1 -- 172.21.15.84:0/1562039260 shutdown_connections 2023-10-11T06:13:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.266+0000 7f19ff2b4640 1 -- 172.21.15.84:0/1562039260 wait complete. 2023-10-11T06:13:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.268+0000 7f19ff2b4640 1 Processor -- start 2023-10-11T06:13:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.268+0000 7f19ff2b4640 1 -- start start 2023-10-11T06:13:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19ff2b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e40039a0 0x7f19f81f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19ff2b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f8156c50 0x7f19f80b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19ff2b4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0fa0 0x7f19f80d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19ff2b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f81f1b00 con 0x7f19f8156c50 2023-10-11T06:13:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19ff2b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f814e8c0 con 0x7f19e40039a0 2023-10-11T06:13:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19ff2b4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d1950 con 0x7f19f80d0fa0 2023-10-11T06:13:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19fc84f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f8156c50 0x7f19f80b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.269+0000 7f19fc84f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f8156c50 0x7f19f80b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52794/0 (socket says 172.21.15.84:52794) 2023-10-11T06:13:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19fc84f640 1 -- 172.21.15.84:0/2242733188 learned_addr learned my addr 172.21.15.84:0/2242733188 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19fc84f640 1 -- 172.21.15.84:0/2242733188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0fa0 msgr2=0x7f19f80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19fc84f640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0fa0 0x7f19f80d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19fc84f640 1 -- 172.21.15.84:0/2242733188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e40039a0 msgr2=0x7f19f81f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19fc84f640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e40039a0 0x7f19f81f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19fc84f640 1 -- 172.21.15.84:0/2242733188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80b4420 con 0x7f19f8156c50 2023-10-11T06:13:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19fc84f640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f8156c50 0x7f19f80b2ea0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c460 tx=0x7f19e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19edffb640 1 -- 172.21.15.84:0/2242733188 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e80052d0 con 0x7f19f8156c50 2023-10-11T06:13:43.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19edffb640 1 -- 172.21.15.84:0/2242733188 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e8005450 con 0x7f19f8156c50 2023-10-11T06:13:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f80b3810 con 0x7f19f8156c50 2023-10-11T06:13:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.270+0000 7f19edffb640 1 -- 172.21.15.84:0/2242733188 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8012870 con 0x7f19f8156c50 2023-10-11T06:13:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.271+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f80b3dd0 con 0x7f19f8156c50 2023-10-11T06:13:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.272+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f8073ed0 con 0x7f19f8156c50 2023-10-11T06:13:43.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.272+0000 7f19edffb640 1 -- 172.21.15.84:0/2242733188 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f19e80129d0 con 0x7f19f8156c50 2023-10-11T06:13:43.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.273+0000 7f19edffb640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19dc06a830 0x7f19dc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.273+0000 7f19edffb640 1 -- 172.21.15.84:0/2242733188 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f19e8088730 con 0x7f19f8156c50 2023-10-11T06:13:43.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.274+0000 7f19fd050640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19dc06a830 0x7f19dc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.274+0000 7f19fd050640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19dc06a830 0x7f19dc06cce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19f400ba00 tx=0x7f19f4032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:43.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.278+0000 7f19edffb640 1 -- 172.21.15.84:0/2242733188 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f19e8053cf0 con 0x7f19f8156c50 2023-10-11T06:13:43.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.468+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19f80b5030 con 0x7f19f8156c50 2023-10-11T06:13:43.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.469+0000 7f19edffb640 1 -- 172.21.15.84:0/2242733188 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19e8057310 con 0x7f19f8156c50 2023-10-11T06:13:43.471 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:13:43.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19dc06a830 msgr2=0x7f19dc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19dc06a830 0x7f19dc06cce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19f400ba00 tx=0x7f19f4032000 comp rx=0 tx=0).stop 2023-10-11T06:13:43.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f8156c50 msgr2=0x7f19f80b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f8156c50 0x7f19f80b2ea0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c460 tx=0x7f19e800c920 comp rx=0 tx=0).stop 2023-10-11T06:13:43.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 shutdown_connections 2023-10-11T06:13:43.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f19dc06a830 0x7f19dc06cce0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19f80d0fa0 0x7f19f80d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f8156c50 0x7f19f80b2ea0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 --2- 172.21.15.84:0/2242733188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e40039a0 0x7f19f81f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 >> 172.21.15.84:0/2242733188 conn(0x7f19f80713a0 msgr2=0x7f19f80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:43.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 shutdown_connections 2023-10-11T06:13:43.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.472+0000 7f19ff2b4640 1 -- 172.21.15.84:0/2242733188 wait complete. 2023-10-11T06:13:43.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:13:43.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.703+0000 7f3a6d610640 1 Processor -- start 2023-10-11T06:13:43.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.703+0000 7f3a6d610640 1 -- start start 2023-10-11T06:13:43.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a6d610640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3a68150300 0x7f3a680c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a6d610640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3a680ce510 0x7f3a680ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a6d610640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a680c9b90 con 0x7f3a6814ff30 2023-10-11T06:13:43.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a6d610640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6815da10 con 0x7f3a68150300 2023-10-11T06:13:43.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a6d610640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680cc990 con 0x7f3a680ce510 2023-10-11T06:13:43.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a5ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3a68150300 0x7f3a680c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a5ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3a68150300 0x7f3a680c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52808/0 (socket says 172.21.15.84:52808) 2023-10-11T06:13:43.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a5ffff640 1 -- 172.21.15.84:0/3105695413 learned_addr learned my addr 172.21.15.84:0/3105695413 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:43.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.705+0000 7f3a677fe640 1 --2- 172.21.15.84:0/3105695413 >> v2:172.21.15.99:3300/0 conn(0x7f3a680ce510 0x7f3a680ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.706+0000 7f3a5ffff640 1 -- 172.21.15.84:0/3105695413 >> v2:172.21.15.99:3300/0 conn(0x7f3a680ce510 msgr2=0x7f3a680ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:43.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.706+0000 7f3a5ffff640 1 --2- 172.21.15.84:0/3105695413 >> v2:172.21.15.99:3300/0 conn(0x7f3a680ce510 0x7f3a680ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.706+0000 7f3a5ffff640 1 -- 172.21.15.84:0/3105695413 >> v1:172.21.15.99:6789/0 conn(0x7f3a6814ff30 legacy=0x7f3a680cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.706+0000 7f3a5ffff640 1 -- 172.21.15.84:0/3105695413 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6815ce00 con 0x7f3a68150300 2023-10-11T06:13:43.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.706+0000 7f3a677fe640 1 --2- 172.21.15.84:0/3105695413 >> v2:172.21.15.99:3300/0 conn(0x7f3a680ce510 0x7f3a680ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:43.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.707+0000 7f3a5ffff640 1 --2- 172.21.15.84:0/3105695413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a68150300 0x7f3a680c9650 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3a6806e3f0 tx=0x7f3a5002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe4f1e537f9cee3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:43.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.707+0000 7f3a667fc640 1 -- 172.21.15.84:0/3105695413 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500317e0 con 0x7f3a68150300 2023-10-11T06:13:43.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.707+0000 7f3a667fc640 1 -- 172.21.15.84:0/3105695413 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50031940 con 0x7f3a68150300 2023-10-11T06:13:43.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.707+0000 7f3a667fc640 1 -- 172.21.15.84:0/3105695413 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50031cd0 con 0x7f3a68150300 2023-10-11T06:13:43.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 -- 172.21.15.84:0/3105695413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a68150300 msgr2=0x7f3a680c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 --2- 172.21.15.84:0/3105695413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a68150300 0x7f3a680c9650 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3a6806e3f0 tx=0x7f3a5002b880 comp rx=0 tx=0).stop 2023-10-11T06:13:43.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 -- 172.21.15.84:0/3105695413 shutdown_connections 2023-10-11T06:13:43.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 --2- 172.21.15.84:0/3105695413 >> v2:172.21.15.99:3300/0 conn(0x7f3a680ce510 0x7f3a680ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 --2- 172.21.15.84:0/3105695413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a68150300 0x7f3a680c9650 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 -- 172.21.15.84:0/3105695413 >> 172.21.15.84:0/3105695413 conn(0x7f3a68069c70 msgr2=0x7f3a6806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:43.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 -- 172.21.15.84:0/3105695413 shutdown_connections 2023-10-11T06:13:43.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.708+0000 7f3a6d610640 1 -- 172.21.15.84:0/3105695413 wait complete. 2023-10-11T06:13:43.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.710+0000 7f3a6d610640 1 Processor -- start 2023-10-11T06:13:43.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.710+0000 7f3a6d610640 1 -- start start 2023-10-11T06:13:43.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a6d610640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a680ce510 0x7f3a6815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a6d610640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a6814ff30 0x7f3a68158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a6d610640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a68150300 0x7f3a6815c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a6d610640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a68070ee0 con 0x7f3a6814ff30 2023-10-11T06:13:43.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a6d610640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6815be30 con 0x7f3a68150300 2023-10-11T06:13:43.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a6d610640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6815bb20 con 0x7f3a680ce510 2023-10-11T06:13:43.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a5ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a6814ff30 0x7f3a68158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a5ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a6814ff30 0x7f3a68158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52820/0 (socket says 172.21.15.84:52820) 2023-10-11T06:13:43.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.711+0000 7f3a5ffff640 1 -- 172.21.15.84:0/192879746 learned_addr learned my addr 172.21.15.84:0/192879746 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:43.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5ffff640 1 -- 172.21.15.84:0/192879746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a680ce510 msgr2=0x7f3a6815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5ffff640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a680ce510 0x7f3a6815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5ffff640 1 -- 172.21.15.84:0/192879746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a68150300 msgr2=0x7f3a6815c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5ffff640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a68150300 0x7f3a6815c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5ffff640 1 -- 172.21.15.84:0/192879746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500094d0 con 0x7f3a6814ff30 2023-10-11T06:13:43.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5ffff640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a6814ff30 0x7f3a68158320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3a5002b8c0 tx=0x7f3a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:43.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5f7fe640 1 -- 172.21.15.84:0/192879746 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50032980 con 0x7f3a6814ff30 2023-10-11T06:13:43.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5f7fe640 1 -- 172.21.15.84:0/192879746 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50032ae0 con 0x7f3a6814ff30 2023-10-11T06:13:43.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a680b4110 con 0x7f3a6814ff30 2023-10-11T06:13:43.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.712+0000 7f3a5f7fe640 1 -- 172.21.15.84:0/192879746 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50032e90 con 0x7f3a6814ff30 2023-10-11T06:13:43.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.713+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a68143da0 con 0x7f3a6814ff30 2023-10-11T06:13:43.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.714+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a680b4630 con 0x7f3a6814ff30 2023-10-11T06:13:43.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.715+0000 7f3a5f7fe640 1 -- 172.21.15.84:0/192879746 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3a5004d3f0 con 0x7f3a6814ff30 2023-10-11T06:13:43.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.716+0000 7f3a5f7fe640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a4806a830 0x7f3a4806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:43.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.716+0000 7f3a5f7fe640 1 -- 172.21.15.84:0/192879746 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a500aabb0 con 0x7f3a6814ff30 2023-10-11T06:13:43.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.716+0000 7f3a66ffd640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a4806a830 0x7f3a4806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:43.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.717+0000 7f3a66ffd640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a4806a830 0x7f3a4806cce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3a58003cc0 tx=0x7f3a580079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:43.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.721+0000 7f3a5f7fe640 1 -- 172.21.15.84:0/192879746 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3a500762c0 con 0x7f3a6814ff30 2023-10-11T06:13:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.910+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3a6815d390 con 0x7f3a6814ff30 2023-10-11T06:13:43.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.911+0000 7f3a5f7fe640 1 -- 172.21.15.84:0/192879746 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3a500798e0 con 0x7f3a6814ff30 2023-10-11T06:13:43.913 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:13:43.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a4806a830 msgr2=0x7f3a4806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a4806a830 0x7f3a4806cce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3a58003cc0 tx=0x7f3a580079b0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a6814ff30 msgr2=0x7f3a68158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a6814ff30 0x7f3a68158320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3a5002b8c0 tx=0x7f3a500046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 shutdown_connections 2023-10-11T06:13:43.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a4806a830 0x7f3a4806cce0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a68150300 0x7f3a6815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a6814ff30 0x7f3a68158320 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 --2- 172.21.15.84:0/192879746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a680ce510 0x7f3a6815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.914+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 >> 172.21.15.84:0/192879746 conn(0x7f3a68069c70 msgr2=0x7f3a6806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:43.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.915+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 shutdown_connections 2023-10-11T06:13:43.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:43.915+0000 7f3a6d610640 1 -- 172.21.15.84:0/192879746 wait complete. 2023-10-11T06:13:43.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-11T06:13:44.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.148+0000 7fd6c65dd640 1 Processor -- start 2023-10-11T06:13:44.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.149+0000 7fd6c65dd640 1 -- start start 2023-10-11T06:13:44.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.150+0000 7fd6c65dd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6c015d320 0x7fd6c00cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.150+0000 7fd6c65dd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd6c0069c70 0x7fd6c006a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.150+0000 7fd6c65dd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c006a5d0 con 0x7fd6c0159cd0 2023-10-11T06:13:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.150+0000 7fd6c65dd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c00ceda0 con 0x7fd6c015d320 2023-10-11T06:13:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.150+0000 7fd6c65dd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c015a0a0 con 0x7fd6c0069c70 2023-10-11T06:13:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.150+0000 7fd6bf7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6c015d320 0x7fd6c00cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.151+0000 7fd6bf7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6c015d320 0x7fd6c00cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52828/0 (socket says 172.21.15.84:52828) 2023-10-11T06:13:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.151+0000 7fd6bf7fe640 1 -- 172.21.15.84:0/3001567390 learned_addr learned my addr 172.21.15.84:0/3001567390 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.151+0000 7fd6c4b7a640 1 --2- 172.21.15.84:0/3001567390 >> v2:172.21.15.99:3300/0 conn(0x7fd6c0069c70 0x7fd6c006a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.151+0000 7fd6bf7fe640 1 -- 172.21.15.84:0/3001567390 >> v2:172.21.15.99:3300/0 conn(0x7fd6c0069c70 msgr2=0x7fd6c006a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.151+0000 7fd6bf7fe640 1 --2- 172.21.15.84:0/3001567390 >> v2:172.21.15.99:3300/0 conn(0x7fd6c0069c70 0x7fd6c006a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.151+0000 7fd6bf7fe640 1 -- 172.21.15.84:0/3001567390 >> v1:172.21.15.99:6789/0 conn(0x7fd6c0159cd0 legacy=0x7fd6c00b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:44.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.151+0000 7fd6bf7fe640 1 -- 172.21.15.84:0/3001567390 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00cf020 con 0x7fd6c015d320 2023-10-11T06:13:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.152+0000 7fd6bf7fe640 1 --2- 172.21.15.84:0/3001567390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c015d320 0x7fd6c00cd2d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00c9ce0 tx=0x7fd6b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9f692622e899b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.152+0000 7fd6beffd640 1 -- 172.21.15.84:0/3001567390 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4031870 con 0x7fd6c015d320 2023-10-11T06:13:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.152+0000 7fd6beffd640 1 -- 172.21.15.84:0/3001567390 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b40319d0 con 0x7fd6c015d320 2023-10-11T06:13:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.152+0000 7fd6beffd640 1 -- 172.21.15.84:0/3001567390 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4031d60 con 0x7fd6c015d320 2023-10-11T06:13:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 -- 172.21.15.84:0/3001567390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c015d320 msgr2=0x7fd6c00cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/3001567390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c015d320 0x7fd6c00cd2d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00c9ce0 tx=0x7fd6b402b910 comp rx=0 tx=0).stop 2023-10-11T06:13:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 -- 172.21.15.84:0/3001567390 shutdown_connections 2023-10-11T06:13:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/3001567390 >> v2:172.21.15.99:3300/0 conn(0x7fd6c0069c70 0x7fd6c006a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/3001567390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c015d320 0x7fd6c00cd2d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 -- 172.21.15.84:0/3001567390 >> 172.21.15.84:0/3001567390 conn(0x7fd6c006b9b0 msgr2=0x7fd6c006d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 -- 172.21.15.84:0/3001567390 shutdown_connections 2023-10-11T06:13:44.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.153+0000 7fd6c65dd640 1 -- 172.21.15.84:0/3001567390 wait complete. 2023-10-11T06:13:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.155+0000 7fd6c65dd640 1 Processor -- start 2023-10-11T06:13:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.155+0000 7fd6c65dd640 1 -- start start 2023-10-11T06:13:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6c65dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c0069c70 0x7fd6c00ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6c65dd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6c0159cd0 0x7fd6c00c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6c65dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c015d320 0x7fd6c01520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6c65dd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0152ca0 con 0x7fd6c0069c70 2023-10-11T06:13:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6c65dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00c32d0 con 0x7fd6c015d320 2023-10-11T06:13:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6c65dd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00c35b0 con 0x7fd6c0159cd0 2023-10-11T06:13:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c0069c70 0x7fd6c00ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6bffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c0069c70 0x7fd6c00ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52844/0 (socket says 172.21.15.84:52844) 2023-10-11T06:13:44.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.156+0000 7fd6bffff640 1 -- 172.21.15.84:0/4185623330 learned_addr learned my addr 172.21.15.84:0/4185623330 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bf7fe640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6c0159cd0 0x7fd6c00c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6c4b7a640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c015d320 0x7fd6c01520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bffff640 1 -- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6c0159cd0 msgr2=0x7fd6c00c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bffff640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6c0159cd0 0x7fd6c00c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bffff640 1 -- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c015d320 msgr2=0x7fd6c01520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bffff640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c015d320 0x7fd6c01520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bffff640 1 -- 172.21.15.84:0/4185623330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40094d0 con 0x7fd6c0069c70 2023-10-11T06:13:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bffff640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c0069c70 0x7fd6c00ae340 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c7a0 tx=0x7fd6b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bcff9640 1 -- 172.21.15.84:0/4185623330 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0003d80 con 0x7fd6c0069c70 2023-10-11T06:13:44.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bcff9640 1 -- 172.21.15.84:0/4185623330 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b0005070 con 0x7fd6c0069c70 2023-10-11T06:13:44.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.157+0000 7fd6bcff9640 1 -- 172.21.15.84:0/4185623330 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0005460 con 0x7fd6c0069c70 2023-10-11T06:13:44.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.158+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c00c3890 con 0x7fd6c0069c70 2023-10-11T06:13:44.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.158+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c00ad560 con 0x7fd6c0069c70 2023-10-11T06:13:44.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.159+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c00b12b0 con 0x7fd6c0069c70 2023-10-11T06:13:44.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.160+0000 7fd6bcff9640 1 -- 172.21.15.84:0/4185623330 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd6b00055c0 con 0x7fd6c0069c70 2023-10-11T06:13:44.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.161+0000 7fd6bcff9640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd6ac06a830 0x7fd6ac06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.161+0000 7fd6bcff9640 1 -- 172.21.15.84:0/4185623330 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd6b0088120 con 0x7fd6c0069c70 2023-10-11T06:13:44.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.161+0000 7fd6bf7fe640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd6ac06a830 0x7fd6ac06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.162+0000 7fd6bf7fe640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd6ac06a830 0x7fd6ac06cce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4036000 tx=0x7fd6b403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:44.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.165+0000 7fd6bcff9640 1 -- 172.21.15.84:0/4185623330 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd6b00536e0 con 0x7fd6c0069c70 2023-10-11T06:13:44.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.356+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd6c00b15b0 con 0x7fd6c0069c70 2023-10-11T06:13:44.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.356+0000 7fd6bcff9640 1 -- 172.21.15.84:0/4185623330 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd6b0056d00 con 0x7fd6c0069c70 2023-10-11T06:13:44.358 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:44.358 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-10-11T06:13:44.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:44.358 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-10-11T06:13:44.359 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-10-11T06:13:44.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:44.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-10-11T06:13:44.359 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 2, 2023-10-11T06:13:44.359 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "rbd" 2023-10-11T06:13:44.360 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:44.360 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-10-11T06:13:44.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd6ac06a830 msgr2=0x7fd6ac06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd6ac06a830 0x7fd6ac06cce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4036000 tx=0x7fd6b403c040 comp rx=0 tx=0).stop 2023-10-11T06:13:44.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c0069c70 msgr2=0x7fd6c00ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c0069c70 0x7fd6c00ae340 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c7a0 tx=0x7fd6b000cc60 comp rx=0 tx=0).stop 2023-10-11T06:13:44.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 shutdown_connections 2023-10-11T06:13:44.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd6ac06a830 0x7fd6ac06cce0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6c015d320 0x7fd6c01520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6c0159cd0 0x7fd6c00c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 --2- 172.21.15.84:0/4185623330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6c0069c70 0x7fd6c00ae340 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 >> 172.21.15.84:0/4185623330 conn(0x7fd6c006b9b0 msgr2=0x7fd6c00af160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:44.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 shutdown_connections 2023-10-11T06:13:44.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.359+0000 7fd6c65dd640 1 -- 172.21.15.84:0/4185623330 wait complete. 2023-10-11T06:13:44.378 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-11T06:13:44.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-11T06:13:44.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.585+0000 7f8b09037640 1 Processor -- start 2023-10-11T06:13:44.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.585+0000 7f8b09037640 1 -- start start 2023-10-11T06:13:44.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.586+0000 7f8b09037640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.586+0000 7f8b09037640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8b04156be0 0x7f8b0415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b09037640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b0406c170 0x7f8b0406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b09037640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040ce140 con 0x7f8b0406c170 2023-10-11T06:13:44.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b09037640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040cde80 con 0x7f8b04156be0 2023-10-11T06:13:44.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b09037640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0415da30 con 0x7f8b04156810 2023-10-11T06:13:44.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b0406c170 0x7f8b0406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b0406c170 0x7f8b0406c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52856/0 (socket says 172.21.15.84:52856) 2023-10-11T06:13:44.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 -- 172.21.15.84:0/3206373025 learned_addr learned my addr 172.21.15.84:0/3206373025 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:44.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b02d9d640 1 --2- 172.21.15.84:0/3206373025 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8afbfff640 1 --2- 172.21.15.84:0/3206373025 >> v2:172.21.15.99:3300/0 conn(0x7f8b04156be0 0x7f8b0415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 -- 172.21.15.84:0/3206373025 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 msgr2=0x7f8b0406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 --2- 172.21.15.84:0/3206373025 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 -- 172.21.15.84:0/3206373025 >> v2:172.21.15.99:3300/0 conn(0x7f8b04156be0 msgr2=0x7f8b0415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:44.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 --2- 172.21.15.84:0/3206373025 >> v2:172.21.15.99:3300/0 conn(0x7f8b04156be0 0x7f8b0415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b0359e640 1 -- 172.21.15.84:0/3206373025 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0406df80 con 0x7f8b0406c170 2023-10-11T06:13:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.587+0000 7f8b02d9d640 1 --2- 172.21.15.84:0/3206373025 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.589+0000 7f8b0359e640 1 --2- 172.21.15.84:0/3206373025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0406c5b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406e290 tx=0x7f8afc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b2c47d803ac5c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.589+0000 7f8b0259c640 1 -- 172.21.15.84:0/3206373025 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc02c9f0 con 0x7f8b0406c170 2023-10-11T06:13:44.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.589+0000 7f8b0259c640 1 -- 172.21.15.84:0/3206373025 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc02cb50 con 0x7f8b0406c170 2023-10-11T06:13:44.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.589+0000 7f8b0259c640 1 -- 172.21.15.84:0/3206373025 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0343f0 con 0x7f8b0406c170 2023-10-11T06:13:44.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 -- 172.21.15.84:0/3206373025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 msgr2=0x7f8b0406c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 --2- 172.21.15.84:0/3206373025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0406c5b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406e290 tx=0x7f8afc02be50 comp rx=0 tx=0).stop 2023-10-11T06:13:44.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 -- 172.21.15.84:0/3206373025 shutdown_connections 2023-10-11T06:13:44.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 --2- 172.21.15.84:0/3206373025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0406c5b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 --2- 172.21.15.84:0/3206373025 >> v2:172.21.15.99:3300/0 conn(0x7f8b04156be0 0x7f8b0415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 --2- 172.21.15.84:0/3206373025 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 -- 172.21.15.84:0/3206373025 >> 172.21.15.84:0/3206373025 conn(0x7f8b04069c70 msgr2=0x7f8b0406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:44.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 -- 172.21.15.84:0/3206373025 shutdown_connections 2023-10-11T06:13:44.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.590+0000 7f8b09037640 1 -- 172.21.15.84:0/3206373025 wait complete. 2023-10-11T06:13:44.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.592+0000 7f8b09037640 1 Processor -- start 2023-10-11T06:13:44.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.592+0000 7f8b09037640 1 -- start start 2023-10-11T06:13:44.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b09037640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b09037640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b09037640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b04156be0 0x7f8b04154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b09037640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04160a50 con 0x7f8b0406c170 2023-10-11T06:13:44.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b09037640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0415fe40 con 0x7f8b04156be0 2023-10-11T06:13:44.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b09037640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b04160120 con 0x7f8b04156810 2023-10-11T06:13:44.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0415f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52866/0 (socket says 172.21.15.84:52866) 2023-10-11T06:13:44.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 -- 172.21.15.84:0/3313359345 learned_addr learned my addr 172.21.15.84:0/3313359345 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:44.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 -- 172.21.15.84:0/3313359345 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 msgr2=0x7f8b0414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 -- 172.21.15.84:0/3313359345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b04156be0 msgr2=0x7f8b04154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b04156be0 0x7f8b04154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.593+0000 7f8b02d9d640 1 -- 172.21.15.84:0/3313359345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc009b30 con 0x7f8b0406c170 2023-10-11T06:13:44.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.594+0000 7f8b02d9d640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0415f5f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c7c0 tx=0x7f8af400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:44.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.594+0000 7f8afb7fe640 1 -- 172.21.15.84:0/3313359345 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4007a30 con 0x7f8b0406c170 2023-10-11T06:13:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.594+0000 7f8afb7fe640 1 -- 172.21.15.84:0/3313359345 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af4007b90 con 0x7f8b0406c170 2023-10-11T06:13:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.594+0000 7f8afb7fe640 1 -- 172.21.15.84:0/3313359345 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400ce70 con 0x7f8b0406c170 2023-10-11T06:13:44.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.594+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b04160400 con 0x7f8b0406c170 2023-10-11T06:13:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.594+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0415fb30 con 0x7f8b0406c170 2023-10-11T06:13:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.595+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b040ae260 con 0x7f8b0406c170 2023-10-11T06:13:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.596+0000 7f8afb7fe640 1 -- 172.21.15.84:0/3313359345 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8af40113f0 con 0x7f8b0406c170 2023-10-11T06:13:44.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.597+0000 7f8afb7fe640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ae406a830 0x7f8ae406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:44.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.597+0000 7f8afb7fe640 1 -- 172.21.15.84:0/3313359345 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8af4088d80 con 0x7f8b0406c170 2023-10-11T06:13:44.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.597+0000 7f8afbfff640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ae406a830 0x7f8ae406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:44.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.598+0000 7f8afbfff640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ae406a830 0x7f8ae406cce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8aec006fd0 tx=0x7f8aec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.601+0000 7f8afb7fe640 1 -- 172.21.15.84:0/3313359345 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8af40543e0 con 0x7f8b0406c170 2023-10-11T06:13:44.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.790+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8b04073ed0 con 0x7f8b0406c170 2023-10-11T06:13:44.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.905+0000 7f8afb7fe640 1 -- 172.21.15.84:0/3313359345 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8af4057a00 con 0x7f8b0406c170 2023-10-11T06:13:44.907 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-10-11T06:13:44.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.908+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ae406a830 msgr2=0x7f8ae406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ae406a830 0x7f8ae406cce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8aec006fd0 tx=0x7f8aec008040 comp rx=0 tx=0).stop 2023-10-11T06:13:44.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 msgr2=0x7f8b0415f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:44.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0415f5f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c7c0 tx=0x7f8af400cc80 comp rx=0 tx=0).stop 2023-10-11T06:13:44.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 shutdown_connections 2023-10-11T06:13:44.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8ae406a830 0x7f8ae406cce0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b04156be0 0x7f8b04154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b04156810 0x7f8b0414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 --2- 172.21.15.84:0/3313359345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b0406c170 0x7f8b0415f5f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:44.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 >> 172.21.15.84:0/3313359345 conn(0x7f8b04069c70 msgr2=0x7f8b040b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:44.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 shutdown_connections 2023-10-11T06:13:44.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:44.909+0000 7f8b09037640 1 -- 172.21.15.84:0/3313359345 wait complete. 2023-10-11T06:13:44.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-11T06:13:45.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.140+0000 7fd35a81a640 1 Processor -- start 2023-10-11T06:13:45.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.140+0000 7fd35a81a640 1 -- start start 2023-10-11T06:13:45.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd35a81a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd35415f7e0 0x7fd3540cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:45.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd35a81a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd3540cd6c0 0x7fd3540d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:45.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd35a81a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:45.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd35a81a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3540d3460 con 0x7fd35415f7e0 2023-10-11T06:13:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd35a81a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3540cf220 con 0x7fd3540cd6c0 2023-10-11T06:13:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd35a81a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540cf500 con 0x7fd3540d3000 2023-10-11T06:13:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd353fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd35415f7e0 0x7fd3540cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd353fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd35415f7e0 0x7fd3540cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52878/0 (socket says 172.21.15.84:52878) 2023-10-11T06:13:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd353fff640 1 -- 172.21.15.84:0/2606586057 learned_addr learned my addr 172.21.15.84:0/2606586057 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:45.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.142+0000 7fd34bfff640 1 --2- 172.21.15.84:0/2606586057 >> v2:172.21.15.99:3300/0 conn(0x7fd3540cd6c0 0x7fd3540d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:45.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd353fff640 1 -- 172.21.15.84:0/2606586057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 msgr2=0x7fd35415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:45.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd358db7640 1 --2- 172.21.15.84:0/2606586057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:45.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd353fff640 1 --2- 172.21.15.84:0/2606586057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd353fff640 1 -- 172.21.15.84:0/2606586057 >> v2:172.21.15.99:3300/0 conn(0x7fd3540cd6c0 msgr2=0x7fd3540d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:45.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd353fff640 1 --2- 172.21.15.84:0/2606586057 >> v2:172.21.15.99:3300/0 conn(0x7fd3540cd6c0 0x7fd3540d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd353fff640 1 -- 172.21.15.84:0/2606586057 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3540cf7e0 con 0x7fd35415f7e0 2023-10-11T06:13:45.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd34bfff640 1 --2- 172.21.15.84:0/2606586057 >> v2:172.21.15.99:3300/0 conn(0x7fd3540cd6c0 0x7fd3540d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:45.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.143+0000 7fd358db7640 1 --2- 172.21.15.84:0/2606586057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.144+0000 7fd353fff640 1 --2- 172.21.15.84:0/2606586057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3540cd180 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd35408adb0 tx=0x7fd34402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4129c069b388e46f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:45.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.144+0000 7fd3537fe640 1 -- 172.21.15.84:0/2606586057 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344031870 con 0x7fd35415f7e0 2023-10-11T06:13:45.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.144+0000 7fd3537fe640 1 -- 172.21.15.84:0/2606586057 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3440319d0 con 0x7fd35415f7e0 2023-10-11T06:13:45.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.144+0000 7fd3537fe640 1 -- 172.21.15.84:0/2606586057 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344031d60 con 0x7fd35415f7e0 2023-10-11T06:13:45.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 -- 172.21.15.84:0/2606586057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 msgr2=0x7fd3540cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:45.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 --2- 172.21.15.84:0/2606586057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3540cd180 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd35408adb0 tx=0x7fd34402b910 comp rx=0 tx=0).stop 2023-10-11T06:13:45.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 -- 172.21.15.84:0/2606586057 shutdown_connections 2023-10-11T06:13:45.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 --2- 172.21.15.84:0/2606586057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 --2- 172.21.15.84:0/2606586057 >> v2:172.21.15.99:3300/0 conn(0x7fd3540cd6c0 0x7fd3540d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 --2- 172.21.15.84:0/2606586057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3540cd180 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 -- 172.21.15.84:0/2606586057 >> 172.21.15.84:0/2606586057 conn(0x7fd3540713a0 msgr2=0x7fd3540717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:45.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 -- 172.21.15.84:0/2606586057 shutdown_connections 2023-10-11T06:13:45.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.145+0000 7fd35a81a640 1 -- 172.21.15.84:0/2606586057 wait complete. 2023-10-11T06:13:45.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.147+0000 7fd35a81a640 1 Processor -- start 2023-10-11T06:13:45.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.147+0000 7fd35a81a640 1 -- start start 2023-10-11T06:13:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.148+0000 7fd35a81a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3540cd6c0 0x7fd35415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.148+0000 7fd35a81a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.148+0000 7fd35a81a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3541f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.148+0000 7fd35a81a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541f1b00 con 0x7fd35415f7e0 2023-10-11T06:13:45.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.148+0000 7fd35a81a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35415d990 con 0x7fd3540cd6c0 2023-10-11T06:13:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.148+0000 7fd35a81a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35415dc70 con 0x7fd3540d3000 2023-10-11T06:13:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3541f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3541f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52888/0 (socket says 172.21.15.84:52888) 2023-10-11T06:13:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 -- 172.21.15.84:0/1727713820 learned_addr learned my addr 172.21.15.84:0/1727713820 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:45.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd353fff640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3540cd6c0 0x7fd35415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd34bfff640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 -- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 msgr2=0x7fd35415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 -- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3540cd6c0 msgr2=0x7fd35415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3540cd6c0 0x7fd35415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 -- 172.21.15.84:0/1727713820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440094d0 con 0x7fd35415f7e0 2023-10-11T06:13:45.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.149+0000 7fd358db7640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3541f15c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00cef0 tx=0x7fd34c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:45.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.150+0000 7fd3517fa640 1 -- 172.21.15.84:0/1727713820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00d840 con 0x7fd35415f7e0 2023-10-11T06:13:45.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.150+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35415df50 con 0x7fd35415f7e0 2023-10-11T06:13:45.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.150+0000 7fd3517fa640 1 -- 172.21.15.84:0/1727713820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c00d9c0 con 0x7fd35415f7e0 2023-10-11T06:13:45.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.150+0000 7fd3517fa640 1 -- 172.21.15.84:0/1727713820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00dd70 con 0x7fd35415f7e0 2023-10-11T06:13:45.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.150+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35415e510 con 0x7fd35415f7e0 2023-10-11T06:13:45.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.151+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35415e730 con 0x7fd35415f7e0 2023-10-11T06:13:45.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.152+0000 7fd3517fa640 1 -- 172.21.15.84:0/1727713820 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd34c01e3b0 con 0x7fd35415f7e0 2023-10-11T06:13:45.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.153+0000 7fd3517fa640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd33406a830 0x7fd33406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:45.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.153+0000 7fd3517fa640 1 -- 172.21.15.84:0/1727713820 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fd34c089900 con 0x7fd35415f7e0 2023-10-11T06:13:45.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.154+0000 7fd353fff640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd33406a830 0x7fd33406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:45.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.154+0000 7fd353fff640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd33406a830 0x7fd33406cce0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd344036000 tx=0x7fd34403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:45.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.158+0000 7fd3517fa640 1 -- 172.21.15.84:0/1727713820 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd34c054d80 con 0x7fd35415f7e0 2023-10-11T06:13:45.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.352+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd354073ed0 con 0x7fd35415f7e0 2023-10-11T06:13:45.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.917+0000 7fd3517fa640 1 -- 172.21.15.84:0/1727713820 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd34c01a020 con 0x7fd35415f7e0 2023-10-11T06:13:45.920 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-10-11T06:13:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.924+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd33406a830 msgr2=0x7fd33406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:45.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.924+0000 7fd35a81a640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd33406a830 0x7fd33406cce0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd344036000 tx=0x7fd34403c040 comp rx=0 tx=0).stop 2023-10-11T06:13:45.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.924+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 msgr2=0x7fd3541f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:45.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.924+0000 7fd35a81a640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3541f15c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00cef0 tx=0x7fd34c00a710 comp rx=0 tx=0).stop 2023-10-11T06:13:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.930+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 shutdown_connections 2023-10-11T06:13:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.930+0000 7fd35a81a640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd33406a830 0x7fd33406cce0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.930+0000 7fd35a81a640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd35415f7e0 0x7fd3541f15c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.930+0000 7fd35a81a640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3540d3000 0x7fd35415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.930+0000 7fd35a81a640 1 --2- 172.21.15.84:0/1727713820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3540cd6c0 0x7fd35415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:45.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.930+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 >> 172.21.15.84:0/1727713820 conn(0x7fd3540713a0 msgr2=0x7fd3540c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:45.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.931+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 shutdown_connections 2023-10-11T06:13:45.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:45.931+0000 7fd35a81a640 1 -- 172.21.15.84:0/1727713820 wait complete. 2023-10-11T06:13:45.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-11T06:13:46.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.146+0000 7f5d53fff640 1 Processor -- start 2023-10-11T06:13:46.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.147+0000 7f5d53fff640 1 -- start start 2023-10-11T06:13:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d53fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d540b1c00 0x7f5d54160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d53fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d540d0520 0x7f5d540d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d53fff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5415fe30 con 0x7f5d540b1830 2023-10-11T06:13:46.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d53fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540d0e70 con 0x7f5d540b1c00 2023-10-11T06:13:46.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d53fff640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540b4960 con 0x7f5d540d0520 2023-10-11T06:13:46.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d527fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d540b1c00 0x7f5d54160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d527fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d540b1c00 0x7f5d54160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52894/0 (socket says 172.21.15.84:52894) 2023-10-11T06:13:46.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.148+0000 7f5d527fc640 1 -- 172.21.15.84:0/1089934889 learned_addr learned my addr 172.21.15.84:0/1089934889 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:46.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.149+0000 7f5d537fe640 1 --2- 172.21.15.84:0/1089934889 >> v2:172.21.15.99:3300/0 conn(0x7f5d540d0520 0x7f5d540d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.149+0000 7f5d527fc640 1 -- 172.21.15.84:0/1089934889 >> v2:172.21.15.99:3300/0 conn(0x7f5d540d0520 msgr2=0x7f5d540d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.149+0000 7f5d527fc640 1 --2- 172.21.15.84:0/1089934889 >> v2:172.21.15.99:3300/0 conn(0x7f5d540d0520 0x7f5d540d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.149+0000 7f5d527fc640 1 -- 172.21.15.84:0/1089934889 >> v1:172.21.15.99:6789/0 conn(0x7f5d540b1830 legacy=0x7f5d540b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:46.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.149+0000 7f5d527fc640 1 -- 172.21.15.84:0/1089934889 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540b4c30 con 0x7f5d540b1c00 2023-10-11T06:13:46.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.150+0000 7f5d527fc640 1 --2- 172.21.15.84:0/1089934889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d54160880 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5d540b20b0 tx=0x7f5d4402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b9440eb3652a9a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:46.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.150+0000 7f5d51ffb640 1 -- 172.21.15.84:0/1089934889 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4402c9f0 con 0x7f5d540b1c00 2023-10-11T06:13:46.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.150+0000 7f5d51ffb640 1 -- 172.21.15.84:0/1089934889 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4402cb50 con 0x7f5d540b1c00 2023-10-11T06:13:46.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.150+0000 7f5d51ffb640 1 -- 172.21.15.84:0/1089934889 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d440343f0 con 0x7f5d540b1c00 2023-10-11T06:13:46.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 -- 172.21.15.84:0/1089934889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 msgr2=0x7f5d54160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 --2- 172.21.15.84:0/1089934889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d54160880 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5d540b20b0 tx=0x7f5d4402be50 comp rx=0 tx=0).stop 2023-10-11T06:13:46.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 -- 172.21.15.84:0/1089934889 shutdown_connections 2023-10-11T06:13:46.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 --2- 172.21.15.84:0/1089934889 >> v2:172.21.15.99:3300/0 conn(0x7f5d540d0520 0x7f5d540d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 --2- 172.21.15.84:0/1089934889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d54160880 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 -- 172.21.15.84:0/1089934889 >> 172.21.15.84:0/1089934889 conn(0x7f5d54071cc0 msgr2=0x7f5d5409bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:46.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 -- 172.21.15.84:0/1089934889 shutdown_connections 2023-10-11T06:13:46.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.151+0000 7f5d53fff640 1 -- 172.21.15.84:0/1089934889 wait complete. 2023-10-11T06:13:46.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.153+0000 7f5d53fff640 1 Processor -- start 2023-10-11T06:13:46.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.153+0000 7f5d53fff640 1 -- start start 2023-10-11T06:13:46.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d53fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d540b1830 0x7f5d540c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d53fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d541505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d53fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d540d0520 0x7f5d540b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d53fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d54150b10 con 0x7f5d540b1c00 2023-10-11T06:13:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d53fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541f2960 con 0x7f5d540b1830 2023-10-11T06:13:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d53fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541f1de0 con 0x7f5d540d0520 2023-10-11T06:13:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d527fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d541505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d527fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d541505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52900/0 (socket says 172.21.15.84:52900) 2023-10-11T06:13:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.154+0000 7f5d527fc640 1 -- 172.21.15.84:0/469971332 learned_addr learned my addr 172.21.15.84:0/469971332 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d527fc640 1 -- 172.21.15.84:0/469971332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d540d0520 msgr2=0x7f5d540b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d527fc640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d540d0520 0x7f5d540b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d527fc640 1 -- 172.21.15.84:0/469971332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d540b1830 msgr2=0x7f5d540c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d52ffd640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d540b1830 0x7f5d540c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d527fc640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d540b1830 0x7f5d540c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d527fc640 1 -- 172.21.15.84:0/469971332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44009b30 con 0x7f5d540b1c00 2023-10-11T06:13:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d527fc640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d541505d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5d44000f80 tx=0x7f5d44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d4b7fe640 1 -- 172.21.15.84:0/469971332 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44009510 con 0x7f5d540b1c00 2023-10-11T06:13:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d4b7fe640 1 -- 172.21.15.84:0/469971332 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4403e450 con 0x7f5d540b1c00 2023-10-11T06:13:46.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d4b7fe640 1 -- 172.21.15.84:0/469971332 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4403e800 con 0x7f5d540b1c00 2023-10-11T06:13:46.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.155+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d541f20c0 con 0x7f5d540b1c00 2023-10-11T06:13:46.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.156+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d541f2680 con 0x7f5d540b1c00 2023-10-11T06:13:46.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.157+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5415fb70 con 0x7f5d540b1c00 2023-10-11T06:13:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.158+0000 7f5d4b7fe640 1 -- 172.21.15.84:0/469971332 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5d4403e960 con 0x7f5d540b1c00 2023-10-11T06:13:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.159+0000 7f5d4b7fe640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d3406a830 0x7f5d3406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.159+0000 7f5d4b7fe640 1 -- 172.21.15.84:0/469971332 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f5d440abd50 con 0x7f5d540b1c00 2023-10-11T06:13:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.159+0000 7f5d52ffd640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d3406a830 0x7f5d3406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.160+0000 7f5d52ffd640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d3406a830 0x7f5d3406cce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00a710 tx=0x7f5d3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.164+0000 7f5d4b7fe640 1 -- 172.21.15.84:0/469971332 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5d44077020 con 0x7f5d540b1c00 2023-10-11T06:13:46.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.387+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5d5404e890 con 0x7f5d540b1c00 2023-10-11T06:13:46.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.398+0000 7f5d4b7fe640 1 -- 172.21.15.84:0/469971332 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5d4407a640 con 0x7f5d540b1c00 2023-10-11T06:13:46.400 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 8 and data pool 9 2023-10-11T06:13:46.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.403+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d3406a830 msgr2=0x7f5d3406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.403+0000 7f5d53fff640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d3406a830 0x7f5d3406cce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00a710 tx=0x7f5d3c008040 comp rx=0 tx=0).stop 2023-10-11T06:13:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.403+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 msgr2=0x7f5d541505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.403+0000 7f5d53fff640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d541505d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5d44000f80 tx=0x7f5d44004a40 comp rx=0 tx=0).stop 2023-10-11T06:13:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.408+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 shutdown_connections 2023-10-11T06:13:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.408+0000 7f5d53fff640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d3406a830 0x7f5d3406cce0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.408+0000 7f5d53fff640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d540d0520 0x7f5d540b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.408+0000 7f5d53fff640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d540b1c00 0x7f5d541505d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.408+0000 7f5d53fff640 1 --2- 172.21.15.84:0/469971332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d540b1830 0x7f5d540c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.408+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 >> 172.21.15.84:0/469971332 conn(0x7f5d54071cc0 msgr2=0x7f5d5409bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:46.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.411+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 shutdown_connections 2023-10-11T06:13:46.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.413+0000 7f5d53fff640 1 -- 172.21.15.84:0/469971332 wait complete. 2023-10-11T06:13:46.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:46.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.615+0000 7f965645f640 1 Processor -- start 2023-10-11T06:13:46.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.615+0000 7f965645f640 1 -- start start 2023-10-11T06:13:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.616+0000 7f965645f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f965015c290 0x7f96500af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.616+0000 7f965645f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96500d25e0 0x7f96500d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.616+0000 7f965645f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96500d0a50 0x7f965006b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f965645f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96500d0eb0 con 0x7f96500d25e0 2023-10-11T06:13:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f965645f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96500d2a30 con 0x7f96500d0a50 2023-10-11T06:13:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f965645f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965006cd10 con 0x7f965015c290 2023-10-11T06:13:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96500d25e0 0x7f96500d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f96500d25e0 0x7f96500d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52904/0 (socket says 172.21.15.84:52904) 2023-10-11T06:13:46.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 -- 172.21.15.84:0/2146938527 learned_addr learned my addr 172.21.15.84:0/2146938527 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 -- 172.21.15.84:0/2146938527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f965015c290 msgr2=0x7f96500af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 --2- 172.21.15.84:0/2146938527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f965015c290 0x7f96500af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 -- 172.21.15.84:0/2146938527 >> v2:172.21.15.99:3300/0 conn(0x7f96500d0a50 msgr2=0x7f965006b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:46.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 --2- 172.21.15.84:0/2146938527 >> v2:172.21.15.99:3300/0 conn(0x7f96500d0a50 0x7f965006b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.617+0000 7f9647fff640 1 -- 172.21.15.84:0/2146938527 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96500afb60 con 0x7f96500d25e0 2023-10-11T06:13:46.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.618+0000 7f9647fff640 1 --2- 172.21.15.84:0/2146938527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96500d25e0 0x7f96500d0510 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f965006a8a0 tx=0x7f963802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce99ac7e5bc7bb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:46.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f964f7fe640 1 -- 172.21.15.84:0/2146938527 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96380317e0 con 0x7f96500d25e0 2023-10-11T06:13:46.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f964f7fe640 1 -- 172.21.15.84:0/2146938527 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638031940 con 0x7f96500d25e0 2023-10-11T06:13:46.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f964f7fe640 1 -- 172.21.15.84:0/2146938527 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638031cd0 con 0x7f96500d25e0 2023-10-11T06:13:46.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 -- 172.21.15.84:0/2146938527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96500d25e0 msgr2=0x7f96500d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 --2- 172.21.15.84:0/2146938527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96500d25e0 0x7f96500d0510 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f965006a8a0 tx=0x7f963802b880 comp rx=0 tx=0).stop 2023-10-11T06:13:46.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 -- 172.21.15.84:0/2146938527 shutdown_connections 2023-10-11T06:13:46.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 --2- 172.21.15.84:0/2146938527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96500d25e0 0x7f96500d0510 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 --2- 172.21.15.84:0/2146938527 >> v2:172.21.15.99:3300/0 conn(0x7f96500d0a50 0x7f965006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 --2- 172.21.15.84:0/2146938527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f965015c290 0x7f96500af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 -- 172.21.15.84:0/2146938527 >> 172.21.15.84:0/2146938527 conn(0x7f9650071cc0 msgr2=0x7f965009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:46.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 -- 172.21.15.84:0/2146938527 shutdown_connections 2023-10-11T06:13:46.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.619+0000 7f965645f640 1 -- 172.21.15.84:0/2146938527 wait complete. 2023-10-11T06:13:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.621+0000 7f965645f640 1 Processor -- start 2023-10-11T06:13:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.621+0000 7f965645f640 1 -- start start 2023-10-11T06:13:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.622+0000 7f965645f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96500d0a50 0x7f9650158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.622+0000 7f965645f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96500d25e0 0x7f965009ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.622+0000 7f965645f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f965015c290 0x7f965009f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.622+0000 7f965645f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965009cae0 con 0x7f965015c290 2023-10-11T06:13:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.622+0000 7f965645f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96500c8260 con 0x7f96500d25e0 2023-10-11T06:13:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.622+0000 7f965645f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96500c8500 con 0x7f96500d0a50 2023-10-11T06:13:46.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.622+0000 7f96549fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f965015c290 0x7f965009f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f965015c290 0x7f965009f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52916/0 (socket says 172.21.15.84:52916) 2023-10-11T06:13:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 -- 172.21.15.84:0/4113621759 learned_addr learned my addr 172.21.15.84:0/4113621759 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f9647fff640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96500d25e0 0x7f965009ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f964ffff640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96500d0a50 0x7f9650158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 -- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96500d0a50 msgr2=0x7f9650158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96500d0a50 0x7f9650158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 -- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96500d25e0 msgr2=0x7f965009ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96500d25e0 0x7f965009ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 -- 172.21.15.84:0/4113621759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380094d0 con 0x7f965015c290 2023-10-11T06:13:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.623+0000 7f96549fc640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f965015c290 0x7f965009f2d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f964800b970 tx=0x7f964800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:46.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.624+0000 7f964d7fa640 1 -- 172.21.15.84:0/4113621759 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800cb40 con 0x7f965015c290 2023-10-11T06:13:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.624+0000 7f964d7fa640 1 -- 172.21.15.84:0/4113621759 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964800ccc0 con 0x7f965015c290 2023-10-11T06:13:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.624+0000 7f964d7fa640 1 -- 172.21.15.84:0/4113621759 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648014680 con 0x7f965015c290 2023-10-11T06:13:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.624+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96500c87e0 con 0x7f965015c290 2023-10-11T06:13:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.624+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96500c8fe0 con 0x7f965015c290 2023-10-11T06:13:46.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.625+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96500c8d00 con 0x7f965015c290 2023-10-11T06:13:46.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.626+0000 7f964d7fa640 1 -- 172.21.15.84:0/4113621759 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f964801d3b0 con 0x7f965015c290 2023-10-11T06:13:46.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.627+0000 7f964d7fa640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f963006a830 0x7f963006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:46.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.627+0000 7f964d7fa640 1 -- 172.21.15.84:0/4113621759 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9648088ee0 con 0x7f965015c290 2023-10-11T06:13:46.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.627+0000 7f964ffff640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f963006a830 0x7f963006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:46.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.628+0000 7f964ffff640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f963006a830 0x7f963006cce0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f964000b810 tx=0x7f9640008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.633+0000 7f964d7fa640 1 -- 172.21.15.84:0/4113621759 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f96480541f0 con 0x7f965015c290 2023-10-11T06:13:46.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.818+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9650073ed0 con 0x7f965015c290 2023-10-11T06:13:46.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.820+0000 7f964d7fa640 1 -- 172.21.15.84:0/4113621759 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17844 (secure 0 0 0) 0x7f9648019020 con 0x7f965015c290 2023-10-11T06:13:46.821 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:46.822 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:46.388657+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:46.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f963006a830 msgr2=0x7f963006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f963006a830 0x7f963006cce0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f964000b810 tx=0x7f9640008040 comp rx=0 tx=0).stop 2023-10-11T06:13:46.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f965015c290 msgr2=0x7f965009f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:46.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f965015c290 0x7f965009f2d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f964800b970 tx=0x7f964800be30 comp rx=0 tx=0).stop 2023-10-11T06:13:46.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 shutdown_connections 2023-10-11T06:13:46.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f963006a830 0x7f963006cce0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f965015c290 0x7f965009f2d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96500d25e0 0x7f965009ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 --2- 172.21.15.84:0/4113621759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96500d0a50 0x7f9650158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 >> 172.21.15.84:0/4113621759 conn(0x7f9650071cc0 msgr2=0x7f96500b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.822+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 shutdown_connections 2023-10-11T06:13:46.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:46.823+0000 7f965645f640 1 -- 172.21.15.84:0/4113621759 wait complete. 2023-10-11T06:13:46.840 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:47.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.040+0000 7f3272918640 1 Processor -- start 2023-10-11T06:13:47.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.040+0000 7f3272918640 1 -- start start 2023-10-11T06:13:47.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3272918640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0adb90 0x7f326c0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3272918640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f326c0cd300 0x7f326c0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3272918640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c0cb960 con 0x7f326c0ce0d0 2023-10-11T06:13:47.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3272918640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c15a340 con 0x7f326c0cd300 2023-10-11T06:13:47.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3272918640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0ce4a0 con 0x7f326c0adb90 2023-10-11T06:13:47.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3270eb5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f326c0cd300 0x7f326c0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3270eb5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f326c0cd300 0x7f326c0cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52928/0 (socket says 172.21.15.84:52928) 2023-10-11T06:13:47.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3270eb5640 1 -- 172.21.15.84:0/1047165357 learned_addr learned my addr 172.21.15.84:0/1047165357 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:47.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f32637fe640 1 --2- 172.21.15.84:0/1047165357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0adb90 0x7f326c0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3270eb5640 1 -- 172.21.15.84:0/1047165357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0adb90 msgr2=0x7f326c0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:47.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3270eb5640 1 --2- 172.21.15.84:0/1047165357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0adb90 0x7f326c0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.042+0000 7f3270eb5640 1 -- 172.21.15.84:0/1047165357 >> v1:172.21.15.99:6789/0 conn(0x7f326c0ce0d0 legacy=0x7f326c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:47.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.043+0000 7f3270eb5640 1 -- 172.21.15.84:0/1047165357 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c1593f0 con 0x7f326c0cd300 2023-10-11T06:13:47.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.043+0000 7f32637fe640 1 --2- 172.21.15.84:0/1047165357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0adb90 0x7f326c0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:47.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.044+0000 7f3270eb5640 1 --2- 172.21.15.84:0/1047165357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0cd300 0x7f326c0cb420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32640098a0 tx=0x7f326402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4265fa4e60918f40 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:47.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.044+0000 7f326b7fe640 1 -- 172.21.15.84:0/1047165357 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264031640 con 0x7f326c0cd300 2023-10-11T06:13:47.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.044+0000 7f326b7fe640 1 -- 172.21.15.84:0/1047165357 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32640317a0 con 0x7f326c0cd300 2023-10-11T06:13:47.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.044+0000 7f326b7fe640 1 -- 172.21.15.84:0/1047165357 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264031b30 con 0x7f326c0cd300 2023-10-11T06:13:47.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 -- 172.21.15.84:0/1047165357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0cd300 msgr2=0x7f326c0cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 --2- 172.21.15.84:0/1047165357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0cd300 0x7f326c0cb420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32640098a0 tx=0x7f326402b8f0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 -- 172.21.15.84:0/1047165357 shutdown_connections 2023-10-11T06:13:47.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 --2- 172.21.15.84:0/1047165357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0cd300 0x7f326c0cb420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 --2- 172.21.15.84:0/1047165357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0adb90 0x7f326c0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 -- 172.21.15.84:0/1047165357 >> 172.21.15.84:0/1047165357 conn(0x7f326c069c70 msgr2=0x7f326c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:47.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 -- 172.21.15.84:0/1047165357 shutdown_connections 2023-10-11T06:13:47.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.045+0000 7f3272918640 1 -- 172.21.15.84:0/1047165357 wait complete. 2023-10-11T06:13:47.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.047+0000 7f3272918640 1 Processor -- start 2023-10-11T06:13:47.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.047+0000 7f3272918640 1 -- start start 2023-10-11T06:13:47.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3272918640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0adb90 0x7f326c0c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3272918640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0cd300 0x7f326c14bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3272918640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0ce0d0 0x7f326c0c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3272918640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c158190 con 0x7f326c0ce0d0 2023-10-11T06:13:47.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3272918640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c14c040 con 0x7f326c0adb90 2023-10-11T06:13:47.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3272918640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0c21f0 con 0x7f326c0cd300 2023-10-11T06:13:47.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0ce0d0 0x7f326c0c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0ce0d0 0x7f326c0c1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52942/0 (socket says 172.21.15.84:52942) 2023-10-11T06:13:47.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 -- 172.21.15.84:0/883493613 learned_addr learned my addr 172.21.15.84:0/883493613 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:47.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f32637fe640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0cd300 0x7f326c14bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f326bfff640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0adb90 0x7f326c0c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 -- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0cd300 msgr2=0x7f326c14bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0cd300 0x7f326c14bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 -- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0adb90 msgr2=0x7f326c0c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0adb90 0x7f326c0c34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f3270eb5640 1 -- 172.21.15.84:0/883493613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640094d0 con 0x7f326c0ce0d0 2023-10-11T06:13:47.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.048+0000 7f326bfff640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0adb90 0x7f326c0c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:47.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.049+0000 7f3270eb5640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0ce0d0 0x7f326c0c1cb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f32640098a0 tx=0x7f32640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:47.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.049+0000 7f32697fa640 1 -- 172.21.15.84:0/883493613 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326403ac60 con 0x7f326c0ce0d0 2023-10-11T06:13:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.049+0000 7f32697fa640 1 -- 172.21.15.84:0/883493613 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326403adc0 con 0x7f326c0ce0d0 2023-10-11T06:13:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.049+0000 7f32697fa640 1 -- 172.21.15.84:0/883493613 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264044630 con 0x7f326c0ce0d0 2023-10-11T06:13:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.049+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c0c2400 con 0x7f326c0ce0d0 2023-10-11T06:13:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.049+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c0c29c0 con 0x7f326c0ce0d0 2023-10-11T06:13:47.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.050+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c073ed0 con 0x7f326c0ce0d0 2023-10-11T06:13:47.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.051+0000 7f32697fa640 1 -- 172.21.15.84:0/883493613 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f326402f050 con 0x7f326c0ce0d0 2023-10-11T06:13:47.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.052+0000 7f32697fa640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f324c06a830 0x7f324c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.052+0000 7f32697fa640 1 -- 172.21.15.84:0/883493613 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32640ab0b0 con 0x7f326c0ce0d0 2023-10-11T06:13:47.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.052+0000 7f326bfff640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f324c06a830 0x7f324c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.053+0000 7f326bfff640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f324c06a830 0x7f324c06cce0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f325c007eb0 tx=0x7f325c0073c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:47.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.056+0000 7f32697fa640 1 -- 172.21.15.84:0/883493613 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3264076320 con 0x7f326c0ce0d0 2023-10-11T06:13:47.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.249+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f326c0ccb90 con 0x7f326c0ce0d0 2023-10-11T06:13:47.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.253+0000 7f32697fa640 1 -- 172.21.15.84:0/883493613 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17844 (secure 0 0 0) 0x7f3264079940 con 0x7f326c0ce0d0 2023-10-11T06:13:47.255 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:47.255 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:46.388657+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:47.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f324c06a830 msgr2=0x7f324c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f324c06a830 0x7f324c06cce0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f325c007eb0 tx=0x7f325c0073c0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0ce0d0 msgr2=0x7f326c0c1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0ce0d0 0x7f326c0c1cb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f32640098a0 tx=0x7f32640046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 shutdown_connections 2023-10-11T06:13:47.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f324c06a830 0x7f324c06cce0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f326c0ce0d0 0x7f326c0c1cb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326c0cd300 0x7f326c14bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 --2- 172.21.15.84:0/883493613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0adb90 0x7f326c0c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 >> 172.21.15.84:0/883493613 conn(0x7f326c069c70 msgr2=0x7f326c06c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:47.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 shutdown_connections 2023-10-11T06:13:47.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.256+0000 7f3272918640 1 -- 172.21.15.84:0/883493613 wait complete. 2023-10-11T06:13:47.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-11T06:13:47.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.490+0000 7f84bef18640 1 Processor -- start 2023-10-11T06:13:47.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.490+0000 7f84bef18640 1 -- start start 2023-10-11T06:13:47.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.492+0000 7f84bef18640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 0x7f84b80c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.492+0000 7f84bef18640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b80c9240 con 0x7f84b815a0b0 2023-10-11T06:13:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.492+0000 7f84bef18640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b8071380 con 0x7f84b80ced30 2023-10-11T06:13:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.492+0000 7f84bef18640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b806e0f0 con 0x7f84b80cb5a0 2023-10-11T06:13:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.492+0000 7f84bccb4640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f84b815a0b0 0x7f84b80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35516/0 (socket says 172.21.15.84:35516) 2023-10-11T06:13:47.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.492+0000 7f84bccb4640 1 -- 172.21.15.84:0/3024426789 learned_addr learned my addr 172.21.15.84:0/3024426789 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:47.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325985028 0 0) 0x7f84ac001110 con 0x7f84b815a0b0 2023-10-11T06:13:47.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84a8003520 con 0x7f84b815a0b0 2023-10-11T06:13:47.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84bd4b5640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 0x7f84b80c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1903699266 0 0) 0x7f84a8003520 con 0x7f84b815a0b0 2023-10-11T06:13:47.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 msgr2=0x7f84b80c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:47.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 0x7f84b80c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 >> v1:172.21.15.99:6789/0 conn(0x7f84b80ced30 legacy=0x7f84b806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b806de10 con 0x7f84b815a0b0 2023-10-11T06:13:47.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f84ac001800 con 0x7f84b815a0b0 2023-10-11T06:13:47.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 >> v1:172.21.15.84:6789/0 conn(0x7f84b815a0b0 legacy=0x7f84b80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 0x7f84a80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8004d00 0x7f84a8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b806e0f0 con 0x7f84a8003f20 2023-10-11T06:13:47.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800bb90 con 0x7f84a8004d00 2023-10-11T06:13:47.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.493+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800bd50 con 0x7f84b80cb5a0 2023-10-11T06:13:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84b7fff640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84bccb4640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 0x7f84a80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84bd4b5640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8004d00 0x7f84a8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84b7fff640 1 -- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 msgr2=0x7f84a80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84b7fff640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 0x7f84a80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84b7fff640 1 -- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8004d00 msgr2=0x7f84a8005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84b7fff640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8004d00 0x7f84a8005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84b7fff640 1 -- 172.21.15.84:0/3024426789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8003520 con 0x7f84a8003f20 2023-10-11T06:13:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.494+0000 7f84bccb4640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b80cb5a0 0x7f84a80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.495+0000 7f84b7fff640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84a80047c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f84b806ad00 tx=0x7f84a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3196cce1806f53d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.495+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40052d0 con 0x7f84a8003f20 2023-10-11T06:13:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a402e050 con 0x7f84a8003f20 2023-10-11T06:13:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84b77fe640 1 -- 172.21.15.84:0/3024426789 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4003830 con 0x7f84a8003f20 2023-10-11T06:13:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84bef18640 1 -- 172.21.15.84:0/3024426789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 msgr2=0x7f84a80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84bef18640 1 --2- 172.21.15.84:0/3024426789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84a80047c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f84b806ad00 tx=0x7f84a80057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84bd4b5640 1 -- 172.21.15.84:0/3024426789 reap_dead start 2023-10-11T06:13:47.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84bef18640 1 -- 172.21.15.84:0/3024426789 shutdown_connections 2023-10-11T06:13:47.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84bef18640 1 -- 172.21.15.84:0/3024426789 >> 172.21.15.84:0/3024426789 conn(0x7f84b8069c70 msgr2=0x7f84b806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:47.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84bef18640 1 -- 172.21.15.84:0/3024426789 shutdown_connections 2023-10-11T06:13:47.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.496+0000 7f84bef18640 1 -- 172.21.15.84:0/3024426789 wait complete. 2023-10-11T06:13:47.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.498+0000 7f84bef18640 1 Processor -- start 2023-10-11T06:13:47.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.498+0000 7f84bef18640 1 -- start start 2023-10-11T06:13:47.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.499+0000 7f84bef18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84b8145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.499+0000 7f84bef18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b8159180 0x7f84b814a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.499+0000 7f84bef18640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b81595f0 0x7f84b81546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.499+0000 7f84bef18640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80cd790 con 0x7f84a8003f20 2023-10-11T06:13:47.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.499+0000 7f84bef18640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80cda70 con 0x7f84b8159180 2023-10-11T06:13:47.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.499+0000 7f84bef18640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80cdd50 con 0x7f84b81595f0 2023-10-11T06:13:47.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.499+0000 7f84bccb4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84b8145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84b8145010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52950/0 (socket says 172.21.15.84:52950) 2023-10-11T06:13:47.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 -- 172.21.15.84:0/1721383283 learned_addr learned my addr 172.21.15.84:0/1721383283 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:47.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bd4b5640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b81595f0 0x7f84b81546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 -- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b81595f0 msgr2=0x7f84b81546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84b7fff640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b8159180 0x7f84b814a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b81595f0 0x7f84b81546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 -- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b8159180 msgr2=0x7f84b814a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:47.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b8159180 0x7f84b814a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:47.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 -- 172.21.15.84:0/1721383283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80ce030 con 0x7f84a8003f20 2023-10-11T06:13:47.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bd4b5640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b81595f0 0x7f84b81546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:47.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.500+0000 7f84bccb4640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84b8145010 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84ac001eb0 tx=0x7f84ac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:47.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.501+0000 7f84b57fa640 1 -- 172.21.15.84:0/1721383283 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0074e0 con 0x7f84a8003f20 2023-10-11T06:13:47.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.501+0000 7f84b57fa640 1 -- 172.21.15.84:0/1721383283 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac007640 con 0x7f84a8003f20 2023-10-11T06:13:47.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.501+0000 7f84b57fa640 1 -- 172.21.15.84:0/1721383283 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0049e0 con 0x7f84a8003f20 2023-10-11T06:13:47.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.501+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b81541b0 con 0x7f84a8003f20 2023-10-11T06:13:47.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.501+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b8157a50 con 0x7f84a8003f20 2023-10-11T06:13:47.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.502+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b8073ed0 con 0x7f84a8003f20 2023-10-11T06:13:47.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.503+0000 7f84b57fa640 1 -- 172.21.15.84:0/1721383283 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f84ac004b40 con 0x7f84a8003f20 2023-10-11T06:13:47.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.504+0000 7f84b57fa640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f849c06a830 0x7f849c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:47.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.504+0000 7f84b57fa640 1 -- 172.21.15.84:0/1721383283 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84ac0887e0 con 0x7f84a8003f20 2023-10-11T06:13:47.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.505+0000 7f84b7fff640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f849c06a830 0x7f849c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:47.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.505+0000 7f84b7fff640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f849c06a830 0x7f849c06cce0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84a4033000 tx=0x7f84a4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:47.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.509+0000 7f84b57fa640 1 -- 172.21.15.84:0/1721383283 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f84ac0539c0 con 0x7f84a8003f20 2023-10-11T06:13:47.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:47.743+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f84b80c9240 con 0x7f84a8003f20 2023-10-11T06:13:48.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.408+0000 7f84b57fa640 1 -- 172.21.15.84:0/1721383283 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f84b80c9240 con 0x7f84a8003f20 2023-10-11T06:13:48.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f849c06a830 msgr2=0x7f849c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:48.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f849c06a830 0x7f849c06cce0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f84a4033000 tx=0x7f84a4009030 comp rx=0 tx=0).stop 2023-10-11T06:13:48.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 msgr2=0x7f84b8145010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:48.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84b8145010 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84ac001eb0 tx=0x7f84ac00dc00 comp rx=0 tx=0).stop 2023-10-11T06:13:48.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 shutdown_connections 2023-10-11T06:13:48.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f849c06a830 0x7f849c06cce0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84b81595f0 0x7f84b81546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84b8159180 0x7f84b814a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 --2- 172.21.15.84:0/1721383283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84a8003f20 0x7f84b8145010 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 >> 172.21.15.84:0/1721383283 conn(0x7f84b8069c70 msgr2=0x7f84b80c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:48.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.412+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 shutdown_connections 2023-10-11T06:13:48.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.413+0000 7f84bef18640 1 -- 172.21.15.84:0/1721383283 wait complete. 2023-10-11T06:13:48.432 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-11T06:13:48.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-11T06:13:48.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.650+0000 7fd1b44b4640 1 Processor -- start 2023-10-11T06:13:48.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.650+0000 7fd1b44b4640 1 -- start start 2023-10-11T06:13:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.652+0000 7fd1b44b4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd1ac14ff30 0x7fd1ac0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.652+0000 7fd1b44b4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0c9b90 con 0x7fd1ac0ce510 2023-10-11T06:13:48.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.652+0000 7fd1b44b4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac15da10 con 0x7fd1ac150300 2023-10-11T06:13:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.652+0000 7fd1b44b4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac0cc990 con 0x7fd1ac14ff30 2023-10-11T06:13:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.652+0000 7fd1b2a51640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd1ac0ce510 0x7fd1ac0ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35518/0 (socket says 172.21.15.84:35518) 2023-10-11T06:13:48.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.652+0000 7fd1b2a51640 1 -- 172.21.15.84:0/3258426365 learned_addr learned my addr 172.21.15.84:0/3258426365 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.652+0000 7fd1b2250640 1 --2- 172.21.15.84:0/3258426365 >> v2:172.21.15.99:3300/0 conn(0x7fd1ac14ff30 0x7fd1ac0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:48.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119959762 0 0) 0x7fd1ac0c9b90 con 0x7fd1ac0ce510 2023-10-11T06:13:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd198003520 con 0x7fd1ac0ce510 2023-10-11T06:13:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3048988853 0 0) 0x7fd198003520 con 0x7fd1ac0ce510 2023-10-11T06:13:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 >> v2:172.21.15.99:3300/0 conn(0x7fd1ac14ff30 msgr2=0x7fd1ac0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 --2- 172.21.15.84:0/3258426365 >> v2:172.21.15.99:3300/0 conn(0x7fd1ac14ff30 0x7fd1ac0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 >> v1:172.21.15.99:6789/0 conn(0x7fd1ac150300 legacy=0x7fd1ac0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac15ce00 con 0x7fd1ac0ce510 2023-10-11T06:13:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fd1a40019f0 con 0x7fd1ac0ce510 2023-10-11T06:13:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 >> v1:172.21.15.84:6789/0 conn(0x7fd1ac0ce510 legacy=0x7fd1ac0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198003970 0x7fd198003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1980047c0 0x7fd198004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd198005170 0x7fd19800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0cc990 con 0x7fd198005170 2023-10-11T06:13:48.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19800e290 con 0x7fd198003970 2023-10-11T06:13:48.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.653+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19800e450 con 0x7fd1980047c0 2023-10-11T06:13:48.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2a51640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd198005170 0x7fd19800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:48.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2250640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198003970 0x7fd198003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:48.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2a51640 1 -- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1980047c0 msgr2=0x7fd198004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:48.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b1a4f640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1980047c0 0x7fd198004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:48.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2a51640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1980047c0 0x7fd198004c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2a51640 1 -- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198003970 msgr2=0x7fd198003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:48.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2a51640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198003970 0x7fd198003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2a51640 1 -- 172.21.15.84:0/3258426365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd198003520 con 0x7fd198005170 2023-10-11T06:13:48.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.654+0000 7fd1b2250640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198003970 0x7fd198003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:48.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.655+0000 7fd1b2a51640 1 -- 172.21.15.84:0/3258426365 reap_dead start 2023-10-11T06:13:48.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.655+0000 7fd1b2a51640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd198005170 0x7fd19800db90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400a820 tx=0x7fd1980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d1187937f7a917 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:48.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.655+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a40072d0 con 0x7fd198005170 2023-10-11T06:13:48.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.655+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a402f070 con 0x7fd198005170 2023-10-11T06:13:48.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.655+0000 7fd1b124e640 1 -- 172.21.15.84:0/3258426365 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a40076b0 con 0x7fd198005170 2023-10-11T06:13:48.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.656+0000 7fd1b44b4640 1 -- 172.21.15.84:0/3258426365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd198005170 msgr2=0x7fd19800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:48.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.656+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd198005170 0x7fd19800db90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400a820 tx=0x7fd1980057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.656+0000 7fd1b44b4640 1 -- 172.21.15.84:0/3258426365 shutdown_connections 2023-10-11T06:13:48.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.656+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/3258426365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd198005170 0x7fd19800db90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.656+0000 7fd1b44b4640 1 -- 172.21.15.84:0/3258426365 >> 172.21.15.84:0/3258426365 conn(0x7fd1ac069c70 msgr2=0x7fd1ac06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:48.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.656+0000 7fd1b44b4640 1 -- 172.21.15.84:0/3258426365 shutdown_connections 2023-10-11T06:13:48.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.656+0000 7fd1b44b4640 1 -- 172.21.15.84:0/3258426365 wait complete. 2023-10-11T06:13:48.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.658+0000 7fd1b44b4640 1 Processor -- start 2023-10-11T06:13:48.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.658+0000 7fd1b44b4640 1 -- start start 2023-10-11T06:13:48.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b44b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198005170 0x7fd1ac1ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b44b4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1ac158710 0x7fd1ac156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b44b4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ac1579b0 0x7fd1ac157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b44b4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac158370 con 0x7fd1ac1579b0 2023-10-11T06:13:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b44b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac14b2c0 con 0x7fd198005170 2023-10-11T06:13:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b44b4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac14b5a0 con 0x7fd1ac158710 2023-10-11T06:13:48.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b2a51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ac1579b0 0x7fd1ac157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b2a51640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ac1579b0 0x7fd1ac157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52962/0 (socket says 172.21.15.84:52962) 2023-10-11T06:13:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.659+0000 7fd1b2a51640 1 -- 172.21.15.84:0/2330832977 learned_addr learned my addr 172.21.15.84:0/2330832977 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b2250640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198005170 0x7fd1ac1ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b2a51640 1 -- 172.21.15.84:0/2330832977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1ac158710 msgr2=0x7fd1ac156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b2a51640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1ac158710 0x7fd1ac156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b2a51640 1 -- 172.21.15.84:0/2330832977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198005170 msgr2=0x7fd1ac1ecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b2a51640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198005170 0x7fd1ac1ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b2a51640 1 -- 172.21.15.84:0/2330832977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac14b880 con 0x7fd1ac1579b0 2023-10-11T06:13:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b2a51640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ac1579b0 0x7fd1ac157e30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000f80 tx=0x7fd1ac159670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1a2ffd640 1 -- 172.21.15.84:0/2330832977 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4004d60 con 0x7fd1ac1579b0 2023-10-11T06:13:48.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1a2ffd640 1 -- 172.21.15.84:0/2330832977 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a4001730 con 0x7fd1ac1579b0 2023-10-11T06:13:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac14bb60 con 0x7fd1ac1579b0 2023-10-11T06:13:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1a2ffd640 1 -- 172.21.15.84:0/2330832977 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4001ae0 con 0x7fd1ac1579b0 2023-10-11T06:13:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.660+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac1569b0 con 0x7fd1ac1579b0 2023-10-11T06:13:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.662+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac14bda0 con 0x7fd1ac1579b0 2023-10-11T06:13:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.662+0000 7fd1a2ffd640 1 -- 172.21.15.84:0/2330832977 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fd1a4031030 con 0x7fd1ac1579b0 2023-10-11T06:13:48.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.663+0000 7fd1a2ffd640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd19006a860 0x7fd19006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:48.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.663+0000 7fd1a2ffd640 1 -- 172.21.15.84:0/2330832977 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1a40aabc0 con 0x7fd1ac1579b0 2023-10-11T06:13:48.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.664+0000 7fd1b2250640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd19006a860 0x7fd19006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:48.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.664+0000 7fd1b2250640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd19006a860 0x7fd19006cd10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80099c0 tx=0x7fd1a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:48.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.668+0000 7fd1a2ffd640 1 -- 172.21.15.84:0/2330832977 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fd1ac14bda0 con 0x7fd1ac1579b0 2023-10-11T06:13:48.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:48.889+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd1ac14bda0 con 0x7fd1ac1579b0 2023-10-11T06:13:49.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.420+0000 7fd1a2ffd640 1 -- 172.21.15.84:0/2330832977 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1ac14bda0 con 0x7fd1ac1579b0 2023-10-11T06:13:49.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd19006a860 msgr2=0x7fd19006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd19006a860 0x7fd19006cd10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80099c0 tx=0x7fd1a8008040 comp rx=0 tx=0).stop 2023-10-11T06:13:49.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ac1579b0 msgr2=0x7fd1ac157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ac1579b0 0x7fd1ac157e30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000f80 tx=0x7fd1ac159670 comp rx=0 tx=0).stop 2023-10-11T06:13:49.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 shutdown_connections 2023-10-11T06:13:49.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fd19006a860 0x7fd19006cd10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ac1579b0 0x7fd1ac157e30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1ac158710 0x7fd1ac156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 --2- 172.21.15.84:0/2330832977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd198005170 0x7fd1ac1ecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 >> 172.21.15.84:0/2330832977 conn(0x7fd1ac069c70 msgr2=0x7fd1ac14e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:49.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 shutdown_connections 2023-10-11T06:13:49.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.423+0000 7fd1b44b4640 1 -- 172.21.15.84:0/2330832977 wait complete. 2023-10-11T06:13:49.443 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-11T06:13:49.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:13:49.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.650+0000 7f192cc3a640 1 Processor -- start 2023-10-11T06:13:49.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.650+0000 7f192cc3a640 1 -- start start 2023-10-11T06:13:49.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f192cc3a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19280d0c20 0x7f19280af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:49.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f192cc3a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f19280d25e0 0x7f19280b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:49.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f192cc3a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280d2a40 con 0x7f192815c290 2023-10-11T06:13:49.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f192cc3a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192806cd10 con 0x7f19280d0c20 2023-10-11T06:13:49.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f192cc3a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19280afb60 con 0x7f19280d25e0 2023-10-11T06:13:49.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1925d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19280d0c20 0x7f19280af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:49.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1925d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f19280d0c20 0x7f19280af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54940/0 (socket says 172.21.15.84:54940) 2023-10-11T06:13:49.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1925d9b640 1 -- 172.21.15.84:0/1786050837 learned_addr learned my addr 172.21.15.84:0/1786050837 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:49.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1926d9d640 1 --2- 172.21.15.84:0/1786050837 >> v2:172.21.15.99:3300/0 conn(0x7f19280d25e0 0x7f19280b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:49.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1925d9b640 1 -- 172.21.15.84:0/1786050837 >> v2:172.21.15.99:3300/0 conn(0x7f19280d25e0 msgr2=0x7f19280b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:49.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1925d9b640 1 --2- 172.21.15.84:0/1786050837 >> v2:172.21.15.99:3300/0 conn(0x7f19280d25e0 0x7f19280b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1925d9b640 1 -- 172.21.15.84:0/1786050837 >> v1:172.21.15.84:6789/0 conn(0x7f192815c290 legacy=0x7f19280d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.652+0000 7f1925d9b640 1 -- 172.21.15.84:0/1786050837 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280afe40 con 0x7f19280d0c20 2023-10-11T06:13:49.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.653+0000 7f1926d9d640 1 --2- 172.21.15.84:0/1786050837 >> v2:172.21.15.99:3300/0 conn(0x7f19280d25e0 0x7f19280b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:49.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.654+0000 7f1925d9b640 1 --2- 172.21.15.84:0/1786050837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19280d0c20 0x7f19280af4d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f19280d18b0 tx=0x7f191002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c5710b27ee5985 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:49.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.654+0000 7f192559a640 1 -- 172.21.15.84:0/1786050837 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191002ca80 con 0x7f19280d0c20 2023-10-11T06:13:49.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.654+0000 7f192559a640 1 -- 172.21.15.84:0/1786050837 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191002cbe0 con 0x7f19280d0c20 2023-10-11T06:13:49.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.654+0000 7f192559a640 1 -- 172.21.15.84:0/1786050837 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100343f0 con 0x7f19280d0c20 2023-10-11T06:13:49.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.654+0000 7f192cc3a640 1 -- 172.21.15.84:0/1786050837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19280d0c20 msgr2=0x7f19280af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.654+0000 7f192cc3a640 1 --2- 172.21.15.84:0/1786050837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19280d0c20 0x7f19280af4d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f19280d18b0 tx=0x7f191002bee0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.655+0000 7f192cc3a640 1 -- 172.21.15.84:0/1786050837 shutdown_connections 2023-10-11T06:13:49.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.655+0000 7f192cc3a640 1 --2- 172.21.15.84:0/1786050837 >> v2:172.21.15.99:3300/0 conn(0x7f19280d25e0 0x7f19280b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.655+0000 7f192cc3a640 1 --2- 172.21.15.84:0/1786050837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19280d0c20 0x7f19280af4d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.655+0000 7f192cc3a640 1 -- 172.21.15.84:0/1786050837 >> 172.21.15.84:0/1786050837 conn(0x7f1928071cc0 msgr2=0x7f192809bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:49.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.655+0000 7f192cc3a640 1 -- 172.21.15.84:0/1786050837 shutdown_connections 2023-10-11T06:13:49.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.655+0000 7f192cc3a640 1 -- 172.21.15.84:0/1786050837 wait complete. 2023-10-11T06:13:49.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.656+0000 7f192cc3a640 1 Processor -- start 2023-10-11T06:13:49.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.657+0000 7f192cc3a640 1 -- start start 2023-10-11T06:13:49.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f192cc3a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19280d0c20 0x7f19281f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:49.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f192cc3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19280d25e0 0x7f19280cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:49.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f192cc3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192815c290 0x7f19280b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:49.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f192cc3a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192809e410 con 0x7f192815c290 2023-10-11T06:13:49.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f192cc3a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192809d7d0 con 0x7f19280d25e0 2023-10-11T06:13:49.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f192cc3a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192809da70 con 0x7f19280d0c20 2023-10-11T06:13:49.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192815c290 0x7f19280b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:49.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192815c290 0x7f19280b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54956/0 (socket says 172.21.15.84:54956) 2023-10-11T06:13:49.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 -- 172.21.15.84:0/4045771190 learned_addr learned my addr 172.21.15.84:0/4045771190 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:49.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1925d9b640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19280d25e0 0x7f19280cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:49.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 -- 172.21.15.84:0/4045771190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19280d0c20 msgr2=0x7f19281f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19280d0c20 0x7f19281f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 -- 172.21.15.84:0/4045771190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19280d25e0 msgr2=0x7f19280cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19280d25e0 0x7f19280cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.658+0000 7f1926d9d640 1 -- 172.21.15.84:0/4045771190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910009b30 con 0x7f192815c290 2023-10-11T06:13:49.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.659+0000 7f1926d9d640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192815c290 0x7f19280b7780 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f192000c7f0 tx=0x7f192000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:49.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.659+0000 7f191effd640 1 -- 172.21.15.84:0/4045771190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920007820 con 0x7f192815c290 2023-10-11T06:13:49.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.659+0000 7f191effd640 1 -- 172.21.15.84:0/4045771190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920007980 con 0x7f192815c290 2023-10-11T06:13:49.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.659+0000 7f191effd640 1 -- 172.21.15.84:0/4045771190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920007d70 con 0x7f192815c290 2023-10-11T06:13:49.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.659+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192809dd50 con 0x7f192815c290 2023-10-11T06:13:49.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.659+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19281f1150 con 0x7f192815c290 2023-10-11T06:13:49.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.660+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19280b0100 con 0x7f192815c290 2023-10-11T06:13:49.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.661+0000 7f191effd640 1 -- 172.21.15.84:0/4045771190 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f19200113f0 con 0x7f192815c290 2023-10-11T06:13:49.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.662+0000 7f191effd640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f190806a830 0x7f190806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:49.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.662+0000 7f191effd640 1 -- 172.21.15.84:0/4045771190 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1920089450 con 0x7f192815c290 2023-10-11T06:13:49.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.662+0000 7f192659c640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f190806a830 0x7f190806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:49.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.663+0000 7f192659c640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f190806a830 0x7f190806cce0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1918000f60 tx=0x7f19180079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:49.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.666+0000 7f191effd640 1 -- 172.21.15.84:0/4045771190 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f19280b0100 con 0x7f192815c290 2023-10-11T06:13:49.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.875+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19280b0100 con 0x7f192815c290 2023-10-11T06:13:49.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.876+0000 7f191effd640 1 -- 172.21.15.84:0/4045771190 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f19280b0100 con 0x7f192815c290 2023-10-11T06:13:49.878 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:49.878 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1835716973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1835716973},{"type":"v1","addr":"172.21.15.99:6833","nonce":1835716973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1415879342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1415879342},{"type":"v1","addr":"172.21.15.99:6835","nonce":1415879342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2122307729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2122307729},{"type":"v1","addr":"172.21.15.84:6836","nonce":2122307729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:13:46.389013+0000","modified":"2023-10-11T06:13:49.406181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6027":{"gid":6027,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2048038445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2048038445},{"type":"v1","addr":"172.21.15.84:6837","nonce":2048038445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T06:13:49.879 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-10-11T06:13:49.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f190806a830 msgr2=0x7f190806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f190806a830 0x7f190806cce0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1918000f60 tx=0x7f19180079b0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192815c290 msgr2=0x7f19280b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:49.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192815c290 0x7f19280b7780 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f192000c7f0 tx=0x7f192000ccb0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 shutdown_connections 2023-10-11T06:13:49.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f190806a830 0x7f190806cce0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f192815c290 0x7f19280b7780 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19280d25e0 0x7f19280cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 --2- 172.21.15.84:0/4045771190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19280d0c20 0x7f19281f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:49.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 >> 172.21.15.84:0/4045771190 conn(0x7f1928071cc0 msgr2=0x7f19280cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:49.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 shutdown_connections 2023-10-11T06:13:49.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:49.878+0000 7f192cc3a640 1 -- 172.21.15.84:0/4045771190 wait complete. 2023-10-11T06:13:49.898 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:50.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.108+0000 7fe715182640 1 Processor -- start 2023-10-11T06:13:50.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.108+0000 7fe715182640 1 -- start start 2023-10-11T06:13:50.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe715182640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7100cd650 0x7fe7100cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe715182640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71015d600 con 0x7fe71006fd80 2023-10-11T06:13:50.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe715182640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7100ce090 con 0x7fe71015d190 2023-10-11T06:13:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe715182640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100ca670 con 0x7fe7100cd650 2023-10-11T06:13:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe70ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7100cd650 0x7fe7100cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe70ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7100cd650 0x7fe7100cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54964/0 (socket says 172.21.15.84:54964) 2023-10-11T06:13:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe70ffff640 1 -- 172.21.15.84:0/1374224637 learned_addr learned my addr 172.21.15.84:0/1374224637 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:50.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe70ffff640 1 -- 172.21.15.84:0/1374224637 >> v1:172.21.15.99:6789/0 conn(0x7fe71015d190 legacy=0x7fe7100c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:50.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe70ffff640 1 -- 172.21.15.84:0/1374224637 >> v1:172.21.15.84:6789/0 conn(0x7fe71006fd80 legacy=0x7fe71006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.110+0000 7fe70ffff640 1 -- 172.21.15.84:0/1374224637 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100ca910 con 0x7fe7100cd650 2023-10-11T06:13:50.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.112+0000 7fe70ffff640 1 --2- 172.21.15.84:0/1374224637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100cd650 0x7fe7100cda20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe700009870 tx=0x7fe70002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5b0a9374bd557b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.112+0000 7fe70effd640 1 -- 172.21.15.84:0/1374224637 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700030860 con 0x7fe7100cd650 2023-10-11T06:13:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.112+0000 7fe70effd640 1 -- 172.21.15.84:0/1374224637 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7000309c0 con 0x7fe7100cd650 2023-10-11T06:13:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.112+0000 7fe70effd640 1 -- 172.21.15.84:0/1374224637 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700030d50 con 0x7fe7100cd650 2023-10-11T06:13:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.112+0000 7fe715182640 1 -- 172.21.15.84:0/1374224637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100cd650 msgr2=0x7fe7100cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.112+0000 7fe715182640 1 --2- 172.21.15.84:0/1374224637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100cd650 0x7fe7100cda20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe700009870 tx=0x7fe70002b9a0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.113+0000 7fe715182640 1 -- 172.21.15.84:0/1374224637 shutdown_connections 2023-10-11T06:13:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.113+0000 7fe715182640 1 --2- 172.21.15.84:0/1374224637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100cd650 0x7fe7100cda20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.113+0000 7fe715182640 1 -- 172.21.15.84:0/1374224637 >> 172.21.15.84:0/1374224637 conn(0x7fe710069c70 msgr2=0x7fe71006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.113+0000 7fe715182640 1 -- 172.21.15.84:0/1374224637 shutdown_connections 2023-10-11T06:13:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.113+0000 7fe715182640 1 -- 172.21.15.84:0/1374224637 wait complete. 2023-10-11T06:13:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.114+0000 7fe715182640 1 Processor -- start 2023-10-11T06:13:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.115+0000 7fe715182640 1 -- start start 2023-10-11T06:13:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.115+0000 7fe715182640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe71006fd80 0x7fe71014d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe715182640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71015d190 0x7fe7100c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe715182640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100c24b0 0x7fe7100c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe715182640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100b1400 con 0x7fe7100c24b0 2023-10-11T06:13:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe715182640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100b16e0 con 0x7fe71015d190 2023-10-11T06:13:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe715182640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100b19c0 con 0x7fe71006fd80 2023-10-11T06:13:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100c24b0 0x7fe7100c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100c24b0 0x7fe7100c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54966/0 (socket says 172.21.15.84:54966) 2023-10-11T06:13:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 -- 172.21.15.84:0/2833605377 learned_addr learned my addr 172.21.15.84:0/2833605377 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe70f7fe640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71015d190 0x7fe7100c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 -- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe71006fd80 msgr2=0x7fe71014d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe71006fd80 0x7fe71014d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 -- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71015d190 msgr2=0x7fe7100c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71015d190 0x7fe7100c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe714981640 1 -- 172.21.15.84:0/2833605377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000094d0 con 0x7fe7100c24b0 2023-10-11T06:13:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.116+0000 7fe70f7fe640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71015d190 0x7fe7100c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.117+0000 7fe714981640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100c24b0 0x7fe7100c2d40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe708001eb0 tx=0x7fe70800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.117+0000 7fe70cff9640 1 -- 172.21.15.84:0/2833605377 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7080052d0 con 0x7fe7100c24b0 2023-10-11T06:13:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.117+0000 7fe70cff9640 1 -- 172.21.15.84:0/2833605377 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708005430 con 0x7fe7100c24b0 2023-10-11T06:13:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.117+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100b1ca0 con 0x7fe7100c24b0 2023-10-11T06:13:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.117+0000 7fe70cff9640 1 -- 172.21.15.84:0/2833605377 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708003810 con 0x7fe7100c24b0 2023-10-11T06:13:50.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.117+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe710150f20 con 0x7fe7100c24b0 2023-10-11T06:13:50.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.118+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7100b21c0 con 0x7fe7100c24b0 2023-10-11T06:13:50.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.119+0000 7fe70cff9640 1 -- 172.21.15.84:0/2833605377 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe708003970 con 0x7fe7100c24b0 2023-10-11T06:13:50.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.120+0000 7fe70cff9640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe6f006a830 0x7fe6f006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.120+0000 7fe70cff9640 1 -- 172.21.15.84:0/2833605377 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe708088500 con 0x7fe7100c24b0 2023-10-11T06:13:50.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.120+0000 7fe70ffff640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe6f006a830 0x7fe6f006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.121+0000 7fe70ffff640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe6f006a830 0x7fe6f006cce0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe700036000 tx=0x7fe70003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:50.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.125+0000 7fe70cff9640 1 -- 172.21.15.84:0/2833605377 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe7100b21c0 con 0x7fe7100c24b0 2023-10-11T06:13:50.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.318+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7100b21c0 con 0x7fe7100c24b0 2023-10-11T06:13:50.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.322+0000 7fe70cff9640 1 -- 172.21.15.84:0/2833605377 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7fe7100b21c0 con 0x7fe7100c24b0 2023-10-11T06:13:50.324 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:50.325 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:50.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe6f006a830 msgr2=0x7fe6f006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe6f006a830 0x7fe6f006cce0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe700036000 tx=0x7fe70003b040 comp rx=0 tx=0).stop 2023-10-11T06:13:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100c24b0 msgr2=0x7fe7100c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100c24b0 0x7fe7100c2d40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe708001eb0 tx=0x7fe70800c890 comp rx=0 tx=0).stop 2023-10-11T06:13:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 shutdown_connections 2023-10-11T06:13:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe6f006a830 0x7fe6f006cce0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7100c24b0 0x7fe7100c2d40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71015d190 0x7fe7100c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 --2- 172.21.15.84:0/2833605377 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe71006fd80 0x7fe71014d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 >> 172.21.15.84:0/2833605377 conn(0x7fe710069c70 msgr2=0x7fe7100cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.324+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 shutdown_connections 2023-10-11T06:13:50.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.325+0000 7fe715182640 1 -- 172.21.15.84:0/2833605377 wait complete. 2023-10-11T06:13:50.346 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-11T06:13:50.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:13:50.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.557+0000 7f43f0e8f640 1 Processor -- start 2023-10-11T06:13:50.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.558+0000 7f43f0e8f640 1 -- start start 2023-10-11T06:13:50.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43f0e8f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43ec15c290 0x7f43ec0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43f0e8f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43ec0d25e0 0x7f43ec0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43f0e8f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d0a50 0x7f43ec06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43f0e8f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0d2a60 con 0x7f43ec0d25e0 2023-10-11T06:13:50.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43f0e8f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0d0e90 con 0x7f43ec15c290 2023-10-11T06:13:50.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43f0e8f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec06cd10 con 0x7f43ec0d0a50 2023-10-11T06:13:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43ec0d25e0 0x7f43ec0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43ec0d25e0 0x7f43ec0d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54972/0 (socket says 172.21.15.84:54972) 2023-10-11T06:13:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.559+0000 7f43e3fff640 1 -- 172.21.15.84:0/3083107332 learned_addr learned my addr 172.21.15.84:0/3083107332 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:50.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43ead9d640 1 --2- 172.21.15.84:0/3083107332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d0a50 0x7f43ec06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43ea59c640 1 --2- 172.21.15.84:0/3083107332 >> v2:172.21.15.99:3300/0 conn(0x7f43ec15c290 0x7f43ec0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43e3fff640 1 -- 172.21.15.84:0/3083107332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d0a50 msgr2=0x7f43ec06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:50.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43e3fff640 1 --2- 172.21.15.84:0/3083107332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d0a50 0x7f43ec06b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43e3fff640 1 -- 172.21.15.84:0/3083107332 >> v2:172.21.15.99:3300/0 conn(0x7f43ec15c290 msgr2=0x7f43ec0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43e3fff640 1 --2- 172.21.15.84:0/3083107332 >> v2:172.21.15.99:3300/0 conn(0x7f43ec15c290 0x7f43ec0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43e3fff640 1 -- 172.21.15.84:0/3083107332 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0afb60 con 0x7f43ec0d25e0 2023-10-11T06:13:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.560+0000 7f43ea59c640 1 --2- 172.21.15.84:0/3083107332 >> v2:172.21.15.99:3300/0 conn(0x7f43ec15c290 0x7f43ec0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.561+0000 7f43e3fff640 1 --2- 172.21.15.84:0/3083107332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec0d25e0 0x7f43ec0d0510 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f43ec072380 tx=0x7f43d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e328a21974b5a304 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.561+0000 7f43e9d9b640 1 -- 172.21.15.84:0/3083107332 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d40317e0 con 0x7f43ec0d25e0 2023-10-11T06:13:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43e9d9b640 1 -- 172.21.15.84:0/3083107332 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d4031940 con 0x7f43ec0d25e0 2023-10-11T06:13:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43e9d9b640 1 -- 172.21.15.84:0/3083107332 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4031cd0 con 0x7f43ec0d25e0 2023-10-11T06:13:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3083107332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec0d25e0 msgr2=0x7f43ec0d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3083107332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec0d25e0 0x7f43ec0d0510 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f43ec072380 tx=0x7f43d402b880 comp rx=0 tx=0).stop 2023-10-11T06:13:50.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3083107332 shutdown_connections 2023-10-11T06:13:50.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3083107332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec0d25e0 0x7f43ec0d0510 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3083107332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d0a50 0x7f43ec06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3083107332 >> v2:172.21.15.99:3300/0 conn(0x7f43ec15c290 0x7f43ec0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3083107332 >> 172.21.15.84:0/3083107332 conn(0x7f43ec071cc0 msgr2=0x7f43ec09bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3083107332 shutdown_connections 2023-10-11T06:13:50.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.562+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3083107332 wait complete. 2023-10-11T06:13:50.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.564+0000 7f43f0e8f640 1 Processor -- start 2023-10-11T06:13:50.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.564+0000 7f43f0e8f640 1 -- start start 2023-10-11T06:13:50.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43f0e8f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec0d0a50 0x7f43ec0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43f0e8f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d25e0 0x7f43ec09c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43f0e8f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec15c290 0x7f43ec0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43f0e8f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec09e880 con 0x7f43ec15c290 2023-10-11T06:13:50.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43f0e8f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec09cb20 con 0x7f43ec0d0a50 2023-10-11T06:13:50.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43f0e8f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec09cda0 con 0x7f43ec0d25e0 2023-10-11T06:13:50.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43ead9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec15c290 0x7f43ec0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43ead9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec15c290 0x7f43ec0b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54978/0 (socket says 172.21.15.84:54978) 2023-10-11T06:13:50.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.565+0000 7f43ead9d640 1 -- 172.21.15.84:0/3486760644 learned_addr learned my addr 172.21.15.84:0/3486760644 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:50.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43ea59c640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec0d0a50 0x7f43ec0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43e3fff640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d25e0 0x7f43ec09c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43ead9d640 1 -- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d25e0 msgr2=0x7f43ec09c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43ead9d640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d25e0 0x7f43ec09c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43ead9d640 1 -- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec0d0a50 msgr2=0x7f43ec0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43ead9d640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec0d0a50 0x7f43ec0c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43ead9d640 1 -- 172.21.15.84:0/3486760644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40094d0 con 0x7f43ec15c290 2023-10-11T06:13:50.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43e3fff640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d25e0 0x7f43ec09c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:50.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43ead9d640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec15c290 0x7f43ec0b2640 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f43e4007e60 tx=0x7f43e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:50.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.566+0000 7f43e2ffd640 1 -- 172.21.15.84:0/3486760644 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4014620 con 0x7f43ec15c290 2023-10-11T06:13:50.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.567+0000 7f43e2ffd640 1 -- 172.21.15.84:0/3486760644 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e400ee90 con 0x7f43ec15c290 2023-10-11T06:13:50.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.567+0000 7f43e2ffd640 1 -- 172.21.15.84:0/3486760644 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40149e0 con 0x7f43ec15c290 2023-10-11T06:13:50.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.567+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec09d080 con 0x7f43ec15c290 2023-10-11T06:13:50.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.567+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec09d640 con 0x7f43ec15c290 2023-10-11T06:13:50.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.568+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec09e9c0 con 0x7f43ec15c290 2023-10-11T06:13:50.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.569+0000 7f43e2ffd640 1 -- 172.21.15.84:0/3486760644 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f43e4014b40 con 0x7f43ec15c290 2023-10-11T06:13:50.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.569+0000 7f43e2ffd640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43cc06a830 0x7f43cc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:50.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.570+0000 7f43e2ffd640 1 -- 172.21.15.84:0/3486760644 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43e40899c0 con 0x7f43ec15c290 2023-10-11T06:13:50.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.570+0000 7f43ea59c640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43cc06a830 0x7f43cc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:50.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.570+0000 7f43ea59c640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43cc06a830 0x7f43cc06cce0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00a950 tx=0x7f43dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:50.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.574+0000 7f43e2ffd640 1 -- 172.21.15.84:0/3486760644 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f43ec09e9c0 con 0x7f43ec15c290 2023-10-11T06:13:50.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.758+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43ec0b0120 con 0x7f43cc06a830 2023-10-11T06:13:50.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.773+0000 7f43e2ffd640 1 -- 172.21.15.84:0/3486760644 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+298279 (secure 0 0 0) 0x7f43dc00c050 con 0x7f43cc06a830 2023-10-11T06:13:50.779 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:50.781 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:13:50.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43cc06a830 msgr2=0x7f43cc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43cc06a830 0x7f43cc06cce0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00a950 tx=0x7f43dc008040 comp rx=0 tx=0).stop 2023-10-11T06:13:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec15c290 msgr2=0x7f43ec0b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec15c290 0x7f43ec0b2640 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f43e4007e60 tx=0x7f43e400e4b0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 shutdown_connections 2023-10-11T06:13:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43cc06a830 0x7f43cc06cce0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43ec15c290 0x7f43ec0b2640 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43ec0d25e0 0x7f43ec09c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 --2- 172.21.15.84:0/3486760644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec0d0a50 0x7f43ec0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.778+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 >> 172.21.15.84:0/3486760644 conn(0x7f43ec071cc0 msgr2=0x7f43ec0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.779+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 shutdown_connections 2023-10-11T06:13:50.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:50.779+0000 7f43f0e8f640 1 -- 172.21.15.84:0/3486760644 wait complete. 2023-10-11T06:13:50.798 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2023-10-11T06:13:48.821999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221416,"kb_used_data":5800,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754753304,"statfs":{"total":773094113280,"available":772867383296,"internally_reserved":0,"allocated":5939200,"data_stored":3218800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000925"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.913123+0000","last_change":"2023-10-11T06:13:44.913123+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.913123+0000","last_undegraded":"2023-10-11T06:13:44.913123+0000","last_fullsized":"2023-10-11T06:13:44.913123+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:10:27.512577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.917012+0000","last_change":"2023-10-11T06:13:44.917012+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.917012+0000","last_undegraded":"2023-10-11T06:13:44.917012+0000","last_fullsized":"2023-10-11T06:13:44.917012+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:35.798111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.913364+0000","last_change":"2023-10-11T06:13:44.913364+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.913364+0000","last_undegraded":"2023-10-11T06:13:44.913364+0000","last_fullsized":"2023-10-11T06:13:44.913364+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:26.580696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.919533+0000","last_change":"2023-10-11T06:13:44.919533+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.919533+0000","last_undegraded":"2023-10-11T06:13:44.919533+0000","last_fullsized":"2023-10-11T06:13:44.919533+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:12:14.685368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.910709+0000","last_change":"2023-10-11T06:13:44.910709+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.910709+0000","last_undegraded":"2023-10-11T06:13:44.910709+0000","last_fullsized":"2023-10-11T06:13:44.910709+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:40:00.524726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.912927+0000","last_change":"2023-10-11T06:13:44.912927+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.912927+0000","last_undegraded":"2023-10-11T06:13:44.912927+0000","last_fullsized":"2023-10-11T06:13:44.912927+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:48.568608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.919541+0000","last_change":"2023-10-11T06:13:44.919541+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.919541+0000","last_undegraded":"2023-10-11T06:13:44.919541+0000","last_fullsized":"2023-10-11T06:13:44.919541+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:12.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.909882+0000","last_change":"2023-10-11T06:13:44.909882+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.909882+0000","last_undegraded":"2023-10-11T06:13:44.909882+0000","last_fullsized":"2023-10-11T06:13:44.909882+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:48.568608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.917510+0000","last_change":"2023-10-11T06:13:44.917510+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.917510+0000","last_undegraded":"2023-10-11T06:13:44.917510+0000","last_fullsized":"2023-10-11T06:13:44.917510+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:25:43.343389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.916079+0000","last_change":"2023-10-11T06:13:44.916079+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.916079+0000","last_undegraded":"2023-10-11T06:13:44.916079+0000","last_fullsized":"2023-10-11T06:13:44.916079+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:26.580696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.918384+0000","last_change":"2023-10-11T06:13:44.918384+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.918384+0000","last_undegraded":"2023-10-11T06:13:44.918384+0000","last_fullsized":"2023-10-11T06:13:44.918384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:52:26.451241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.909909+0000","last_change":"2023-10-11T06:13:44.909909+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.909909+0000","last_undegraded":"2023-10-11T06:13:44.909909+0000","last_fullsized":"2023-10-11T06:13:44.909909+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:43:42.843730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.916518+0000","last_change":"2023-10-11T06:13:44.916518+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.916518+0000","last_undegraded":"2023-10-11T06:13:44.916518+0000","last_fullsized":"2023-10-11T06:13:44.916518+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:10:27.512577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.907120+0000","last_change":"2023-10-11T06:13:44.907120+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.907120+0000","last_undegraded":"2023-10-11T06:13:44.907120+0000","last_fullsized":"2023-10-11T06:13:44.907120+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:19:54.106151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.916799+0000","last_change":"2023-10-11T06:13:44.916799+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.916799+0000","last_undegraded":"2023-10-11T06:13:44.916799+0000","last_fullsized":"2023-10-11T06:13:44.916799+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:10:27.512577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.916799+0000","last_change":"2023-10-11T06:13:44.916799+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.916799+0000","last_undegraded":"2023-10-11T06:13:44.916799+0000","last_fullsized":"2023-10-11T06:13:44.916799+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:26.827270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.920700+0000","last_change":"2023-10-11T06:13:44.920700+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.920700+0000","last_undegraded":"2023-10-11T06:13:44.920700+0000","last_fullsized":"2023-10-11T06:13:44.920700+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:53:25.614701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.912991+0000","last_change":"2023-10-11T06:13:44.912991+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.912991+0000","last_undegraded":"2023-10-11T06:13:44.912991+0000","last_fullsized":"2023-10-11T06:13:44.912991+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:41:13.469887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.906895+0000","last_change":"2023-10-11T06:13:44.906895+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.906895+0000","last_undegraded":"2023-10-11T06:13:44.906895+0000","last_fullsized":"2023-10-11T06:13:44.906895+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:42:47.261508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.917257+0000","last_change":"2023-10-11T06:13:44.917257+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.917257+0000","last_undegraded":"2023-10-11T06:13:44.917257+0000","last_fullsized":"2023-10-11T06:13:44.917257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:41:13.469887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.920148+0000","last_change":"2023-10-11T06:13:44.920148+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.920148+0000","last_undegraded":"2023-10-11T06:13:44.920148+0000","last_fullsized":"2023-10-11T06:13:44.920148+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:50:00.126365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.908186+0000","last_change":"2023-10-11T06:13:44.908186+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.908186+0000","last_undegraded":"2023-10-11T06:13:44.908186+0000","last_fullsized":"2023-10-11T06:13:44.908186+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:30:03.816589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.920179+0000","last_change":"2023-10-11T06:13:44.920179+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.920179+0000","last_undegraded":"2023-10-11T06:13:44.920179+0000","last_fullsized":"2023-10-11T06:13:44.920179+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:07:51.398606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.915448+0000","last_change":"2023-10-11T06:13:44.915448+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.915448+0000","last_undegraded":"2023-10-11T06:13:44.915448+0000","last_fullsized":"2023-10-11T06:13:44.915448+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:50:08.238522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.920454+0000","last_change":"2023-10-11T06:13:44.920454+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.920454+0000","last_undegraded":"2023-10-11T06:13:44.920454+0000","last_fullsized":"2023-10-11T06:13:44.920454+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:35.798111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:42.052270+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:13:42.052270+0000","last_peered":"2023-10-11T06:13:42.052270+0000","last_clean":"2023-10-11T06:13:42.052270+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:13:42.052270+0000","last_undegraded":"2023-10-11T06:13:42.052270+0000","last_fullsized":"2023-10-11T06:13:42.052270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.913013+0000","last_change":"2023-10-11T06:13:44.913013+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.913013+0000","last_undegraded":"2023-10-11T06:13:44.913013+0000","last_fullsized":"2023-10-11T06:13:44.913013+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:12.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:42.053480+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:13:42.053480+0000","last_peered":"2023-10-11T06:13:42.053480+0000","last_clean":"2023-10-11T06:13:42.053480+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:13:42.053480+0000","last_undegraded":"2023-10-11T06:13:42.053480+0000","last_fullsized":"2023-10-11T06:13:42.053480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.915867+0000","last_change":"2023-10-11T06:13:44.915867+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.915867+0000","last_undegraded":"2023-10-11T06:13:44.915867+0000","last_fullsized":"2023-10-11T06:13:44.915867+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:12.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:43.379422+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:13:43.379422+0000","last_peered":"2023-10-11T06:13:43.379422+0000","last_clean":"2023-10-11T06:13:43.379422+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:13:43.379422+0000","last_undegraded":"2023-10-11T06:13:43.379422+0000","last_fullsized":"2023-10-11T06:13:43.379422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.913678+0000","last_change":"2023-10-11T06:13:44.913678+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.913678+0000","last_undegraded":"2023-10-11T06:13:44.913678+0000","last_fullsized":"2023-10-11T06:13:44.913678+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:30:03.816589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:42.053575+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:13:42.053575+0000","last_peered":"2023-10-11T06:13:42.053575+0000","last_clean":"2023-10-11T06:13:42.053575+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:13:42.053575+0000","last_undegraded":"2023-10-11T06:13:42.053575+0000","last_fullsized":"2023-10-11T06:13:42.053575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.921507+0000","last_change":"2023-10-11T06:13:44.921507+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.921507+0000","last_undegraded":"2023-10-11T06:13:44.921507+0000","last_fullsized":"2023-10-11T06:13:44.921507+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:26.580696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:42.544314+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:13:42.544314+0000","last_peered":"2023-10-11T06:13:42.544314+0000","last_clean":"2023-10-11T06:13:42.544314+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:13:42.544314+0000","last_undegraded":"2023-10-11T06:13:42.544314+0000","last_fullsized":"2023-10-11T06:13:42.544314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.915698+0000","last_change":"2023-10-11T06:13:44.915698+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.915698+0000","last_undegraded":"2023-10-11T06:13:44.915698+0000","last_fullsized":"2023-10-11T06:13:44.915698+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:12:14.685368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:42.052489+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:13:42.052489+0000","last_peered":"2023-10-11T06:13:42.052489+0000","last_clean":"2023-10-11T06:13:42.052489+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:13:42.052489+0000","last_undegraded":"2023-10-11T06:13:42.052489+0000","last_fullsized":"2023-10-11T06:13:42.052489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.920163+0000","last_change":"2023-10-11T06:13:44.920163+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.920163+0000","last_undegraded":"2023-10-11T06:13:44.920163+0000","last_fullsized":"2023-10-11T06:13:44.920163+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:28:29.789596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:43.379486+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:13:43.379486+0000","last_peered":"2023-10-11T06:13:43.379486+0000","last_clean":"2023-10-11T06:13:43.379486+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:13:43.379486+0000","last_undegraded":"2023-10-11T06:13:43.379486+0000","last_fullsized":"2023-10-11T06:13:43.379486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-11T06:13:45.900598+0000","last_change":"2023-10-11T06:13:45.900598+0000","last_active":"2023-10-11T06:13:45.900598+0000","last_peered":"2023-10-11T06:13:45.900598+0000","last_clean":"2023-10-11T06:13:45.900598+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:45.900598+0000","last_undegraded":"2023-10-11T06:13:45.900598+0000","last_fullsized":"2023-10-11T06:13:45.900598+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2023-10-11T06:13:44.907085+0000","last_change":"2023-10-11T06:13:44.907085+0000","last_active":"2023-10-11T06:13:44.888736+0000","last_peered":"2023-10-11T06:13:44.888736+0000","last_clean":"2023-10-11T06:13:44.888736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-11T06:13:44.907085+0000","last_undegraded":"2023-10-11T06:13:44.907085+0000","last_fullsized":"2023-10-11T06:13:44.907085+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:53:25.614701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:42.544403+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:13:42.544403+0000","last_peered":"2023-10-11T06:13:42.544403+0000","last_clean":"2023-10-11T06:13:42.544403+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:13:42.544403+0000","last_undegraded":"2023-10-11T06:13:42.544403+0000","last_fullsized":"2023-10-11T06:13:42.544403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-11T06:13:43.379468+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:13:43.379468+0000","last_peered":"2023-10-11T06:13:43.379468+0000","last_clean":"2023-10-11T06:13:43.379468+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:13:43.379468+0000","last_undegraded":"2023-10-11T06:13:43.379468+0000","last_fullsized":"2023-10-11T06:13:43.379468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1187840,"data_stored":845126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.0609999999999999,"15min":1.0409999999999999},"min":{"1min":0.77200000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.466,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.171,"15min":1.161},"min":{"1min":0.83699999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6160000000000001,"5min":1.788,"15min":1.788},"last":0.91200000000000003}]},{"osd":1,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.1020000000000001,"15min":1.105},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6679999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1220000000000001,"15min":1.1279999999999999},"min":{"1min":0.67900000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6930000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1}]},{"osd":2,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.68799999999999994,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4059999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.292},"min":{"1min":0.72799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.758,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.96199999999999997}]},{"osd":3,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.24,"15min":1.248},"min":{"1min":0.86599999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6040000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.874},{"interface":"front","average":{"1min":1.23,"5min":1.2589999999999999,"15min":1.264},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7969999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.018}]},{"osd":4,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4059999999999999,"15min":1.4179999999999999},"min":{"1min":0.86799999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9139999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.385,"5min":1.3129999999999999,"15min":1.3009999999999999},"min":{"1min":0.89600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.8700000000000001,"5min":1.913,"15min":1.913},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.389,"15min":1.395},"min":{"1min":0.86899999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0139999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.446,"5min":1.5680000000000001,"15min":1.5880000000000001},"min":{"1min":1.135,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0019999999999998,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.478,"15min":1.476},"min":{"1min":1.131,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.069,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.747},{"interface":"front","average":{"1min":1.546,"5min":1.5620000000000001,"15min":1.5640000000000001},"min":{"1min":1.0720000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.9830000000000001,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":0.79300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1080000000000001,"15min":1.111},"min":{"1min":0.78600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2090000000000001,"15min":1.2150000000000001},"min":{"1min":0.82699999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.49,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.4730000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.028,"15min":1.034},"min":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.143,"15min":1.161},"min":{"1min":0.70299999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.343,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.341}]},{"osd":2,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0669999999999999,"15min":1.071},"min":{"1min":0.69799999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4039999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.21,"15min":1.2150000000000001},"min":{"1min":0.75600000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5669999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.1890000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.3029999999999999,"15min":1.3260000000000001},"min":{"1min":0.77800000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4750000000000001,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":1.54},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.2509999999999999},"min":{"1min":0.86099999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.526,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.2509999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.359,"15min":1.379},"min":{"1min":0.91100000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.583,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":1.393},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.444,"15min":1.466},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5780000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.6799999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.373,"15min":1.3919999999999999},"min":{"1min":0.89200000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.617,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.413},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":1.1000000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7809999999999999,"5min":1.992,"15min":1.992},"last":1.516}]},{"osd":7,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.5680000000000001,"15min":1.591},"min":{"1min":0.97599999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.659,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.917},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5249999999999999,"15min":1.5429999999999999},"min":{"1min":1.0349999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6890000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.5609999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.997,"15min":0.995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.476,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.01,"15min":1.0149999999999999},"min":{"1min":0.76700000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3620000000000001,"5min":1.621,"15min":1.621},"last":0.92500000000000004}]},{"osd":1,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.048,"15min":1.056},"min":{"1min":0.71099999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.335,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.629},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.68100000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4710000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.236,"15min":1.248},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.55,"5min":2.508,"15min":2.508},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81899999999999995,"5min":0.752,"15min":0.752},"max":{"1min":1.5009999999999999,"5min":2.524,"15min":2.524},"last":1.7689999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.155,"15min":1.153},"min":{"1min":0.74199999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5589999999999999,"5min":2.052,"15min":2.052},"last":1.083},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2849999999999999,"15min":1.3089999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.3819999999999999,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.877}]},{"osd":4,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.276,"15min":1.286},"min":{"1min":0.77200000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.403,"5min":2.242,"15min":2.242},"last":1.325},{"interface":"front","average":{"1min":1.333,"5min":1.3620000000000001,"15min":1.367},"min":{"1min":0.92100000000000004,"5min":0.747,"15min":0.747},"max":{"1min":1.6519999999999999,"5min":2.427,"15min":2.427},"last":1.46}]},{"osd":6,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.401,"15min":1.4319999999999999},"min":{"1min":0.84599999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.514,"5min":2.472,"15min":2.472},"last":1.96},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.45,"15min":1.46},"min":{"1min":0.97599999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7529999999999999,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.6870000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":1.071,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.579,"5min":2.617,"15min":2.617},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5620000000000001,"15min":1.5840000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7110000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":2.04}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":1.0780000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3089999999999999,"5min":1.841,"15min":1.841},"last":1.125},{"interface":"front","average":{"1min":0.995,"5min":0.96299999999999997,"15min":0.95699999999999996},"min":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.413,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.0740000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.071,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.1180000000000001,"15min":1.1319999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.755}]},{"osd":2,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0469999999999999,"15min":1.0529999999999999},"min":{"1min":0.76300000000000001,"5min":0.63,"15min":0.63},"max":{"1min":1.4470000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.91300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.5069999999999999,"5min":1.744,"15min":1.744},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.244,"15min":1.262},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3779999999999999,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.016},{"interface":"front","average":{"1min":1.149,"5min":1.2050000000000001,"15min":1.2150000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5589999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.4390000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.321},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.825,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4039999999999999,"15min":1.4179999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.776,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.1899999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.323,"15min":1.339},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.575,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.45,"15min":1.456},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.6319999999999999,"5min":1.972,"15min":1.972},"last":1.4710000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5309999999999999,"15min":1.5580000000000001},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.069,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.4770000000000001,"15min":1.4990000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.675,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.6679999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1910000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.755},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.96499999999999997,"15min":0.97099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.474,"5min":2.097,"15min":2.097},"last":0.872}]},{"osd":1,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89800000000000002,"15min":0.90900000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1299999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.997,"5min":1.0940000000000001,"15min":1.1100000000000001},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.655,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.04,"15min":1.0489999999999999},"min":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5940000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.05,"5min":1.1839999999999999,"15min":1.206},"min":{"1min":0.47199999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.417,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.94399999999999995}]},{"osd":4,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.216,"15min":1.2310000000000001},"min":{"1min":0.753,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5740000000000001,"5min":3.286,"15min":3.286},"last":1.115},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3,"15min":1.302},"min":{"1min":0.77100000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7889999999999999,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":1.4910000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4159999999999999,"15min":1.427},"min":{"1min":0.86599999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.756,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":0.82699999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6850000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.159}]},{"osd":6,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.385,"15min":1.391},"min":{"1min":0.79200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.113,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.083},{"interface":"front","average":{"1min":1.349,"5min":1.5449999999999999,"15min":1.577},"min":{"1min":0.73599999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9650000000000001,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":1.345}]},{"osd":7,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3799999999999999,"15min":1.391},"min":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9099999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4179999999999999,"15min":1.4199999999999999},"min":{"1min":0.88100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8620000000000001,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.389}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.81499999999999995,"15min":0.83299999999999996},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.77800000000000002,"15min":0.79300000000000004},"min":{"1min":0.36699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.86099999999999999,"15min":0.87},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1299999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.51800000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.91100000000000003,"15min":0.92600000000000005},"min":{"1min":0.50600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.3480000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.1890000000000001,"15min":1.2370000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.573,"5min":3.032,"15min":3.032},"last":0.90500000000000003}]},{"osd":4,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.6579999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.018},{"interface":"front","average":{"1min":1.147,"5min":1.248,"15min":1.2649999999999999},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.589,"5min":3.069,"15min":3.069},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.234,"15min":1.248},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.0030000000000001,"5min":2.181,"15min":2.181},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.3620000000000001,"15min":1.3979999999999999},"min":{"1min":0.81499999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4350000000000001,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.3460000000000001,"15min":1.365},"min":{"1min":0.80100000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.571,"5min":2.633,"15min":2.633},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":0.78400000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.1539999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.1140000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.399,"15min":1.4259999999999999},"min":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6599999999999999,"5min":2.794,"15min":2.794},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.319,"15min":1.337},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.081,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.042}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":593920,"data_stored":254758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81000000000000005,"15min":0.80900000000000005},"min":{"1min":0.38700000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.877},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56999999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.2170000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0820000000000001}]},{"osd":2,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.84499999999999997,"15min":0.86699999999999999},"min":{"1min":0.53400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.319,"5min":1.768,"15min":1.768},"last":1.4530000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.129,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.98799999999999999,"15min":1.002},"min":{"1min":0.56000000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.454,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.4990000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2829999999999999,"15min":1.294},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.738,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.327,"15min":1.351},"min":{"1min":0.81799999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6279999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":2.0259999999999998}]},{"osd":5,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2849999999999999,"15min":1.3009999999999999},"min":{"1min":0.92000000000000004,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.694,"5min":3.319,"15min":3.319},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.3109999999999999,"15min":1.327},"min":{"1min":0.77200000000000002,"5min":0.755,"15min":0.755},"max":{"1min":1.5509999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.9930000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3160000000000001,"15min":1.3320000000000001},"min":{"1min":1.0369999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5580000000000001,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.931},{"interface":"front","average":{"1min":1.264,"5min":1.298,"15min":1.304},"min":{"1min":0.86799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6679999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.7070000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.407,"15min":1.427},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6859999999999999,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.423,"15min":1.4319999999999999},"min":{"1min":0.98399999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":1.766,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":2.3460000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1187840,"data_stored":845126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.47599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1819999999999999,"5min":1.329,"15min":1.329},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.038,"15min":1.0760000000000001},"min":{"1min":0.49099999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.083,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.69399999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.50900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.252,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2410000000000001,"5min":2.444,"15min":2.444},"last":0.84599999999999997}]},{"osd":3,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.873},"min":{"1min":0.504,"5min":0.31,"15min":0.31},"max":{"1min":1.165,"5min":1.474,"15min":1.474},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.123,"15min":1.133},"min":{"1min":0.58399999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.474,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.2090000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.331,"15min":1.341},"min":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.665,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.411},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.3009999999999999},"min":{"1min":0.93600000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.679,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.3380000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2410000000000001,"15min":1.2490000000000001},"min":{"1min":0.92100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.7929999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3,"15min":1.3009999999999999},"min":{"1min":0.91000000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.671,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.0169999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3640000000000001,"15min":1.3700000000000001},"min":{"1min":0.879,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5860000000000001,"5min":2.552,"15min":2.552},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.3640000000000001,"15min":1.3859999999999999},"min":{"1min":0.86499999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5940000000000001,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3779999999999999,"15min":1.3999999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.702,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3500000000000001,"15min":1.367},"min":{"1min":0.81299999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.4419999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:13:50.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:13:51.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.051+0000 7f0d004f7640 1 Processor -- start 2023-10-11T06:13:51.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.051+0000 7f0d004f7640 1 -- start start 2023-10-11T06:13:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.053+0000 7f0d004f7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0cf80ce0d0 0x7f0cf80c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.053+0000 7f0d004f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf80caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.053+0000 7f0d004f7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf81593f0 con 0x7f0cf80cd300 2023-10-11T06:13:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.053+0000 7f0d004f7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf815a340 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.053+0000 7f0d004f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80ce4a0 con 0x7f0cf80adb90 2023-10-11T06:13:51.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.054+0000 7f0cfda92640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf80caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.054+0000 7f0cfda92640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf80caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:41902/0 (socket says 172.21.15.84:41902) 2023-10-11T06:13:51.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.054+0000 7f0cfda92640 1 -- 172.21.15.84:0/2241538224 learned_addr learned my addr 172.21.15.84:0/2241538224 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:51.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.054+0000 7f0cfe293640 1 --2- 172.21.15.84:0/2241538224 >> v2:172.21.15.99:3300/0 conn(0x7f0cf80ce0d0 0x7f0cf80c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.055+0000 7f0cfda92640 1 -- 172.21.15.84:0/2241538224 >> v2:172.21.15.99:3300/0 conn(0x7f0cf80ce0d0 msgr2=0x7f0cf80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.055+0000 7f0cfda92640 1 --2- 172.21.15.84:0/2241538224 >> v2:172.21.15.99:3300/0 conn(0x7f0cf80ce0d0 0x7f0cf80c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.055+0000 7f0cfda92640 1 -- 172.21.15.84:0/2241538224 >> v1:172.21.15.99:6789/0 conn(0x7f0cf80cd300 legacy=0x7f0cf80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.055+0000 7f0cfda92640 1 -- 172.21.15.84:0/2241538224 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf81596c0 con 0x7f0cf80adb90 2023-10-11T06:13:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.056+0000 7f0cfda92640 1 --2- 172.21.15.84:0/2241538224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf80caee0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009870 tx=0x7f0ce802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dce9f89e95deae0e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.056+0000 7f0cfd291640 1 -- 172.21.15.84:0/2241538224 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8030840 con 0x7f0cf80adb90 2023-10-11T06:13:51.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.056+0000 7f0cfd291640 1 -- 172.21.15.84:0/2241538224 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce80309a0 con 0x7f0cf80adb90 2023-10-11T06:13:51.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.056+0000 7f0cfd291640 1 -- 172.21.15.84:0/2241538224 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8030d30 con 0x7f0cf80adb90 2023-10-11T06:13:51.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 -- 172.21.15.84:0/2241538224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 msgr2=0x7f0cf80caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 --2- 172.21.15.84:0/2241538224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf80caee0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009870 tx=0x7f0ce802b980 comp rx=0 tx=0).stop 2023-10-11T06:13:51.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 -- 172.21.15.84:0/2241538224 shutdown_connections 2023-10-11T06:13:51.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 --2- 172.21.15.84:0/2241538224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf80caee0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 --2- 172.21.15.84:0/2241538224 >> v2:172.21.15.99:3300/0 conn(0x7f0cf80ce0d0 0x7f0cf80c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 -- 172.21.15.84:0/2241538224 >> 172.21.15.84:0/2241538224 conn(0x7f0cf8069c70 msgr2=0x7f0cf806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:51.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 -- 172.21.15.84:0/2241538224 shutdown_connections 2023-10-11T06:13:51.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.057+0000 7f0d004f7640 1 -- 172.21.15.84:0/2241538224 wait complete. 2023-10-11T06:13:51.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.059+0000 7f0d004f7640 1 Processor -- start 2023-10-11T06:13:51.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.060+0000 7f0d004f7640 1 -- start start 2023-10-11T06:13:51.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.061+0000 7f0d004f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf81f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.061+0000 7f0d004f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf80cd300 0x7f0cf80b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.061+0000 7f0d004f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cf80ce0d0 0x7f0cf80b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.061+0000 7f0d004f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80ae060 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.061+0000 7f0d004f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80b0280 con 0x7f0cf80cd300 2023-10-11T06:13:51.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.061+0000 7f0d004f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81f1c40 con 0x7f0cf80adb90 2023-10-11T06:13:51.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.061+0000 7f0cfea94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cf80ce0d0 0x7f0cf80b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cf80ce0d0 0x7f0cf80b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54986/0 (socket says 172.21.15.84:54986) 2023-10-11T06:13:51.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 -- 172.21.15.84:0/863179639 learned_addr learned my addr 172.21.15.84:0/863179639 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:51.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfe293640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf81f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfda92640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf80cd300 0x7f0cf80b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 -- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 msgr2=0x7f0cf81f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf81f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 -- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf80cd300 msgr2=0x7f0cf80b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf80cd300 0x7f0cf80b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 -- 172.21.15.84:0/863179639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80094d0 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfda92640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf80cd300 0x7f0cf80b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:13:51.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.062+0000 7f0cfea94640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cf80ce0d0 0x7f0cf80b1430 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000c6b0 tx=0x7f0cf000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:51.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.063+0000 7f0ceeffd640 1 -- 172.21.15.84:0/863179639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0003b70 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.063+0000 7f0ceeffd640 1 -- 172.21.15.84:0/863179639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0003cf0 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.063+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf80ae270 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.063+0000 7f0ceeffd640 1 -- 172.21.15.84:0/863179639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0003ea0 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.063+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf80aff10 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.064+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf80b0430 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.065+0000 7f0ceeffd640 1 -- 172.21.15.84:0/863179639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0cf0005310 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.066+0000 7f0ceeffd640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cdc06a830 0x7f0cdc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.066+0000 7f0ceeffd640 1 -- 172.21.15.84:0/863179639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0cf0088dc0 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.066+0000 7f0cfe293640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cdc06a830 0x7f0cdc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.067+0000 7f0cfe293640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cdc06a830 0x7f0cdc06cce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009b30 tx=0x7f0cf4009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:51.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.071+0000 7f0ceeffd640 1 -- 172.21.15.84:0/863179639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0cf80b0430 con 0x7f0cf80ce0d0 2023-10-11T06:13:51.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.258+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cf8159c80 con 0x7f0cdc06a830 2023-10-11T06:13:51.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.268+0000 7f0ceeffd640 1 -- 172.21.15.84:0/863179639 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309226 (secure 0 0 0) 0x7f0cf4011a60 con 0x7f0cdc06a830 2023-10-11T06:13:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:51.276 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:13:51.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.273+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cdc06a830 msgr2=0x7f0cdc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.273+0000 7f0d004f7640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cdc06a830 0x7f0cdc06cce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009b30 tx=0x7f0cf4009110 comp rx=0 tx=0).stop 2023-10-11T06:13:51.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.273+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cf80ce0d0 msgr2=0x7f0cf80b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.273+0000 7f0d004f7640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cf80ce0d0 0x7f0cf80b1430 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000c6b0 tx=0x7f0cf000cb70 comp rx=0 tx=0).stop 2023-10-11T06:13:51.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.273+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 shutdown_connections 2023-10-11T06:13:51.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.274+0000 7f0d004f7640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0cdc06a830 0x7f0cdc06cce0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.274+0000 7f0d004f7640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0cf80ce0d0 0x7f0cf80b1430 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.274+0000 7f0d004f7640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf80cd300 0x7f0cf80b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.274+0000 7f0d004f7640 1 --2- 172.21.15.84:0/863179639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0cf80adb90 0x7f0cf81f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.274+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 >> 172.21.15.84:0/863179639 conn(0x7f0cf8069c70 msgr2=0x7f0cf80c4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:51.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.274+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 shutdown_connections 2023-10-11T06:13:51.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.274+0000 7f0d004f7640 1 -- 172.21.15.84:0/863179639 wait complete. 2023-10-11T06:13:51.294 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2023-10-11T06:13:50.822522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221864,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754752856,"statfs":{"total":773094113280,"available":772866924544,"internally_reserved":0,"allocated":6397952,"data_stored":3436092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.920179"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.408196+0000","last_change":"2023-10-11T06:13:46.415766+0000","last_active":"2023-10-11T06:13:47.408196+0000","last_peered":"2023-10-11T06:13:47.408196+0000","last_clean":"2023-10-11T06:13:47.408196+0000","last_became_active":"2023-10-11T06:13:46.415271+0000","last_became_peered":"2023-10-11T06:13:46.415271+0000","last_unstale":"2023-10-11T06:13:47.408196+0000","last_undegraded":"2023-10-11T06:13:47.408196+0000","last_fullsized":"2023-10-11T06:13:47.408196+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:53:26.626563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407862+0000","last_change":"2023-10-11T06:13:46.410204+0000","last_active":"2023-10-11T06:13:47.407862+0000","last_peered":"2023-10-11T06:13:47.407862+0000","last_clean":"2023-10-11T06:13:47.407862+0000","last_became_active":"2023-10-11T06:13:46.409902+0000","last_became_peered":"2023-10-11T06:13:46.409902+0000","last_unstale":"2023-10-11T06:13:47.407862+0000","last_undegraded":"2023-10-11T06:13:47.407862+0000","last_fullsized":"2023-10-11T06:13:47.407862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:30:54.072035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576526+0000","last_change":"2023-10-11T06:13:46.436762+0000","last_active":"2023-10-11T06:13:47.576526+0000","last_peered":"2023-10-11T06:13:47.576526+0000","last_clean":"2023-10-11T06:13:47.576526+0000","last_became_active":"2023-10-11T06:13:46.435953+0000","last_became_peered":"2023-10-11T06:13:46.435953+0000","last_unstale":"2023-10-11T06:13:47.576526+0000","last_undegraded":"2023-10-11T06:13:47.576526+0000","last_fullsized":"2023-10-11T06:13:47.576526+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:49:28.622959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576200+0000","last_change":"2023-10-11T06:13:46.419328+0000","last_active":"2023-10-11T06:13:47.576200+0000","last_peered":"2023-10-11T06:13:47.576200+0000","last_clean":"2023-10-11T06:13:47.576200+0000","last_became_active":"2023-10-11T06:13:46.418757+0000","last_became_peered":"2023-10-11T06:13:46.418757+0000","last_unstale":"2023-10-11T06:13:47.576200+0000","last_undegraded":"2023-10-11T06:13:47.576200+0000","last_fullsized":"2023-10-11T06:13:47.576200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:58:06.154499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407711+0000","last_change":"2023-10-11T06:13:46.429859+0000","last_active":"2023-10-11T06:13:47.407711+0000","last_peered":"2023-10-11T06:13:47.407711+0000","last_clean":"2023-10-11T06:13:47.407711+0000","last_became_active":"2023-10-11T06:13:46.429584+0000","last_became_peered":"2023-10-11T06:13:46.429584+0000","last_unstale":"2023-10-11T06:13:47.407711+0000","last_undegraded":"2023-10-11T06:13:47.407711+0000","last_fullsized":"2023-10-11T06:13:47.407711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:19:55.118013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573863+0000","last_change":"2023-10-11T06:13:46.415439+0000","last_active":"2023-10-11T06:13:47.573863+0000","last_peered":"2023-10-11T06:13:47.573863+0000","last_clean":"2023-10-11T06:13:47.573863+0000","last_became_active":"2023-10-11T06:13:46.415162+0000","last_became_peered":"2023-10-11T06:13:46.415162+0000","last_unstale":"2023-10-11T06:13:47.573863+0000","last_undegraded":"2023-10-11T06:13:47.573863+0000","last_fullsized":"2023-10-11T06:13:47.573863+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:49:28.622959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573218+0000","last_change":"2023-10-11T06:13:46.424681+0000","last_active":"2023-10-11T06:13:47.573218+0000","last_peered":"2023-10-11T06:13:47.573218+0000","last_clean":"2023-10-11T06:13:47.573218+0000","last_became_active":"2023-10-11T06:13:46.424421+0000","last_became_peered":"2023-10-11T06:13:46.424421+0000","last_unstale":"2023-10-11T06:13:47.573218+0000","last_undegraded":"2023-10-11T06:13:47.573218+0000","last_fullsized":"2023-10-11T06:13:47.573218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:04:59.783970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.406903+0000","last_change":"2023-10-11T06:13:46.424691+0000","last_active":"2023-10-11T06:13:47.406903+0000","last_peered":"2023-10-11T06:13:47.406903+0000","last_clean":"2023-10-11T06:13:47.406903+0000","last_became_active":"2023-10-11T06:13:46.424434+0000","last_became_peered":"2023-10-11T06:13:46.424434+0000","last_unstale":"2023-10-11T06:13:47.406903+0000","last_undegraded":"2023-10-11T06:13:47.406903+0000","last_fullsized":"2023-10-11T06:13:47.406903+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:58:22.653166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573911+0000","last_change":"2023-10-11T06:13:46.428854+0000","last_active":"2023-10-11T06:13:47.573911+0000","last_peered":"2023-10-11T06:13:47.573911+0000","last_clean":"2023-10-11T06:13:47.573911+0000","last_became_active":"2023-10-11T06:13:46.428528+0000","last_became_peered":"2023-10-11T06:13:46.428528+0000","last_unstale":"2023-10-11T06:13:47.573911+0000","last_undegraded":"2023-10-11T06:13:47.573911+0000","last_fullsized":"2023-10-11T06:13:47.573911+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:49:28.622959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573116+0000","last_change":"2023-10-11T06:13:46.421487+0000","last_active":"2023-10-11T06:13:47.573116+0000","last_peered":"2023-10-11T06:13:47.573116+0000","last_clean":"2023-10-11T06:13:47.573116+0000","last_became_active":"2023-10-11T06:13:46.421003+0000","last_became_peered":"2023-10-11T06:13:46.421003+0000","last_unstale":"2023-10-11T06:13:47.573116+0000","last_undegraded":"2023-10-11T06:13:47.573116+0000","last_fullsized":"2023-10-11T06:13:47.573116+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:54:21.850263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573574+0000","last_change":"2023-10-11T06:13:46.411180+0000","last_active":"2023-10-11T06:13:47.573574+0000","last_peered":"2023-10-11T06:13:47.573574+0000","last_clean":"2023-10-11T06:13:47.573574+0000","last_became_active":"2023-10-11T06:13:46.410910+0000","last_became_peered":"2023-10-11T06:13:46.410910+0000","last_unstale":"2023-10-11T06:13:47.573574+0000","last_undegraded":"2023-10-11T06:13:47.573574+0000","last_fullsized":"2023-10-11T06:13:47.573574+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:18:42.921779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576323+0000","last_change":"2023-10-11T06:13:46.420453+0000","last_active":"2023-10-11T06:13:47.576323+0000","last_peered":"2023-10-11T06:13:47.576323+0000","last_clean":"2023-10-11T06:13:47.576323+0000","last_became_active":"2023-10-11T06:13:46.420093+0000","last_became_peered":"2023-10-11T06:13:46.420093+0000","last_unstale":"2023-10-11T06:13:47.576323+0000","last_undegraded":"2023-10-11T06:13:47.576323+0000","last_fullsized":"2023-10-11T06:13:47.576323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:34:14.178507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.410027+0000","last_change":"2023-10-11T06:13:46.415289+0000","last_active":"2023-10-11T06:13:47.410027+0000","last_peered":"2023-10-11T06:13:47.410027+0000","last_clean":"2023-10-11T06:13:47.410027+0000","last_became_active":"2023-10-11T06:13:46.414680+0000","last_became_peered":"2023-10-11T06:13:46.414680+0000","last_unstale":"2023-10-11T06:13:47.410027+0000","last_undegraded":"2023-10-11T06:13:47.410027+0000","last_fullsized":"2023-10-11T06:13:47.410027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:54:50.053352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573262+0000","last_change":"2023-10-11T06:13:46.423529+0000","last_active":"2023-10-11T06:13:47.573262+0000","last_peered":"2023-10-11T06:13:47.573262+0000","last_clean":"2023-10-11T06:13:47.573262+0000","last_became_active":"2023-10-11T06:13:46.423184+0000","last_became_peered":"2023-10-11T06:13:46.423184+0000","last_unstale":"2023-10-11T06:13:47.573262+0000","last_undegraded":"2023-10-11T06:13:47.573262+0000","last_fullsized":"2023-10-11T06:13:47.573262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:34:33.788385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574079+0000","last_change":"2023-10-11T06:13:46.418589+0000","last_active":"2023-10-11T06:13:47.574079+0000","last_peered":"2023-10-11T06:13:47.574079+0000","last_clean":"2023-10-11T06:13:47.574079+0000","last_became_active":"2023-10-11T06:13:46.413219+0000","last_became_peered":"2023-10-11T06:13:46.413219+0000","last_unstale":"2023-10-11T06:13:47.574079+0000","last_undegraded":"2023-10-11T06:13:47.574079+0000","last_fullsized":"2023-10-11T06:13:47.574079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:26:24.488013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574136+0000","last_change":"2023-10-11T06:13:46.424225+0000","last_active":"2023-10-11T06:13:47.574136+0000","last_peered":"2023-10-11T06:13:47.574136+0000","last_clean":"2023-10-11T06:13:47.574136+0000","last_became_active":"2023-10-11T06:13:46.423980+0000","last_became_peered":"2023-10-11T06:13:46.423980+0000","last_unstale":"2023-10-11T06:13:47.574136+0000","last_undegraded":"2023-10-11T06:13:47.574136+0000","last_fullsized":"2023-10-11T06:13:47.574136+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:34:14.178507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407100+0000","last_change":"2023-10-11T06:13:46.412521+0000","last_active":"2023-10-11T06:13:47.407100+0000","last_peered":"2023-10-11T06:13:47.407100+0000","last_clean":"2023-10-11T06:13:47.407100+0000","last_became_active":"2023-10-11T06:13:46.412267+0000","last_became_peered":"2023-10-11T06:13:46.412267+0000","last_unstale":"2023-10-11T06:13:47.407100+0000","last_undegraded":"2023-10-11T06:13:47.407100+0000","last_fullsized":"2023-10-11T06:13:47.407100+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:00:56.117041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576779+0000","last_change":"2023-10-11T06:13:46.434860+0000","last_active":"2023-10-11T06:13:47.576779+0000","last_peered":"2023-10-11T06:13:47.576779+0000","last_clean":"2023-10-11T06:13:47.576779+0000","last_became_active":"2023-10-11T06:13:46.433010+0000","last_became_peered":"2023-10-11T06:13:46.433010+0000","last_unstale":"2023-10-11T06:13:47.576779+0000","last_undegraded":"2023-10-11T06:13:47.576779+0000","last_fullsized":"2023-10-11T06:13:47.576779+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:31:01.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.408084+0000","last_change":"2023-10-11T06:13:46.415085+0000","last_active":"2023-10-11T06:13:47.408084+0000","last_peered":"2023-10-11T06:13:47.408084+0000","last_clean":"2023-10-11T06:13:47.408084+0000","last_became_active":"2023-10-11T06:13:46.414832+0000","last_became_peered":"2023-10-11T06:13:46.414832+0000","last_unstale":"2023-10-11T06:13:47.408084+0000","last_undegraded":"2023-10-11T06:13:47.408084+0000","last_fullsized":"2023-10-11T06:13:47.408084+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:42:59.943267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407087+0000","last_change":"2023-10-11T06:13:46.413458+0000","last_active":"2023-10-11T06:13:47.407087+0000","last_peered":"2023-10-11T06:13:47.407087+0000","last_clean":"2023-10-11T06:13:47.407087+0000","last_became_active":"2023-10-11T06:13:46.413186+0000","last_became_peered":"2023-10-11T06:13:46.413186+0000","last_unstale":"2023-10-11T06:13:47.407087+0000","last_undegraded":"2023-10-11T06:13:47.407087+0000","last_fullsized":"2023-10-11T06:13:47.407087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:58:06.154499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575835+0000","last_change":"2023-10-11T06:13:46.436421+0000","last_active":"2023-10-11T06:13:47.575835+0000","last_peered":"2023-10-11T06:13:47.575835+0000","last_clean":"2023-10-11T06:13:47.575835+0000","last_became_active":"2023-10-11T06:13:46.435759+0000","last_became_peered":"2023-10-11T06:13:46.435759+0000","last_unstale":"2023-10-11T06:13:47.575835+0000","last_undegraded":"2023-10-11T06:13:47.575835+0000","last_fullsized":"2023-10-11T06:13:47.575835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:41.070650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.406995+0000","last_change":"2023-10-11T06:13:46.416565+0000","last_active":"2023-10-11T06:13:47.406995+0000","last_peered":"2023-10-11T06:13:47.406995+0000","last_clean":"2023-10-11T06:13:47.406995+0000","last_became_active":"2023-10-11T06:13:46.416331+0000","last_became_peered":"2023-10-11T06:13:46.416331+0000","last_unstale":"2023-10-11T06:13:47.406995+0000","last_undegraded":"2023-10-11T06:13:47.406995+0000","last_fullsized":"2023-10-11T06:13:47.406995+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:08:08.814504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575531+0000","last_change":"2023-10-11T06:13:46.429607+0000","last_active":"2023-10-11T06:13:47.575531+0000","last_peered":"2023-10-11T06:13:47.575531+0000","last_clean":"2023-10-11T06:13:47.575531+0000","last_became_active":"2023-10-11T06:13:46.429342+0000","last_became_peered":"2023-10-11T06:13:46.429342+0000","last_unstale":"2023-10-11T06:13:47.575531+0000","last_undegraded":"2023-10-11T06:13:47.575531+0000","last_fullsized":"2023-10-11T06:13:47.575531+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:44:34.965313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.409686+0000","last_change":"2023-10-11T06:13:46.413907+0000","last_active":"2023-10-11T06:13:47.409686+0000","last_peered":"2023-10-11T06:13:47.409686+0000","last_clean":"2023-10-11T06:13:47.409686+0000","last_became_active":"2023-10-11T06:13:46.413263+0000","last_became_peered":"2023-10-11T06:13:46.413263+0000","last_unstale":"2023-10-11T06:13:47.409686+0000","last_undegraded":"2023-10-11T06:13:47.409686+0000","last_fullsized":"2023-10-11T06:13:47.409686+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:36:50.035544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.408321+0000","last_change":"2023-10-11T06:13:46.415829+0000","last_active":"2023-10-11T06:13:47.408321+0000","last_peered":"2023-10-11T06:13:47.408321+0000","last_clean":"2023-10-11T06:13:47.408321+0000","last_became_active":"2023-10-11T06:13:46.415499+0000","last_became_peered":"2023-10-11T06:13:46.415499+0000","last_unstale":"2023-10-11T06:13:47.408321+0000","last_undegraded":"2023-10-11T06:13:47.408321+0000","last_fullsized":"2023-10-11T06:13:47.408321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:28:30.801458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574805+0000","last_change":"2023-10-11T06:13:46.430494+0000","last_active":"2023-10-11T06:13:47.574805+0000","last_peered":"2023-10-11T06:13:47.574805+0000","last_clean":"2023-10-11T06:13:47.574805+0000","last_became_active":"2023-10-11T06:13:46.430021+0000","last_became_peered":"2023-10-11T06:13:46.430021+0000","last_unstale":"2023-10-11T06:13:47.574805+0000","last_undegraded":"2023-10-11T06:13:47.574805+0000","last_fullsized":"2023-10-11T06:13:47.574805+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:49:31.986494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574940+0000","last_change":"2023-10-11T06:13:46.418273+0000","last_active":"2023-10-11T06:13:47.574940+0000","last_peered":"2023-10-11T06:13:47.574940+0000","last_clean":"2023-10-11T06:13:47.574940+0000","last_became_active":"2023-10-11T06:13:46.418005+0000","last_became_peered":"2023-10-11T06:13:46.418005+0000","last_unstale":"2023-10-11T06:13:47.574940+0000","last_undegraded":"2023-10-11T06:13:47.574940+0000","last_fullsized":"2023-10-11T06:13:47.574940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:41.070650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407710+0000","last_change":"2023-10-11T06:13:46.425497+0000","last_active":"2023-10-11T06:13:47.407710+0000","last_peered":"2023-10-11T06:13:47.407710+0000","last_clean":"2023-10-11T06:13:47.407710+0000","last_became_active":"2023-10-11T06:13:46.425244+0000","last_became_peered":"2023-10-11T06:13:46.425244+0000","last_unstale":"2023-10-11T06:13:47.407710+0000","last_undegraded":"2023-10-11T06:13:47.407710+0000","last_fullsized":"2023-10-11T06:13:47.407710+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:42:48.273370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407480+0000","last_change":"2023-10-11T06:13:46.419043+0000","last_active":"2023-10-11T06:13:47.407480+0000","last_peered":"2023-10-11T06:13:47.407480+0000","last_clean":"2023-10-11T06:13:47.407480+0000","last_became_active":"2023-10-11T06:13:46.418802+0000","last_became_peered":"2023-10-11T06:13:46.418802+0000","last_unstale":"2023-10-11T06:13:47.407480+0000","last_undegraded":"2023-10-11T06:13:47.407480+0000","last_fullsized":"2023-10-11T06:13:47.407480+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:45:50.506464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575472+0000","last_change":"2023-10-11T06:13:46.435007+0000","last_active":"2023-10-11T06:13:47.575472+0000","last_peered":"2023-10-11T06:13:47.575472+0000","last_clean":"2023-10-11T06:13:47.575472+0000","last_became_active":"2023-10-11T06:13:46.433086+0000","last_became_peered":"2023-10-11T06:13:46.433086+0000","last_unstale":"2023-10-11T06:13:47.575472+0000","last_undegraded":"2023-10-11T06:13:47.575472+0000","last_fullsized":"2023-10-11T06:13:47.575472+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:34:33.788385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407498+0000","last_change":"2023-10-11T06:13:46.412144+0000","last_active":"2023-10-11T06:13:47.407498+0000","last_peered":"2023-10-11T06:13:47.407498+0000","last_clean":"2023-10-11T06:13:47.407498+0000","last_became_active":"2023-10-11T06:13:46.411935+0000","last_became_peered":"2023-10-11T06:13:46.411935+0000","last_unstale":"2023-10-11T06:13:47.407498+0000","last_undegraded":"2023-10-11T06:13:47.407498+0000","last_fullsized":"2023-10-11T06:13:47.407498+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:07:52.410468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574231+0000","last_change":"2023-10-11T06:13:46.408971+0000","last_active":"2023-10-11T06:13:47.574231+0000","last_peered":"2023-10-11T06:13:47.574231+0000","last_clean":"2023-10-11T06:13:47.574231+0000","last_became_active":"2023-10-11T06:13:46.408703+0000","last_became_peered":"2023-10-11T06:13:46.408703+0000","last_unstale":"2023-10-11T06:13:47.574231+0000","last_undegraded":"2023-10-11T06:13:47.574231+0000","last_fullsized":"2023-10-11T06:13:47.574231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:47:52.646040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575170+0000","last_change":"2023-10-11T06:13:46.427812+0000","last_active":"2023-10-11T06:13:47.575170+0000","last_peered":"2023-10-11T06:13:47.575170+0000","last_clean":"2023-10-11T06:13:47.575170+0000","last_became_active":"2023-10-11T06:13:46.427495+0000","last_became_peered":"2023-10-11T06:13:46.427495+0000","last_unstale":"2023-10-11T06:13:47.575170+0000","last_undegraded":"2023-10-11T06:13:47.575170+0000","last_fullsized":"2023-10-11T06:13:47.575170+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:08:49.498643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.430809+0000","last_change":"2023-10-11T06:13:45.939633+0000","last_active":"2023-10-11T06:13:46.430809+0000","last_peered":"2023-10-11T06:13:46.430809+0000","last_clean":"2023-10-11T06:13:46.430809+0000","last_became_active":"2023-10-11T06:13:45.939381+0000","last_became_peered":"2023-10-11T06:13:45.939381+0000","last_unstale":"2023-10-11T06:13:46.430809+0000","last_undegraded":"2023-10-11T06:13:46.430809+0000","last_fullsized":"2023-10-11T06:13:46.430809+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:10:27.512577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573734+0000","last_change":"2023-10-11T06:13:46.426542+0000","last_active":"2023-10-11T06:13:47.573734+0000","last_peered":"2023-10-11T06:13:47.573734+0000","last_clean":"2023-10-11T06:13:47.573734+0000","last_became_active":"2023-10-11T06:13:46.426305+0000","last_became_peered":"2023-10-11T06:13:46.426305+0000","last_unstale":"2023-10-11T06:13:47.573734+0000","last_undegraded":"2023-10-11T06:13:47.573734+0000","last_fullsized":"2023-10-11T06:13:47.573734+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:52:13.921672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.426455+0000","last_change":"2023-10-11T06:13:45.942935+0000","last_active":"2023-10-11T06:13:46.426455+0000","last_peered":"2023-10-11T06:13:46.426455+0000","last_clean":"2023-10-11T06:13:46.426455+0000","last_became_active":"2023-10-11T06:13:45.942626+0000","last_became_peered":"2023-10-11T06:13:45.942626+0000","last_unstale":"2023-10-11T06:13:46.426455+0000","last_undegraded":"2023-10-11T06:13:46.426455+0000","last_fullsized":"2023-10-11T06:13:46.426455+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:35.798111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574633+0000","last_change":"2023-10-11T06:13:46.414171+0000","last_active":"2023-10-11T06:13:47.574633+0000","last_peered":"2023-10-11T06:13:47.574633+0000","last_clean":"2023-10-11T06:13:47.574633+0000","last_became_active":"2023-10-11T06:13:46.413928+0000","last_became_peered":"2023-10-11T06:13:46.413928+0000","last_unstale":"2023-10-11T06:13:47.574633+0000","last_undegraded":"2023-10-11T06:13:47.574633+0000","last_fullsized":"2023-10-11T06:13:47.574633+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:31:01.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.422920+0000","last_change":"2023-10-11T06:13:45.938276+0000","last_active":"2023-10-11T06:13:46.422920+0000","last_peered":"2023-10-11T06:13:46.422920+0000","last_clean":"2023-10-11T06:13:46.422920+0000","last_became_active":"2023-10-11T06:13:45.938024+0000","last_became_peered":"2023-10-11T06:13:45.938024+0000","last_unstale":"2023-10-11T06:13:46.422920+0000","last_undegraded":"2023-10-11T06:13:46.422920+0000","last_fullsized":"2023-10-11T06:13:46.422920+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:26.580696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575279+0000","last_change":"2023-10-11T06:13:46.419569+0000","last_active":"2023-10-11T06:13:47.575279+0000","last_peered":"2023-10-11T06:13:47.575279+0000","last_clean":"2023-10-11T06:13:47.575279+0000","last_became_active":"2023-10-11T06:13:46.419326+0000","last_became_peered":"2023-10-11T06:13:46.419326+0000","last_unstale":"2023-10-11T06:13:47.575279+0000","last_undegraded":"2023-10-11T06:13:47.575279+0000","last_fullsized":"2023-10-11T06:13:47.575279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:30:07.936139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.439425+0000","last_change":"2023-10-11T06:13:45.949038+0000","last_active":"2023-10-11T06:13:46.439425+0000","last_peered":"2023-10-11T06:13:46.439425+0000","last_clean":"2023-10-11T06:13:46.439425+0000","last_became_active":"2023-10-11T06:13:45.948723+0000","last_became_peered":"2023-10-11T06:13:45.948723+0000","last_unstale":"2023-10-11T06:13:46.439425+0000","last_undegraded":"2023-10-11T06:13:46.439425+0000","last_fullsized":"2023-10-11T06:13:46.439425+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:12:14.685368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407273+0000","last_change":"2023-10-11T06:13:46.412278+0000","last_active":"2023-10-11T06:13:47.407273+0000","last_peered":"2023-10-11T06:13:47.407273+0000","last_clean":"2023-10-11T06:13:47.407273+0000","last_became_active":"2023-10-11T06:13:46.412044+0000","last_became_peered":"2023-10-11T06:13:46.412044+0000","last_unstale":"2023-10-11T06:13:47.407273+0000","last_undegraded":"2023-10-11T06:13:47.407273+0000","last_fullsized":"2023-10-11T06:13:47.407273+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:57:43.093763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.433581+0000","last_change":"2023-10-11T06:13:45.940518+0000","last_active":"2023-10-11T06:13:46.433581+0000","last_peered":"2023-10-11T06:13:46.433581+0000","last_clean":"2023-10-11T06:13:46.433581+0000","last_became_active":"2023-10-11T06:13:45.940172+0000","last_became_peered":"2023-10-11T06:13:45.940172+0000","last_unstale":"2023-10-11T06:13:46.433581+0000","last_undegraded":"2023-10-11T06:13:46.433581+0000","last_fullsized":"2023-10-11T06:13:46.433581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:40:00.524726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407257+0000","last_change":"2023-10-11T06:13:46.406544+0000","last_active":"2023-10-11T06:13:47.407257+0000","last_peered":"2023-10-11T06:13:47.407257+0000","last_clean":"2023-10-11T06:13:47.407257+0000","last_became_active":"2023-10-11T06:13:46.406313+0000","last_became_peered":"2023-10-11T06:13:46.406313+0000","last_unstale":"2023-10-11T06:13:47.407257+0000","last_undegraded":"2023-10-11T06:13:47.407257+0000","last_fullsized":"2023-10-11T06:13:47.407257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:30:07.936139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.430181+0000","last_change":"2023-10-11T06:13:45.941320+0000","last_active":"2023-10-11T06:13:46.430181+0000","last_peered":"2023-10-11T06:13:46.430181+0000","last_clean":"2023-10-11T06:13:46.430181+0000","last_became_active":"2023-10-11T06:13:45.941000+0000","last_became_peered":"2023-10-11T06:13:45.941000+0000","last_unstale":"2023-10-11T06:13:46.430181+0000","last_undegraded":"2023-10-11T06:13:46.430181+0000","last_fullsized":"2023-10-11T06:13:46.430181+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:48.568608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.409783+0000","last_change":"2023-10-11T06:13:46.428836+0000","last_active":"2023-10-11T06:13:47.409783+0000","last_peered":"2023-10-11T06:13:47.409783+0000","last_clean":"2023-10-11T06:13:47.409783+0000","last_became_active":"2023-10-11T06:13:46.428539+0000","last_became_peered":"2023-10-11T06:13:46.428539+0000","last_unstale":"2023-10-11T06:13:47.409783+0000","last_undegraded":"2023-10-11T06:13:47.409783+0000","last_fullsized":"2023-10-11T06:13:47.409783+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:44:11.896183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.415428+0000","last_change":"2023-10-11T06:13:45.943590+0000","last_active":"2023-10-11T06:13:46.415428+0000","last_peered":"2023-10-11T06:13:46.415428+0000","last_clean":"2023-10-11T06:13:46.415428+0000","last_became_active":"2023-10-11T06:13:45.943023+0000","last_became_peered":"2023-10-11T06:13:45.943023+0000","last_unstale":"2023-10-11T06:13:46.415428+0000","last_undegraded":"2023-10-11T06:13:46.415428+0000","last_fullsized":"2023-10-11T06:13:46.415428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:12.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407460+0000","last_change":"2023-10-11T06:13:46.413326+0000","last_active":"2023-10-11T06:13:47.407460+0000","last_peered":"2023-10-11T06:13:47.407460+0000","last_clean":"2023-10-11T06:13:47.407460+0000","last_became_active":"2023-10-11T06:13:46.412914+0000","last_became_peered":"2023-10-11T06:13:46.412914+0000","last_unstale":"2023-10-11T06:13:47.407460+0000","last_undegraded":"2023-10-11T06:13:47.407460+0000","last_fullsized":"2023-10-11T06:13:47.407460+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:41:24.165930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.408133+0000","last_change":"2023-10-11T06:13:45.940252+0000","last_active":"2023-10-11T06:13:46.408133+0000","last_peered":"2023-10-11T06:13:46.408133+0000","last_clean":"2023-10-11T06:13:46.408133+0000","last_became_active":"2023-10-11T06:13:45.939961+0000","last_became_peered":"2023-10-11T06:13:45.939961+0000","last_unstale":"2023-10-11T06:13:46.408133+0000","last_undegraded":"2023-10-11T06:13:46.408133+0000","last_fullsized":"2023-10-11T06:13:46.408133+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:46:48.568608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573921+0000","last_change":"2023-10-11T06:13:46.426650+0000","last_active":"2023-10-11T06:13:47.573921+0000","last_peered":"2023-10-11T06:13:47.573921+0000","last_clean":"2023-10-11T06:13:47.573921+0000","last_became_active":"2023-10-11T06:13:46.425086+0000","last_became_peered":"2023-10-11T06:13:46.425086+0000","last_unstale":"2023-10-11T06:13:47.573921+0000","last_undegraded":"2023-10-11T06:13:47.573921+0000","last_fullsized":"2023-10-11T06:13:47.573921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:12:21.710813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.424565+0000","last_change":"2023-10-11T06:13:45.947120+0000","last_active":"2023-10-11T06:13:46.424565+0000","last_peered":"2023-10-11T06:13:46.424565+0000","last_clean":"2023-10-11T06:13:46.424565+0000","last_became_active":"2023-10-11T06:13:45.946794+0000","last_became_peered":"2023-10-11T06:13:45.946794+0000","last_unstale":"2023-10-11T06:13:46.424565+0000","last_undegraded":"2023-10-11T06:13:46.424565+0000","last_fullsized":"2023-10-11T06:13:46.424565+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:25:43.343389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.577297+0000","last_change":"2023-10-11T06:13:46.435370+0000","last_active":"2023-10-11T06:13:47.577297+0000","last_peered":"2023-10-11T06:13:47.577297+0000","last_clean":"2023-10-11T06:13:47.577297+0000","last_became_active":"2023-10-11T06:13:46.434791+0000","last_became_peered":"2023-10-11T06:13:46.434791+0000","last_unstale":"2023-10-11T06:13:47.577297+0000","last_undegraded":"2023-10-11T06:13:47.577297+0000","last_fullsized":"2023-10-11T06:13:47.577297+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:12:21.710813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.398575+0000","last_change":"2023-10-11T06:13:45.933557+0000","last_active":"2023-10-11T06:13:46.398575+0000","last_peered":"2023-10-11T06:13:46.398575+0000","last_clean":"2023-10-11T06:13:46.398575+0000","last_became_active":"2023-10-11T06:13:45.933263+0000","last_became_peered":"2023-10-11T06:13:45.933263+0000","last_unstale":"2023-10-11T06:13:46.398575+0000","last_undegraded":"2023-10-11T06:13:46.398575+0000","last_fullsized":"2023-10-11T06:13:46.398575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:26.580696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573487+0000","last_change":"2023-10-11T06:13:46.421022+0000","last_active":"2023-10-11T06:13:47.573487+0000","last_peered":"2023-10-11T06:13:47.573487+0000","last_clean":"2023-10-11T06:13:47.573487+0000","last_became_active":"2023-10-11T06:13:46.420578+0000","last_became_peered":"2023-10-11T06:13:46.420578+0000","last_unstale":"2023-10-11T06:13:47.573487+0000","last_undegraded":"2023-10-11T06:13:47.573487+0000","last_fullsized":"2023-10-11T06:13:47.573487+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:40:50.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.440288+0000","last_change":"2023-10-11T06:13:45.941195+0000","last_active":"2023-10-11T06:13:46.440288+0000","last_peered":"2023-10-11T06:13:46.440288+0000","last_clean":"2023-10-11T06:13:46.440288+0000","last_became_active":"2023-10-11T06:13:45.940922+0000","last_became_peered":"2023-10-11T06:13:45.940922+0000","last_unstale":"2023-10-11T06:13:46.440288+0000","last_undegraded":"2023-10-11T06:13:46.440288+0000","last_fullsized":"2023-10-11T06:13:46.440288+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:52:26.451241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576983+0000","last_change":"2023-10-11T06:13:46.437760+0000","last_active":"2023-10-11T06:13:47.576983+0000","last_peered":"2023-10-11T06:13:47.576983+0000","last_clean":"2023-10-11T06:13:47.576983+0000","last_became_active":"2023-10-11T06:13:46.436222+0000","last_became_peered":"2023-10-11T06:13:46.436222+0000","last_unstale":"2023-10-11T06:13:47.576983+0000","last_undegraded":"2023-10-11T06:13:47.576983+0000","last_fullsized":"2023-10-11T06:13:47.576983+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:53:25.863567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.436999+0000","last_change":"2023-10-11T06:13:45.941275+0000","last_active":"2023-10-11T06:13:46.436999+0000","last_peered":"2023-10-11T06:13:46.436999+0000","last_clean":"2023-10-11T06:13:46.436999+0000","last_became_active":"2023-10-11T06:13:45.940852+0000","last_became_peered":"2023-10-11T06:13:45.940852+0000","last_unstale":"2023-10-11T06:13:46.436999+0000","last_undegraded":"2023-10-11T06:13:46.436999+0000","last_fullsized":"2023-10-11T06:13:46.436999+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:43:42.843730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574271+0000","last_change":"2023-10-11T06:13:46.414933+0000","last_active":"2023-10-11T06:13:47.574271+0000","last_peered":"2023-10-11T06:13:47.574271+0000","last_clean":"2023-10-11T06:13:47.574271+0000","last_became_active":"2023-10-11T06:13:46.413527+0000","last_became_peered":"2023-10-11T06:13:46.413527+0000","last_unstale":"2023-10-11T06:13:47.574271+0000","last_undegraded":"2023-10-11T06:13:47.574271+0000","last_fullsized":"2023-10-11T06:13:47.574271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:52:33.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.424618+0000","last_change":"2023-10-11T06:13:45.933551+0000","last_active":"2023-10-11T06:13:46.424618+0000","last_peered":"2023-10-11T06:13:46.424618+0000","last_clean":"2023-10-11T06:13:46.424618+0000","last_became_active":"2023-10-11T06:13:45.933263+0000","last_became_peered":"2023-10-11T06:13:45.933263+0000","last_unstale":"2023-10-11T06:13:46.424618+0000","last_undegraded":"2023-10-11T06:13:46.424618+0000","last_fullsized":"2023-10-11T06:13:46.424618+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:10:27.512577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576409+0000","last_change":"2023-10-11T06:13:46.435627+0000","last_active":"2023-10-11T06:13:47.576409+0000","last_peered":"2023-10-11T06:13:47.576409+0000","last_clean":"2023-10-11T06:13:47.576409+0000","last_became_active":"2023-10-11T06:13:46.432983+0000","last_became_peered":"2023-10-11T06:13:46.432983+0000","last_unstale":"2023-10-11T06:13:47.576409+0000","last_undegraded":"2023-10-11T06:13:47.576409+0000","last_fullsized":"2023-10-11T06:13:47.576409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:49:31.986494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.404553+0000","last_change":"2023-10-11T06:13:45.944981+0000","last_active":"2023-10-11T06:13:46.404553+0000","last_peered":"2023-10-11T06:13:46.404553+0000","last_clean":"2023-10-11T06:13:46.404553+0000","last_became_active":"2023-10-11T06:13:45.944710+0000","last_became_peered":"2023-10-11T06:13:45.944710+0000","last_unstale":"2023-10-11T06:13:46.404553+0000","last_undegraded":"2023-10-11T06:13:46.404553+0000","last_fullsized":"2023-10-11T06:13:46.404553+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:19:54.106151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573100+0000","last_change":"2023-10-11T06:13:46.439747+0000","last_active":"2023-10-11T06:13:47.573100+0000","last_peered":"2023-10-11T06:13:47.573100+0000","last_clean":"2023-10-11T06:13:47.573100+0000","last_became_active":"2023-10-11T06:13:46.439547+0000","last_became_peered":"2023-10-11T06:13:46.439547+0000","last_unstale":"2023-10-11T06:13:47.573100+0000","last_undegraded":"2023-10-11T06:13:47.573100+0000","last_fullsized":"2023-10-11T06:13:47.573100+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:53:25.863567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.415579+0000","last_change":"2023-10-11T06:13:45.949048+0000","last_active":"2023-10-11T06:13:46.415579+0000","last_peered":"2023-10-11T06:13:46.415579+0000","last_clean":"2023-10-11T06:13:46.415579+0000","last_became_active":"2023-10-11T06:13:45.948718+0000","last_became_peered":"2023-10-11T06:13:45.948718+0000","last_unstale":"2023-10-11T06:13:46.415579+0000","last_undegraded":"2023-10-11T06:13:46.415579+0000","last_fullsized":"2023-10-11T06:13:46.415579+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:10:27.512577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575569+0000","last_change":"2023-10-11T06:13:46.433050+0000","last_active":"2023-10-11T06:13:47.575569+0000","last_peered":"2023-10-11T06:13:47.575569+0000","last_clean":"2023-10-11T06:13:47.575569+0000","last_became_active":"2023-10-11T06:13:46.432743+0000","last_became_peered":"2023-10-11T06:13:46.432743+0000","last_unstale":"2023-10-11T06:13:47.575569+0000","last_undegraded":"2023-10-11T06:13:47.575569+0000","last_fullsized":"2023-10-11T06:13:47.575569+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:04:59.783970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.440802+0000","last_change":"2023-10-11T06:13:45.950195+0000","last_active":"2023-10-11T06:13:46.440802+0000","last_peered":"2023-10-11T06:13:46.440802+0000","last_clean":"2023-10-11T06:13:46.440802+0000","last_became_active":"2023-10-11T06:13:45.949881+0000","last_became_peered":"2023-10-11T06:13:45.949881+0000","last_unstale":"2023-10-11T06:13:46.440802+0000","last_undegraded":"2023-10-11T06:13:46.440802+0000","last_fullsized":"2023-10-11T06:13:46.440802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:40:26.827270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575083+0000","last_change":"2023-10-11T06:13:46.419161+0000","last_active":"2023-10-11T06:13:47.575083+0000","last_peered":"2023-10-11T06:13:47.575083+0000","last_clean":"2023-10-11T06:13:47.575083+0000","last_became_active":"2023-10-11T06:13:46.417036+0000","last_became_peered":"2023-10-11T06:13:46.417036+0000","last_unstale":"2023-10-11T06:13:47.575083+0000","last_undegraded":"2023-10-11T06:13:47.575083+0000","last_fullsized":"2023-10-11T06:13:47.575083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:31:01.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.429750+0000","last_change":"2023-10-11T06:13:45.936148+0000","last_active":"2023-10-11T06:13:46.429750+0000","last_peered":"2023-10-11T06:13:46.429750+0000","last_clean":"2023-10-11T06:13:46.429750+0000","last_became_active":"2023-10-11T06:13:45.935905+0000","last_became_peered":"2023-10-11T06:13:45.935905+0000","last_unstale":"2023-10-11T06:13:46.429750+0000","last_undegraded":"2023-10-11T06:13:46.429750+0000","last_fullsized":"2023-10-11T06:13:46.429750+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:53:25.614701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575579+0000","last_change":"2023-10-11T06:13:46.430095+0000","last_active":"2023-10-11T06:13:47.575579+0000","last_peered":"2023-10-11T06:13:47.575579+0000","last_clean":"2023-10-11T06:13:47.575579+0000","last_became_active":"2023-10-11T06:13:46.429782+0000","last_became_peered":"2023-10-11T06:13:46.429782+0000","last_unstale":"2023-10-11T06:13:47.575579+0000","last_undegraded":"2023-10-11T06:13:47.575579+0000","last_fullsized":"2023-10-11T06:13:47.575579+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:30:07.936139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.404892+0000","last_change":"2023-10-11T06:13:45.942900+0000","last_active":"2023-10-11T06:13:46.404892+0000","last_peered":"2023-10-11T06:13:46.404892+0000","last_clean":"2023-10-11T06:13:46.404892+0000","last_became_active":"2023-10-11T06:13:45.942605+0000","last_became_peered":"2023-10-11T06:13:45.942605+0000","last_unstale":"2023-10-11T06:13:46.404892+0000","last_undegraded":"2023-10-11T06:13:46.404892+0000","last_fullsized":"2023-10-11T06:13:46.404892+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:41:13.469887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573738+0000","last_change":"2023-10-11T06:13:46.420847+0000","last_active":"2023-10-11T06:13:47.573738+0000","last_peered":"2023-10-11T06:13:47.573738+0000","last_clean":"2023-10-11T06:13:47.573738+0000","last_became_active":"2023-10-11T06:13:46.420249+0000","last_became_peered":"2023-10-11T06:13:46.420249+0000","last_unstale":"2023-10-11T06:13:47.573738+0000","last_undegraded":"2023-10-11T06:13:47.573738+0000","last_fullsized":"2023-10-11T06:13:47.573738+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:04:34.325228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.408570+0000","last_change":"2023-10-11T06:13:45.942277+0000","last_active":"2023-10-11T06:13:46.408570+0000","last_peered":"2023-10-11T06:13:46.408570+0000","last_clean":"2023-10-11T06:13:46.408570+0000","last_became_active":"2023-10-11T06:13:45.941982+0000","last_became_peered":"2023-10-11T06:13:45.941982+0000","last_unstale":"2023-10-11T06:13:46.408570+0000","last_undegraded":"2023-10-11T06:13:46.408570+0000","last_fullsized":"2023-10-11T06:13:46.408570+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:42:47.261508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575025+0000","last_change":"2023-10-11T06:13:46.420130+0000","last_active":"2023-10-11T06:13:47.575025+0000","last_peered":"2023-10-11T06:13:47.575025+0000","last_clean":"2023-10-11T06:13:47.575025+0000","last_became_active":"2023-10-11T06:13:46.419902+0000","last_became_peered":"2023-10-11T06:13:46.419902+0000","last_unstale":"2023-10-11T06:13:47.575025+0000","last_undegraded":"2023-10-11T06:13:47.575025+0000","last_fullsized":"2023-10-11T06:13:47.575025+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:49:31.986494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.416743+0000","last_change":"2023-10-11T06:13:45.943949+0000","last_active":"2023-10-11T06:13:46.416743+0000","last_peered":"2023-10-11T06:13:46.416743+0000","last_clean":"2023-10-11T06:13:46.416743+0000","last_became_active":"2023-10-11T06:13:45.942662+0000","last_became_peered":"2023-10-11T06:13:45.942662+0000","last_unstale":"2023-10-11T06:13:46.416743+0000","last_undegraded":"2023-10-11T06:13:46.416743+0000","last_fullsized":"2023-10-11T06:13:46.416743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:41:13.469887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407365+0000","last_change":"2023-10-11T06:13:46.411790+0000","last_active":"2023-10-11T06:13:47.407365+0000","last_peered":"2023-10-11T06:13:47.407365+0000","last_clean":"2023-10-11T06:13:47.407365+0000","last_became_active":"2023-10-11T06:13:46.411529+0000","last_became_peered":"2023-10-11T06:13:46.411529+0000","last_unstale":"2023-10-11T06:13:47.407365+0000","last_undegraded":"2023-10-11T06:13:47.407365+0000","last_fullsized":"2023-10-11T06:13:47.407365+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:35:05.134653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.408083+0000","last_change":"2023-10-11T06:13:45.947242+0000","last_active":"2023-10-11T06:13:46.408083+0000","last_peered":"2023-10-11T06:13:46.408083+0000","last_clean":"2023-10-11T06:13:46.408083+0000","last_became_active":"2023-10-11T06:13:45.946884+0000","last_became_peered":"2023-10-11T06:13:45.946884+0000","last_unstale":"2023-10-11T06:13:46.408083+0000","last_undegraded":"2023-10-11T06:13:46.408083+0000","last_fullsized":"2023-10-11T06:13:46.408083+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:50:00.126365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576121+0000","last_change":"2023-10-11T06:13:46.435831+0000","last_active":"2023-10-11T06:13:47.576121+0000","last_peered":"2023-10-11T06:13:47.576121+0000","last_clean":"2023-10-11T06:13:47.576121+0000","last_became_active":"2023-10-11T06:13:46.435343+0000","last_became_peered":"2023-10-11T06:13:46.435343+0000","last_unstale":"2023-10-11T06:13:47.576121+0000","last_undegraded":"2023-10-11T06:13:47.576121+0000","last_fullsized":"2023-10-11T06:13:47.576121+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:52:33.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.419014+0000","last_change":"2023-10-11T06:13:45.942103+0000","last_active":"2023-10-11T06:13:46.419014+0000","last_peered":"2023-10-11T06:13:46.419014+0000","last_clean":"2023-10-11T06:13:46.419014+0000","last_became_active":"2023-10-11T06:13:45.941827+0000","last_became_peered":"2023-10-11T06:13:45.941827+0000","last_unstale":"2023-10-11T06:13:46.419014+0000","last_undegraded":"2023-10-11T06:13:46.419014+0000","last_fullsized":"2023-10-11T06:13:46.419014+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:30:03.816589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.572913+0000","last_change":"2023-10-11T06:13:46.421561+0000","last_active":"2023-10-11T06:13:47.572913+0000","last_peered":"2023-10-11T06:13:47.572913+0000","last_clean":"2023-10-11T06:13:47.572913+0000","last_became_active":"2023-10-11T06:13:46.420938+0000","last_became_peered":"2023-10-11T06:13:46.420938+0000","last_unstale":"2023-10-11T06:13:47.572913+0000","last_undegraded":"2023-10-11T06:13:47.572913+0000","last_fullsized":"2023-10-11T06:13:47.572913+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:09:19.008454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.407067+0000","last_change":"2023-10-11T06:13:45.931103+0000","last_active":"2023-10-11T06:13:46.407067+0000","last_peered":"2023-10-11T06:13:46.407067+0000","last_clean":"2023-10-11T06:13:46.407067+0000","last_became_active":"2023-10-11T06:13:45.930710+0000","last_became_peered":"2023-10-11T06:13:45.930710+0000","last_unstale":"2023-10-11T06:13:46.407067+0000","last_undegraded":"2023-10-11T06:13:46.407067+0000","last_fullsized":"2023-10-11T06:13:46.407067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:07:51.398606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407194+0000","last_change":"2023-10-11T06:13:46.413160+0000","last_active":"2023-10-11T06:13:47.407194+0000","last_peered":"2023-10-11T06:13:47.407194+0000","last_clean":"2023-10-11T06:13:47.407194+0000","last_became_active":"2023-10-11T06:13:46.412564+0000","last_became_peered":"2023-10-11T06:13:46.412564+0000","last_unstale":"2023-10-11T06:13:47.407194+0000","last_undegraded":"2023-10-11T06:13:47.407194+0000","last_fullsized":"2023-10-11T06:13:47.407194+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:08:49.498643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.397031+0000","last_change":"2023-10-11T06:13:45.938114+0000","last_active":"2023-10-11T06:13:46.397031+0000","last_peered":"2023-10-11T06:13:46.397031+0000","last_clean":"2023-10-11T06:13:46.397031+0000","last_became_active":"2023-10-11T06:13:45.937799+0000","last_became_peered":"2023-10-11T06:13:45.937799+0000","last_unstale":"2023-10-11T06:13:46.397031+0000","last_undegraded":"2023-10-11T06:13:46.397031+0000","last_fullsized":"2023-10-11T06:13:46.397031+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:50:08.238522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407413+0000","last_change":"2023-10-11T06:13:46.424657+0000","last_active":"2023-10-11T06:13:47.407413+0000","last_peered":"2023-10-11T06:13:47.407413+0000","last_clean":"2023-10-11T06:13:47.407413+0000","last_became_active":"2023-10-11T06:13:46.422059+0000","last_became_peered":"2023-10-11T06:13:46.422059+0000","last_unstale":"2023-10-11T06:13:47.407413+0000","last_undegraded":"2023-10-11T06:13:47.407413+0000","last_fullsized":"2023-10-11T06:13:47.407413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:27:10.517623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.439443+0000","last_change":"2023-10-11T06:13:45.946231+0000","last_active":"2023-10-11T06:13:46.439443+0000","last_peered":"2023-10-11T06:13:46.439443+0000","last_clean":"2023-10-11T06:13:46.439443+0000","last_became_active":"2023-10-11T06:13:45.945168+0000","last_became_peered":"2023-10-11T06:13:45.945168+0000","last_unstale":"2023-10-11T06:13:46.439443+0000","last_undegraded":"2023-10-11T06:13:46.439443+0000","last_fullsized":"2023-10-11T06:13:46.439443+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:57:35.798111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.411681+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:13:46.411681+0000","last_peered":"2023-10-11T06:13:46.411681+0000","last_clean":"2023-10-11T06:13:46.411681+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:13:46.411681+0000","last_undegraded":"2023-10-11T06:13:46.411681+0000","last_fullsized":"2023-10-11T06:13:46.411681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576790+0000","last_change":"2023-10-11T06:13:46.437768+0000","last_active":"2023-10-11T06:13:47.576790+0000","last_peered":"2023-10-11T06:13:47.576790+0000","last_clean":"2023-10-11T06:13:47.576790+0000","last_became_active":"2023-10-11T06:13:46.436487+0000","last_became_peered":"2023-10-11T06:13:46.436487+0000","last_unstale":"2023-10-11T06:13:47.576790+0000","last_undegraded":"2023-10-11T06:13:47.576790+0000","last_fullsized":"2023-10-11T06:13:47.576790+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T13:26:24.488013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.405622+0000","last_change":"2023-10-11T06:13:45.935050+0000","last_active":"2023-10-11T06:13:46.405622+0000","last_peered":"2023-10-11T06:13:46.405622+0000","last_clean":"2023-10-11T06:13:46.405622+0000","last_became_active":"2023-10-11T06:13:45.934763+0000","last_became_peered":"2023-10-11T06:13:45.934763+0000","last_unstale":"2023-10-11T06:13:46.405622+0000","last_undegraded":"2023-10-11T06:13:46.405622+0000","last_fullsized":"2023-10-11T06:13:46.405622+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:12.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.408211+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:13:46.408211+0000","last_peered":"2023-10-11T06:13:46.408211+0000","last_clean":"2023-10-11T06:13:46.408211+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:13:46.408211+0000","last_undegraded":"2023-10-11T06:13:46.408211+0000","last_fullsized":"2023-10-11T06:13:46.408211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.576897+0000","last_change":"2023-10-11T06:13:46.435663+0000","last_active":"2023-10-11T06:13:47.576897+0000","last_peered":"2023-10-11T06:13:47.576897+0000","last_clean":"2023-10-11T06:13:47.576897+0000","last_became_active":"2023-10-11T06:13:46.435381+0000","last_became_peered":"2023-10-11T06:13:46.435381+0000","last_unstale":"2023-10-11T06:13:47.576897+0000","last_undegraded":"2023-10-11T06:13:47.576897+0000","last_fullsized":"2023-10-11T06:13:47.576897+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:47:52.646040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.425915+0000","last_change":"2023-10-11T06:13:45.934059+0000","last_active":"2023-10-11T06:13:46.425915+0000","last_peered":"2023-10-11T06:13:46.425915+0000","last_clean":"2023-10-11T06:13:46.425915+0000","last_became_active":"2023-10-11T06:13:45.933767+0000","last_became_peered":"2023-10-11T06:13:45.933767+0000","last_unstale":"2023-10-11T06:13:46.425915+0000","last_undegraded":"2023-10-11T06:13:46.425915+0000","last_fullsized":"2023-10-11T06:13:46.425915+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:43:12.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.422455+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:13:46.422455+0000","last_peered":"2023-10-11T06:13:46.422455+0000","last_clean":"2023-10-11T06:13:46.422455+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:13:46.422455+0000","last_undegraded":"2023-10-11T06:13:46.422455+0000","last_fullsized":"2023-10-11T06:13:46.422455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575340+0000","last_change":"2023-10-11T06:13:46.423342+0000","last_active":"2023-10-11T06:13:47.575340+0000","last_peered":"2023-10-11T06:13:47.575340+0000","last_clean":"2023-10-11T06:13:47.575340+0000","last_became_active":"2023-10-11T06:13:46.422623+0000","last_became_peered":"2023-10-11T06:13:46.422623+0000","last_unstale":"2023-10-11T06:13:47.575340+0000","last_undegraded":"2023-10-11T06:13:47.575340+0000","last_fullsized":"2023-10-11T06:13:47.575340+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:38:41.070650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.417195+0000","last_change":"2023-10-11T06:13:45.934173+0000","last_active":"2023-10-11T06:13:46.417195+0000","last_peered":"2023-10-11T06:13:46.417195+0000","last_clean":"2023-10-11T06:13:46.417195+0000","last_became_active":"2023-10-11T06:13:45.933853+0000","last_became_peered":"2023-10-11T06:13:45.933853+0000","last_unstale":"2023-10-11T06:13:46.417195+0000","last_undegraded":"2023-10-11T06:13:46.417195+0000","last_fullsized":"2023-10-11T06:13:46.417195+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:30:03.816589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.406461+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:13:46.406461+0000","last_peered":"2023-10-11T06:13:46.406461+0000","last_clean":"2023-10-11T06:13:46.406461+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:13:46.406461+0000","last_undegraded":"2023-10-11T06:13:46.406461+0000","last_fullsized":"2023-10-11T06:13:46.406461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574303+0000","last_change":"2023-10-11T06:13:46.428761+0000","last_active":"2023-10-11T06:13:47.574303+0000","last_peered":"2023-10-11T06:13:47.574303+0000","last_clean":"2023-10-11T06:13:47.574303+0000","last_became_active":"2023-10-11T06:13:46.428394+0000","last_became_peered":"2023-10-11T06:13:46.428394+0000","last_unstale":"2023-10-11T06:13:47.574303+0000","last_undegraded":"2023-10-11T06:13:47.574303+0000","last_fullsized":"2023-10-11T06:13:47.574303+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:10:04.609152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.414483+0000","last_change":"2023-10-11T06:13:45.940104+0000","last_active":"2023-10-11T06:13:46.414483+0000","last_peered":"2023-10-11T06:13:46.414483+0000","last_clean":"2023-10-11T06:13:46.414483+0000","last_became_active":"2023-10-11T06:13:45.939827+0000","last_became_peered":"2023-10-11T06:13:45.939827+0000","last_unstale":"2023-10-11T06:13:46.414483+0000","last_undegraded":"2023-10-11T06:13:46.414483+0000","last_fullsized":"2023-10-11T06:13:46.414483+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:26.580696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.415254+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:13:46.415254+0000","last_peered":"2023-10-11T06:13:46.415254+0000","last_clean":"2023-10-11T06:13:46.415254+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:13:46.415254+0000","last_undegraded":"2023-10-11T06:13:46.415254+0000","last_fullsized":"2023-10-11T06:13:46.415254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574519+0000","last_change":"2023-10-11T06:13:46.414454+0000","last_active":"2023-10-11T06:13:47.574519+0000","last_peered":"2023-10-11T06:13:47.574519+0000","last_clean":"2023-10-11T06:13:47.574519+0000","last_became_active":"2023-10-11T06:13:46.413186+0000","last_became_peered":"2023-10-11T06:13:46.413186+0000","last_unstale":"2023-10-11T06:13:47.574519+0000","last_undegraded":"2023-10-11T06:13:47.574519+0000","last_fullsized":"2023-10-11T06:13:47.574519+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:04:59.783970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.425199+0000","last_change":"2023-10-11T06:13:45.941660+0000","last_active":"2023-10-11T06:13:46.425199+0000","last_peered":"2023-10-11T06:13:46.425199+0000","last_clean":"2023-10-11T06:13:46.425199+0000","last_became_active":"2023-10-11T06:13:45.941395+0000","last_became_peered":"2023-10-11T06:13:45.941395+0000","last_unstale":"2023-10-11T06:13:46.425199+0000","last_undegraded":"2023-10-11T06:13:46.425199+0000","last_fullsized":"2023-10-11T06:13:46.425199+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:12:14.685368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.401929+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:13:46.401929+0000","last_peered":"2023-10-11T06:13:46.401929+0000","last_clean":"2023-10-11T06:13:46.401929+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:13:46.401929+0000","last_undegraded":"2023-10-11T06:13:46.401929+0000","last_fullsized":"2023-10-11T06:13:46.401929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574732+0000","last_change":"2023-10-11T06:13:46.423889+0000","last_active":"2023-10-11T06:13:47.574732+0000","last_peered":"2023-10-11T06:13:47.574732+0000","last_clean":"2023-10-11T06:13:47.574732+0000","last_became_active":"2023-10-11T06:13:46.423638+0000","last_became_peered":"2023-10-11T06:13:46.423638+0000","last_unstale":"2023-10-11T06:13:47.574732+0000","last_undegraded":"2023-10-11T06:13:47.574732+0000","last_fullsized":"2023-10-11T06:13:47.574732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:58:06.154499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.406624+0000","last_change":"2023-10-11T06:13:45.934165+0000","last_active":"2023-10-11T06:13:46.406624+0000","last_peered":"2023-10-11T06:13:46.406624+0000","last_clean":"2023-10-11T06:13:46.406624+0000","last_became_active":"2023-10-11T06:13:45.933906+0000","last_became_peered":"2023-10-11T06:13:45.933906+0000","last_unstale":"2023-10-11T06:13:46.406624+0000","last_undegraded":"2023-10-11T06:13:46.406624+0000","last_fullsized":"2023-10-11T06:13:46.406624+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:28:29.789596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.417566+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:13:46.417566+0000","last_peered":"2023-10-11T06:13:46.417566+0000","last_clean":"2023-10-11T06:13:46.417566+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:13:46.417566+0000","last_undegraded":"2023-10-11T06:13:46.417566+0000","last_fullsized":"2023-10-11T06:13:46.417566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.407277+0000","last_change":"2023-10-11T06:13:46.406090+0000","last_active":"2023-10-11T06:13:47.407277+0000","last_peered":"2023-10-11T06:13:47.407277+0000","last_clean":"2023-10-11T06:13:47.407277+0000","last_became_active":"2023-10-11T06:13:46.405793+0000","last_became_peered":"2023-10-11T06:13:46.405793+0000","last_unstale":"2023-10-11T06:13:47.407277+0000","last_undegraded":"2023-10-11T06:13:47.407277+0000","last_fullsized":"2023-10-11T06:13:47.407277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:45.900598+0000","last_clean_scrub_stamp":"2023-10-11T06:13:45.900598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:31:01.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.404358+0000","last_change":"2023-10-11T06:13:45.941240+0000","last_active":"2023-10-11T06:13:46.404358+0000","last_peered":"2023-10-11T06:13:46.404358+0000","last_clean":"2023-10-11T06:13:46.404358+0000","last_became_active":"2023-10-11T06:13:45.940899+0000","last_became_peered":"2023-10-11T06:13:45.940899+0000","last_unstale":"2023-10-11T06:13:46.404358+0000","last_undegraded":"2023-10-11T06:13:46.404358+0000","last_fullsized":"2023-10-11T06:13:46.404358+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:13:44.888736+0000","last_clean_scrub_stamp":"2023-10-11T06:13:44.888736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T17:53:25.614701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.414841+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:13:46.414841+0000","last_peered":"2023-10-11T06:13:46.414841+0000","last_clean":"2023-10-11T06:13:46.414841+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:13:46.414841+0000","last_undegraded":"2023-10-11T06:13:46.414841+0000","last_fullsized":"2023-10-11T06:13:46.414841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-11T06:13:46.417557+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:13:46.417557+0000","last_peered":"2023-10-11T06:13:46.417557+0000","last_clean":"2023-10-11T06:13:46.417557+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:13:46.417557+0000","last_undegraded":"2023-10-11T06:13:46.417557+0000","last_fullsized":"2023-10-11T06:13:46.417557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1245184,"data_stored":871857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.0609999999999999,"15min":1.0409999999999999},"min":{"1min":0.77200000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.466,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.079},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.171,"15min":1.161},"min":{"1min":0.83699999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6160000000000001,"5min":1.788,"15min":1.788},"last":1.2769999999999999}]},{"osd":1,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.1020000000000001,"15min":1.105},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6679999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.383},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1220000000000001,"15min":1.1279999999999999},"min":{"1min":0.67900000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6930000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.98799999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.68799999999999994,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4059999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.292},"min":{"1min":0.72799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.758,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.3560000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.24,"15min":1.248},"min":{"1min":0.86599999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6040000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.23,"5min":1.2589999999999999,"15min":1.264},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7969999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.196}]},{"osd":4,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.4059999999999999,"15min":1.4179999999999999},"min":{"1min":0.86799999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9139999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3129999999999999,"15min":1.3009999999999999},"min":{"1min":0.89600000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.8700000000000001,"5min":1.913,"15min":1.913},"last":0.90800000000000003}]},{"osd":5,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.389,"15min":1.395},"min":{"1min":0.86899999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0139999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.5680000000000001,"15min":1.5880000000000001},"min":{"1min":1.135,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0019999999999998,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.4510000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:13:04 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.478,"15min":1.476},"min":{"1min":1.131,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.069,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.5620000000000001,"15min":1.5640000000000001},"min":{"1min":1.0720000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.9830000000000001,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.1220000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":643072,"data_stored":281329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1080000000000001,"15min":1.111},"min":{"1min":0.78600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.306},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2090000000000001,"15min":1.2150000000000001},"min":{"1min":0.82699999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.49,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.8}]},{"osd":1,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.028,"15min":1.034},"min":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.143,"15min":1.161},"min":{"1min":0.70299999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.343,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.3720000000000001}]},{"osd":2,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0669999999999999,"15min":1.071},"min":{"1min":0.69799999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4039999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.21,"15min":1.2150000000000001},"min":{"1min":0.75600000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5669999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.1140000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.3029999999999999,"15min":1.3260000000000001},"min":{"1min":0.77800000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4750000000000001,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.2509999999999999},"min":{"1min":0.86099999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.526,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.6339999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.359,"15min":1.379},"min":{"1min":0.91100000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.583,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.444,"15min":1.466},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5780000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.248}]},{"osd":5,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.373,"15min":1.3919999999999999},"min":{"1min":0.89200000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.617,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.458,"15min":1.4670000000000001},"min":{"1min":1.1000000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.7809999999999999,"5min":1.992,"15min":1.992},"last":1.444}]},{"osd":7,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.5680000000000001,"15min":1.591},"min":{"1min":0.97599999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.659,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5249999999999999,"15min":1.5429999999999999},"min":{"1min":1.0349999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6890000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.7210000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":651264,"data_stored":281981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.997,"15min":0.995},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.476,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.01,"15min":1.0149999999999999},"min":{"1min":0.76700000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3620000000000001,"5min":1.621,"15min":1.621},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.048,"15min":1.056},"min":{"1min":0.71099999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.335,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.272},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.68100000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4710000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.222}]},{"osd":2,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.236,"15min":1.248},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.55,"5min":2.508,"15min":2.508},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.3300000000000001,"15min":1.359},"min":{"1min":0.81899999999999995,"5min":0.752,"15min":0.752},"max":{"1min":1.5009999999999999,"5min":2.524,"15min":2.524},"last":0.88600000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.155,"15min":1.153},"min":{"1min":0.74199999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5589999999999999,"5min":2.052,"15min":2.052},"last":1.073},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2849999999999999,"15min":1.3089999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.3819999999999999,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.276,"15min":1.286},"min":{"1min":0.77200000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.403,"5min":2.242,"15min":2.242},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.333,"5min":1.3620000000000001,"15min":1.367},"min":{"1min":0.92100000000000004,"5min":0.747,"15min":0.747},"max":{"1min":1.6519999999999999,"5min":2.427,"15min":2.427},"last":1.3180000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.401,"15min":1.4319999999999999},"min":{"1min":0.84599999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.514,"5min":2.472,"15min":2.472},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.45,"15min":1.46},"min":{"1min":0.97599999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7529999999999999,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.4359999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":1.071,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.579,"5min":2.617,"15min":2.617},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5620000000000001,"15min":1.5840000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7110000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.1619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":655360,"data_stored":282893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":1.0780000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3089999999999999,"5min":1.841,"15min":1.841},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.96299999999999997,"15min":0.95699999999999996},"min":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.413,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.244}]},{"osd":1,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.071,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.1180000000000001,"15min":1.1319999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.286}]},{"osd":2,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0469999999999999,"15min":1.0529999999999999},"min":{"1min":0.76300000000000001,"5min":0.63,"15min":0.63},"max":{"1min":1.4470000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.208,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.91300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.5069999999999999,"5min":1.744,"15min":1.744},"last":0.86199999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.244,"15min":1.262},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3779999999999999,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.21},{"interface":"front","average":{"1min":1.149,"5min":1.2050000000000001,"15min":1.2150000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5589999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.4119999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.321},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.825,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.018},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4039999999999999,"15min":1.4179999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.776,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.0449999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.323,"15min":1.339},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.575,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.45,"15min":1.456},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.6319999999999999,"5min":1.972,"15min":1.972},"last":1.482}]},{"osd":7,"last update":"Wed Oct 11 06:13:00 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5309999999999999,"15min":1.5580000000000001},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.069,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.385},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.4770000000000001,"15min":1.4990000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.675,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.331}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":651264,"data_stored":281385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1910000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.96499999999999997,"15min":0.97099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.474,"5min":2.097,"15min":2.097},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.89800000000000002,"15min":0.90900000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1299999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.997,"5min":1.0940000000000001,"15min":1.1100000000000001},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.655,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.04,"15min":1.0489999999999999},"min":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5940000000000001,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":1.05,"5min":1.1839999999999999,"15min":1.206},"min":{"1min":0.47199999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.417,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":0.5}]},{"osd":4,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.216,"15min":1.2310000000000001},"min":{"1min":0.753,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5740000000000001,"5min":3.286,"15min":3.286},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3,"15min":1.302},"min":{"1min":0.77100000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7889999999999999,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.84999999999999998}]},{"osd":5,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4159999999999999,"15min":1.427},"min":{"1min":0.86599999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.756,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.093},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":0.82699999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6850000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":0.80500000000000005}]},{"osd":6,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.385,"15min":1.391},"min":{"1min":0.79200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.113,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.024},{"interface":"front","average":{"1min":1.349,"5min":1.5449999999999999,"15min":1.577},"min":{"1min":0.73599999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9650000000000001,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":0.82799999999999996}]},{"osd":7,"last update":"Wed Oct 11 06:13:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3799999999999999,"15min":1.391},"min":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9099999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4179999999999999,"15min":1.4199999999999999},"min":{"1min":0.88100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8620000000000001,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.95999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":651264,"data_stored":281385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.81499999999999995,"15min":0.83299999999999996},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.77800000000000002,"15min":0.79300000000000004},"min":{"1min":0.36699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.86099999999999999,"15min":0.87},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1299999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.51800000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.91100000000000003,"15min":0.92600000000000005},"min":{"1min":0.50600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.3480000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.1890000000000001,"15min":1.2370000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.573,"5min":3.032,"15min":3.032},"last":0.505}]},{"osd":4,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.6579999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.147,"5min":1.248,"15min":1.2649999999999999},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.589,"5min":3.069,"15min":3.069},"last":0.84299999999999997}]},{"osd":5,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.234,"15min":1.248},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.0030000000000001,"5min":2.181,"15min":2.181},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.3620000000000001,"15min":1.3979999999999999},"min":{"1min":0.81499999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4350000000000001,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.3460000000000001,"15min":1.365},"min":{"1min":0.80100000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.571,"5min":2.633,"15min":2.633},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":0.78400000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.1539999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.1499999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.399,"15min":1.4259999999999999},"min":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6599999999999999,"5min":2.794,"15min":2.794},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.319,"15min":1.337},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.081,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.1850000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":647168,"data_stored":281419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81000000000000005,"15min":0.80900000000000005},"min":{"1min":0.38700000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56999999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.2170000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.84499999999999997,"15min":0.86699999999999999},"min":{"1min":0.53400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.92000000000000004,"15min":0.91600000000000004},"min":{"1min":0.505,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.319,"5min":1.768,"15min":1.768},"last":0.501}]},{"osd":3,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.129,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.98799999999999999,"15min":1.002},"min":{"1min":0.56000000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.454,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2829999999999999,"15min":1.294},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.738,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.327,"15min":1.351},"min":{"1min":0.81799999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6279999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.0009999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2849999999999999,"15min":1.3009999999999999},"min":{"1min":0.92000000000000004,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.694,"5min":3.319,"15min":3.319},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.212,"5min":1.3109999999999999,"15min":1.327},"min":{"1min":0.77200000000000002,"5min":0.755,"15min":0.755},"max":{"1min":1.5509999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3160000000000001,"15min":1.3320000000000001},"min":{"1min":1.0369999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5580000000000001,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.298,"15min":1.304},"min":{"1min":0.86799999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6679999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:13:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.407,"15min":1.427},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.6859999999999999,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.367,"5min":1.423,"15min":1.4319999999999999},"min":{"1min":0.98399999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":1.766,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.0229999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1253376,"data_stored":873843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.874},"min":{"1min":0.47599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1819999999999999,"5min":1.329,"15min":1.329},"last":1.427},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.038,"15min":1.0760000000000001},"min":{"1min":0.49099999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.083,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.50900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.252,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.98299999999999998,"15min":1.0089999999999999},"min":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2410000000000001,"5min":2.444,"15min":2.444},"last":1.278}]},{"osd":3,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.873},"min":{"1min":0.504,"5min":0.31,"15min":0.31},"max":{"1min":1.165,"5min":1.474,"15min":1.474},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.123,"15min":1.133},"min":{"1min":0.58399999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.474,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.514}]},{"osd":4,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.331,"15min":1.341},"min":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.665,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.367},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2969999999999999,"15min":1.3009999999999999},"min":{"1min":0.93600000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.679,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.256}]},{"osd":5,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2410000000000001,"15min":1.2490000000000001},"min":{"1min":0.92100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.7929999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.05},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3,"15min":1.3009999999999999},"min":{"1min":0.91000000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.671,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.5640000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3640000000000001,"15min":1.3700000000000001},"min":{"1min":0.879,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5860000000000001,"5min":2.552,"15min":2.552},"last":1.605},{"interface":"front","average":{"1min":1.236,"5min":1.3640000000000001,"15min":1.3859999999999999},"min":{"1min":0.86499999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.5940000000000001,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.0109999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3779999999999999,"15min":1.3999999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.702,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.452},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3500000000000001,"15min":1.367},"min":{"1min":0.81299999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.4419999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.2030000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:13:51.300 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-11T06:13:51.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-11T06:13:51.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.561+0000 7f80186c7640 1 Processor -- start 2023-10-11T06:13:51.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.561+0000 7f80186c7640 1 -- start start 2023-10-11T06:13:51.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.562+0000 7f80186c7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f80101591b0 0x7f801015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.562+0000 7f80186c7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 0x7f80100cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.562+0000 7f80186c7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801006d6e0 con 0x7f80100cd2b0 2023-10-11T06:13:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.562+0000 7f80186c7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8010159580 con 0x7f80101591b0 2023-10-11T06:13:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.562+0000 7f80186c7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801006b030 con 0x7f801015d930 2023-10-11T06:13:51.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8016c64640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f80100cd2b0 0x7f80100c9fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54520/0 (socket says 172.21.15.84:54520) 2023-10-11T06:13:51.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8016c64640 1 -- 172.21.15.84:0/3844142034 learned_addr learned my addr 172.21.15.84:0/3844142034 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:51.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8015c62640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 0x7f80100cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8016463640 1 --2- 172.21.15.84:0/3844142034 >> v2:172.21.15.99:3300/0 conn(0x7f80101591b0 0x7f801015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329013949 0 0) 0x7f801006d6e0 con 0x7f80100cd2b0 2023-10-11T06:13:51.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ffc005730 con 0x7f80100cd2b0 2023-10-11T06:13:51.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1896410114 0 0) 0x7f7ffc005730 con 0x7f80100cd2b0 2023-10-11T06:13:51.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.563+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 msgr2=0x7f80100cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:51.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 0x7f80100cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 >> v2:172.21.15.99:3300/0 conn(0x7f80101591b0 msgr2=0x7f801015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:51.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 --2- 172.21.15.84:0/3844142034 >> v2:172.21.15.99:3300/0 conn(0x7f80101591b0 0x7f801015d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801015ccc0 con 0x7f80100cd2b0 2023-10-11T06:13:51.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f80080019f0 con 0x7f80100cd2b0 2023-10-11T06:13:51.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 >> v1:172.21.15.84:6789/0 conn(0x7f80100cd2b0 legacy=0x7f80100c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 0x7f7ffc005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ffc006150 0x7f7ffc0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f7ffc007380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8010159580 con 0x7f7ffc006f10 2023-10-11T06:13:51.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801006b030 con 0x7f7ffc006150 2023-10-11T06:13:51.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc00db90 con 0x7f801015d930 2023-10-11T06:13:51.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8016c64640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f7ffc007380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8016463640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 0x7f7ffc005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8016c64640 1 -- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 msgr2=0x7f7ffc005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:51.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8016c64640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015d930 0x7f7ffc005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8016c64640 1 -- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ffc006150 msgr2=0x7f7ffc0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:51.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8016c64640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ffc006150 0x7f7ffc0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.564+0000 7f8016c64640 1 -- 172.21.15.84:0/3844142034 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc005730 con 0x7f7ffc006f10 2023-10-11T06:13:51.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.565+0000 7f8016c64640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f7ffc007380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f800800a790 tx=0x7f7ffc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869f7d9e8764c0f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:51.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008034630 con 0x7f7ffc006f10 2023-10-11T06:13:51.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800802f070 con 0x7f7ffc006f10 2023-10-11T06:13:51.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f8015461640 1 -- 172.21.15.84:0/3844142034 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008034a50 con 0x7f7ffc006f10 2023-10-11T06:13:51.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f80186c7640 1 -- 172.21.15.84:0/3844142034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 msgr2=0x7f7ffc007380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f80186c7640 1 --2- 172.21.15.84:0/3844142034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f7ffc007380 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f800800a790 tx=0x7f7ffc007a00 comp rx=0 tx=0).stop 2023-10-11T06:13:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f8016c64640 1 -- 172.21.15.84:0/3844142034 reap_dead start 2023-10-11T06:13:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f80186c7640 1 -- 172.21.15.84:0/3844142034 shutdown_connections 2023-10-11T06:13:51.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f80186c7640 1 -- 172.21.15.84:0/3844142034 >> 172.21.15.84:0/3844142034 conn(0x7f80100b16c0 msgr2=0x7f80100ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:51.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f80186c7640 1 -- 172.21.15.84:0/3844142034 shutdown_connections 2023-10-11T06:13:51.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.566+0000 7f80186c7640 1 -- 172.21.15.84:0/3844142034 wait complete. 2023-10-11T06:13:51.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.568+0000 7f80186c7640 1 Processor -- start 2023-10-11T06:13:51.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.568+0000 7f80186c7640 1 -- start start 2023-10-11T06:13:51.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f80186c7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f80100cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f80186c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f801015a140 0x7f80100af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f80186c7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015a5b0 0x7f80100af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f80186c7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8010070b20 con 0x7f7ffc006f10 2023-10-11T06:13:51.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f80186c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8010070e00 con 0x7f801015a140 2023-10-11T06:13:51.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f80186c7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80100710e0 con 0x7f801015a5b0 2023-10-11T06:13:51.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f8016463640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f80100cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f8016463640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f80100cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55002/0 (socket says 172.21.15.84:55002) 2023-10-11T06:13:51.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.569+0000 7f8016463640 1 -- 172.21.15.84:0/2750862638 learned_addr learned my addr 172.21.15.84:0/2750862638 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:51.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8016463640 1 -- 172.21.15.84:0/2750862638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015a5b0 msgr2=0x7f80100af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8016463640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015a5b0 0x7f80100af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8016463640 1 -- 172.21.15.84:0/2750862638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f801015a140 msgr2=0x7f80100af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8016463640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f801015a140 0x7f80100af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8016463640 1 -- 172.21.15.84:0/2750862638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100713c0 con 0x7f7ffc006f10 2023-10-11T06:13:51.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8016463640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f80100cab40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f800c0116d0 tx=0x7f800c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:51.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8006ffd640 1 -- 172.21.15.84:0/2750862638 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c00f940 con 0x7f7ffc006f10 2023-10-11T06:13:51.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8006ffd640 1 -- 172.21.15.84:0/2750862638 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c00faa0 con 0x7f7ffc006f10 2023-10-11T06:13:51.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f8006ffd640 1 -- 172.21.15.84:0/2750862638 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c00fe10 con 0x7f7ffc006f10 2023-10-11T06:13:51.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.570+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80100707d0 con 0x7f7ffc006f10 2023-10-11T06:13:51.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.571+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80100cb260 con 0x7f7ffc006f10 2023-10-11T06:13:51.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.572+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801006d240 con 0x7f7ffc006f10 2023-10-11T06:13:51.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.573+0000 7f8006ffd640 1 -- 172.21.15.84:0/2750862638 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f800c015460 con 0x7f7ffc006f10 2023-10-11T06:13:51.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.574+0000 7f8006ffd640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7ff406a830 0x7f7ff406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:51.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.574+0000 7f8006ffd640 1 -- 172.21.15.84:0/2750862638 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f800c089350 con 0x7f7ffc006f10 2023-10-11T06:13:51.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.574+0000 7f8015c62640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7ff406a830 0x7f7ff406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:51.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.574+0000 7f8015c62640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7ff406a830 0x7f7ff406cce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8000009a20 tx=0x7f80000094e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:51.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.578+0000 7f8006ffd640 1 -- 172.21.15.84:0/2750862638 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f800c0545c0 con 0x7f7ffc006f10 2023-10-11T06:13:51.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.831+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f80100cb260 con 0x7f7ffc006f10 2023-10-11T06:13:51.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.840+0000 7f8006ffd640 1 -- 172.21.15.84:0/2750862638 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f80100cb260 con 0x7f7ffc006f10 2023-10-11T06:13:51.842 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:13:51.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7ff406a830 msgr2=0x7f7ff406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7ff406a830 0x7f7ff406cce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8000009a20 tx=0x7f80000094e0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 msgr2=0x7f80100cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:51.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f80100cab40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f800c0116d0 tx=0x7f800c011b90 comp rx=0 tx=0).stop 2023-10-11T06:13:51.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 shutdown_connections 2023-10-11T06:13:51.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7ff406a830 0x7f7ff406cce0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f801015a5b0 0x7f80100af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f801015a140 0x7f80100af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 --2- 172.21.15.84:0/2750862638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ffc006f10 0x7f80100cab40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:51.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.842+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 >> 172.21.15.84:0/2750862638 conn(0x7f80100b16c0 msgr2=0x7f801006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:51.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.843+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 shutdown_connections 2023-10-11T06:13:51.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:51.843+0000 7f80186c7640 1 -- 172.21.15.84:0/2750862638 wait complete. 2023-10-11T06:13:51.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:13:51.895 INFO:tasks.ceph.mds.c.smithi084.stderr:2023-10-11T06:13:51.893+0000 7f5dd0408640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T06:13:52.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.072+0000 7f7661a26640 1 Processor -- start 2023-10-11T06:13:52.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.073+0000 7f7661a26640 1 -- start start 2023-10-11T06:13:52.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f7661a26640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f765c15c290 0x7f765c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f7661a26640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f765c0d25e0 0x7f765c0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f7661a26640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c0d2a60 con 0x7f765c0d0a50 2023-10-11T06:13:52.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f7661a26640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c0d0e90 con 0x7f765c0d25e0 2023-10-11T06:13:52.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f7661a26640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c06cd10 con 0x7f765c15c290 2023-10-11T06:13:52.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f765a7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f765c0d25e0 0x7f765c0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f765a7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f765c0d25e0 0x7f765c0d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55014/0 (socket says 172.21.15.84:55014) 2023-10-11T06:13:52.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.074+0000 7f765a7fc640 1 -- 172.21.15.84:0/3023591220 learned_addr learned my addr 172.21.15.84:0/3023591220 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:52.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.075+0000 7f765a7fc640 1 -- 172.21.15.84:0/3023591220 >> v2:172.21.15.99:3300/0 conn(0x7f765c15c290 msgr2=0x7f765c0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:52.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.075+0000 7f765affd640 1 --2- 172.21.15.84:0/3023591220 >> v2:172.21.15.99:3300/0 conn(0x7f765c15c290 0x7f765c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.075+0000 7f765a7fc640 1 --2- 172.21.15.84:0/3023591220 >> v2:172.21.15.99:3300/0 conn(0x7f765c15c290 0x7f765c0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.075+0000 7f765a7fc640 1 -- 172.21.15.84:0/3023591220 >> v1:172.21.15.99:6789/0 conn(0x7f765c0d0a50 legacy=0x7f765c06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:52.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.075+0000 7f765a7fc640 1 -- 172.21.15.84:0/3023591220 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0afb60 con 0x7f765c0d25e0 2023-10-11T06:13:52.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.076+0000 7f765a7fc640 1 --2- 172.21.15.84:0/3023591220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d25e0 0x7f765c0d0510 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f765c072380 tx=0x7f764402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f8be4ab94a7b91 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:52.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.076+0000 7f7659ffb640 1 -- 172.21.15.84:0/3023591220 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76440317e0 con 0x7f765c0d25e0 2023-10-11T06:13:52.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.076+0000 7f7659ffb640 1 -- 172.21.15.84:0/3023591220 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644031940 con 0x7f765c0d25e0 2023-10-11T06:13:52.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.076+0000 7f7659ffb640 1 -- 172.21.15.84:0/3023591220 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644031cd0 con 0x7f765c0d25e0 2023-10-11T06:13:52.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 -- 172.21.15.84:0/3023591220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d25e0 msgr2=0x7f765c0d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 --2- 172.21.15.84:0/3023591220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d25e0 0x7f765c0d0510 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f765c072380 tx=0x7f764402b880 comp rx=0 tx=0).stop 2023-10-11T06:13:52.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 -- 172.21.15.84:0/3023591220 shutdown_connections 2023-10-11T06:13:52.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 --2- 172.21.15.84:0/3023591220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d25e0 0x7f765c0d0510 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 --2- 172.21.15.84:0/3023591220 >> v2:172.21.15.99:3300/0 conn(0x7f765c15c290 0x7f765c0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 -- 172.21.15.84:0/3023591220 >> 172.21.15.84:0/3023591220 conn(0x7f765c071cc0 msgr2=0x7f765c09bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:52.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 -- 172.21.15.84:0/3023591220 shutdown_connections 2023-10-11T06:13:52.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.077+0000 7f7661a26640 1 -- 172.21.15.84:0/3023591220 wait complete. 2023-10-11T06:13:52.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.079+0000 7f7661a26640 1 Processor -- start 2023-10-11T06:13:52.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.079+0000 7f7661a26640 1 -- start start 2023-10-11T06:13:52.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f7661a26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d0a50 0x7f765c0a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f7661a26640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f765c0d25e0 0x7f765c09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f7661a26640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f765c15c290 0x7f765c0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f7661a26640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c09db80 con 0x7f765c0d0a50 2023-10-11T06:13:52.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f7661a26640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c09de60 con 0x7f765c15c290 2023-10-11T06:13:52.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f7661a26640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c09e140 con 0x7f765c0d25e0 2023-10-11T06:13:52.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d0a50 0x7f765c0a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d0a50 0x7f765c0a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55026/0 (socket says 172.21.15.84:55026) 2023-10-11T06:13:52.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 -- 172.21.15.84:0/946056016 learned_addr learned my addr 172.21.15.84:0/946056016 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:52.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 -- 172.21.15.84:0/946056016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f765c0d25e0 msgr2=0x7f765c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f765c0d25e0 0x7f765c09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 -- 172.21.15.84:0/946056016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f765c15c290 msgr2=0x7f765c0cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f765c15c290 0x7f765c0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 -- 172.21.15.84:0/946056016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440094d0 con 0x7f765c0d0a50 2023-10-11T06:13:52.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.080+0000 7f765affd640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d0a50 0x7f765c0a02c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f764c00ccd0 tx=0x7f764c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:52.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.081+0000 7f7653fff640 1 -- 172.21.15.84:0/946056016 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c003990 con 0x7f765c0d0a50 2023-10-11T06:13:52.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.081+0000 7f7653fff640 1 -- 172.21.15.84:0/946056016 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c003af0 con 0x7f765c0d0a50 2023-10-11T06:13:52.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.081+0000 7f7653fff640 1 -- 172.21.15.84:0/946056016 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c004020 con 0x7f765c0d0a50 2023-10-11T06:13:52.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.081+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c09e420 con 0x7f765c0d0a50 2023-10-11T06:13:52.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.081+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c09f160 con 0x7f765c0d0a50 2023-10-11T06:13:52.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.082+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c09e940 con 0x7f765c0d0a50 2023-10-11T06:13:52.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.083+0000 7f7653fff640 1 -- 172.21.15.84:0/946056016 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f764c004180 con 0x7f765c0d0a50 2023-10-11T06:13:52.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.084+0000 7f7653fff640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f763c06a830 0x7f763c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.084+0000 7f7653fff640 1 -- 172.21.15.84:0/946056016 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f764c0893c0 con 0x7f765c0d0a50 2023-10-11T06:13:52.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.084+0000 7f765a7fc640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f763c06a830 0x7f763c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.085+0000 7f765a7fc640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f763c06a830 0x7f763c06cce0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7644036000 tx=0x7f7644030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:52.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.088+0000 7f7653fff640 1 -- 172.21.15.84:0/946056016 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f765c09e940 con 0x7f765c0d0a50 2023-10-11T06:13:52.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.314+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f765c09e940 con 0x7f765c0d0a50 2023-10-11T06:13:52.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.316+0000 7f7653fff640 1 -- 172.21.15.84:0/946056016 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f765c09e940 con 0x7f765c0d0a50 2023-10-11T06:13:52.318 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:52.318 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1835716973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1835716973},{"type":"v1","addr":"172.21.15.99:6833","nonce":1835716973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1415879342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1415879342},{"type":"v1","addr":"172.21.15.99:6835","nonce":1415879342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2122307729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2122307729},{"type":"v1","addr":"172.21.15.84:6836","nonce":2122307729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:13:46.389013+0000","modified":"2023-10-11T06:13:49.406181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6027":{"gid":6027,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2048038445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2048038445},{"type":"v1","addr":"172.21.15.84:6837","nonce":2048038445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T06:13:52.319 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-10-11T06:13:52.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f763c06a830 msgr2=0x7f763c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f763c06a830 0x7f763c06cce0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7644036000 tx=0x7f7644030000 comp rx=0 tx=0).stop 2023-10-11T06:13:52.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d0a50 msgr2=0x7f765c0a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d0a50 0x7f765c0a02c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f764c00ccd0 tx=0x7f764c00a5c0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 shutdown_connections 2023-10-11T06:13:52.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f763c06a830 0x7f763c06cce0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f765c15c290 0x7f765c0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f765c0d25e0 0x7f765c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 --2- 172.21.15.84:0/946056016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0d0a50 0x7f765c0a02c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.318+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 >> 172.21.15.84:0/946056016 conn(0x7f765c071cc0 msgr2=0x7f765c0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:52.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.319+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 shutdown_connections 2023-10-11T06:13:52.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.319+0000 7f7661a26640 1 -- 172.21.15.84:0/946056016 wait complete. 2023-10-11T06:13:52.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-11T06:13:46.389013+0000', 'modified': '2023-10-11T06:13:49.406181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6027}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6027': {'gid': 6027, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6837/2048038445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6835', 'nonce': 2048038445}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 2048038445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-11T06:13:52.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-11T06:13:52.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-11T06:13:52.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.970+0000 7fbf9ffff640 1 Processor -- start 2023-10-11T06:13:52.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.971+0000 7fbf9ffff640 1 -- start start 2023-10-11T06:13:52.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfa00cf750 0x7fbfa0160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00d0300 0x7fbfa00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9ffff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa015fc20 con 0x7fbfa00d2630 2023-10-11T06:13:52.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9ffff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa00d0060 con 0x7fbfa00cf750 2023-10-11T06:13:52.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9ffff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01601e0 con 0x7fbfa00d0300 2023-10-11T06:13:52.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfa00cf750 0x7fbfa0160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9effd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfa00cf750 0x7fbfa0160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55038/0 (socket says 172.21.15.84:55038) 2023-10-11T06:13:52.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.972+0000 7fbf9effd640 1 -- 172.21.15.84:0/2603455250 learned_addr learned my addr 172.21.15.84:0/2603455250 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:52.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.973+0000 7fbf9effd640 1 -- 172.21.15.84:0/2603455250 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00d0300 msgr2=0x7fbfa00cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:52.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.973+0000 7fbf9effd640 1 --2- 172.21.15.84:0/2603455250 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00d0300 0x7fbfa00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.973+0000 7fbf9effd640 1 -- 172.21.15.84:0/2603455250 >> v1:172.21.15.99:6789/0 conn(0x7fbfa00d2630 legacy=0x7fbfa00d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:52.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.973+0000 7fbf9effd640 1 -- 172.21.15.84:0/2603455250 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa015c510 con 0x7fbfa00cf750 2023-10-11T06:13:52.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.973+0000 7fbf9effd640 1 --2- 172.21.15.84:0/2603455250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00cf750 0x7fbfa0160670 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009850 tx=0x7fbf9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559e5f8ecaaf2ac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:52.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9dffb640 1 -- 172.21.15.84:0/2603455250 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90004230 con 0x7fbfa00cf750 2023-10-11T06:13:52.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9dffb640 1 -- 172.21.15.84:0/2603455250 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf90004390 con 0x7fbfa00cf750 2023-10-11T06:13:52.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9dffb640 1 -- 172.21.15.84:0/2603455250 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90031830 con 0x7fbfa00cf750 2023-10-11T06:13:52.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9ffff640 1 -- 172.21.15.84:0/2603455250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00cf750 msgr2=0x7fbfa0160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/2603455250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00cf750 0x7fbfa0160670 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009850 tx=0x7fbf9002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9ffff640 1 -- 172.21.15.84:0/2603455250 shutdown_connections 2023-10-11T06:13:52.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/2603455250 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00d0300 0x7fbfa00cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/2603455250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00cf750 0x7fbfa0160670 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9ffff640 1 -- 172.21.15.84:0/2603455250 >> 172.21.15.84:0/2603455250 conn(0x7fbfa0071cc0 msgr2=0x7fbfa009bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:52.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.974+0000 7fbf9ffff640 1 -- 172.21.15.84:0/2603455250 shutdown_connections 2023-10-11T06:13:52.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.975+0000 7fbf9ffff640 1 -- 172.21.15.84:0/2603455250 wait complete. 2023-10-11T06:13:52.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.976+0000 7fbf9ffff640 1 Processor -- start 2023-10-11T06:13:52.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.976+0000 7fbf9ffff640 1 -- start start 2023-10-11T06:13:52.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00cf750 0x7fbfa015d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfa00d0300 0x7fbfa0150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00d2630 0x7fbfa015e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9ffff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa015c720 con 0x7fbfa00d2630 2023-10-11T06:13:52.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9ffff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa015c970 con 0x7fbfa00d0300 2023-10-11T06:13:52.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9ffff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa015cc50 con 0x7fbfa00cf750 2023-10-11T06:13:52.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00d2630 0x7fbfa015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00d2630 0x7fbfa015e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55040/0 (socket says 172.21.15.84:55040) 2023-10-11T06:13:52.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.977+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/3618385903 learned_addr learned my addr 172.21.15.84:0/3618385903 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:52.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/3618385903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00cf750 msgr2=0x7fbfa015d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9f7fe640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00cf750 0x7fbfa015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/3618385903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfa00d0300 msgr2=0x7fbfa0150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9f7fe640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfa00d0300 0x7fbfa0150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/3618385903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90009460 con 0x7fbfa00d2630 2023-10-11T06:13:52.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9f7fe640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00d2630 0x7fbfa015e980 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b990 tx=0x7fbf9800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:52.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf977fe640 1 -- 172.21.15.84:0/3618385903 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800cba0 con 0x7fbfa00d2630 2023-10-11T06:13:52.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf977fe640 1 -- 172.21.15.84:0/3618385903 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9800cd00 con 0x7fbfa00d2630 2023-10-11T06:13:52.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa015cf30 con 0x7fbfa00d2630 2023-10-11T06:13:52.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf977fe640 1 -- 172.21.15.84:0/3618385903 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98014680 con 0x7fbfa00d2630 2023-10-11T06:13:52.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.978+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa015eec0 con 0x7fbfa00d2630 2023-10-11T06:13:52.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.981+0000 7fbf977fe640 1 -- 172.21.15.84:0/3618385903 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbf9801d3b0 con 0x7fbfa00d2630 2023-10-11T06:13:52.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.981+0000 7fbf977fe640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbf8006a830 0x7fbf8006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.982+0000 7fbf977fe640 1 -- 172.21.15.84:0/3618385903 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf98088fa0 con 0x7fbfa00d2630 2023-10-11T06:13:52.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.982+0000 7fbf9effd640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbf8006a830 0x7fbf8006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.982+0000 7fbfa807c740 1 Processor -- start 2023-10-11T06:13:52.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.982+0000 7fbfa807c740 1 -- start start 2023-10-11T06:13:52.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.982+0000 7fbf9effd640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbf8006a830 0x7fbf8006cce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbf90035000 tx=0x7fbf90030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:52.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e464e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e467e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 0x5576e467edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbfa807c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e462a250 con 0x5576e46776d0 2023-10-11T06:13:52.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e45bef80 con 0x5576e464ef20 2023-10-11T06:13:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576e45bc710 con 0x5576e4677300 2023-10-11T06:13:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbf9e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e467e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbf9e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e467e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55056/0 (socket says 172.21.15.84:55056) 2023-10-11T06:13:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.983+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/4043685100 learned_addr learned my addr 172.21.15.84:0/4043685100 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:52.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9effd640 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e464e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 msgr2=0x5576e464e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:52.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e464e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9f7fe640 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 0x5576e467edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:52.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 msgr2=0x5576e467edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:52.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 0x5576e467edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/4043685100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e457f690 con 0x5576e46776d0 2023-10-11T06:13:52.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e467e870 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf880097e0 tx=0x7fbf88009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb1b6661bb74758 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:52.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf94ff9640 1 -- 172.21.15.84:0/4043685100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8800bad0 con 0x5576e46776d0 2023-10-11T06:13:52.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf94ff9640 1 -- 172.21.15.84:0/4043685100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8800bc30 con 0x5576e46776d0 2023-10-11T06:13:52.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.984+0000 7fbf94ff9640 1 -- 172.21.15.84:0/4043685100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88011420 con 0x5576e46776d0 2023-10-11T06:13:52.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 -- 172.21.15.84:0/4043685100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 msgr2=0x5576e467e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:52.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e467e870 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf880097e0 tx=0x7fbf88009ca0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 -- 172.21.15.84:0/4043685100 shutdown_connections 2023-10-11T06:13:52.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 0x5576e467edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e467e870 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 --2- 172.21.15.84:0/4043685100 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e464e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:52.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 -- 172.21.15.84:0/4043685100 >> 172.21.15.84:0/4043685100 conn(0x5576e4650cf0 msgr2=0x5576e4699870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:52.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 -- 172.21.15.84:0/4043685100 shutdown_connections 2023-10-11T06:13:52.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.985+0000 7fbfa807c740 1 -- 172.21.15.84:0/4043685100 wait complete. 2023-10-11T06:13:52.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.986+0000 7fbfa807c740 1 -- start start 2023-10-11T06:13:52.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.986+0000 7fbfa807c740 1 Processor -- start 2023-10-11T06:13:52.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e464ef20 0x5576e467a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e4664d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 0x5576e4662480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbfa807c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e45b8000 con 0x5576e464ef20 2023-10-11T06:13:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e45597d0 con 0x5576e46776d0 2023-10-11T06:13:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576e4559470 con 0x5576e4677300 2023-10-11T06:13:52.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbfa807c740 10 client.0 _ll_register_callbacks cb 0x5576e45a81b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T06:13:53.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbf9effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e464ef20 0x5576e467a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbf9effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e464ef20 0x5576e467a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55060/0 (socket says 172.21.15.84:55060) 2023-10-11T06:13:53.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.987+0000 7fbf9effd640 1 -- 172.21.15.84:0/1661059908 learned_addr learned my addr 172.21.15.84:0/1661059908 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:53.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf9effd640 1 -- 172.21.15.84:0/1661059908 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 msgr2=0x5576e4664d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf9effd640 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e4664d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf9effd640 1 -- 172.21.15.84:0/1661059908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 msgr2=0x5576e4662480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf9effd640 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 0x5576e4662480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf9effd640 1 -- 172.21.15.84:0/1661059908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf880093b0 con 0x5576e464ef20 2023-10-11T06:13:53.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf9effd640 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e464ef20 0x5576e467a1f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009970 tx=0x7fbf90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b153871480474c82 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:53.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-11T06:13:53.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf7dffb640 1 -- 172.21.15.84:0/1661059908 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90003810 con 0x5576e464ef20 2023-10-11T06:13:53.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf7dffb640 1 -- 172.21.15.84:0/1661059908 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf90041070 con 0x5576e464ef20 2023-10-11T06:13:53.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576e45e2550 con 0x5576e464ef20 2023-10-11T06:13:53.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbf7dffb640 1 -- 172.21.15.84:0/1661059908 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf90003970 con 0x5576e464ef20 2023-10-11T06:13:53.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbfa807c740 10 client.6165 fetch_fsmap learned FSMap version 24 2023-10-11T06:13:53.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.988+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5576e45b6680 con 0x5576e464ef20 2023-10-11T06:13:53.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbf7dffb640 1 -- 172.21.15.84:0/1661059908 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5576e45e2550 con 0x5576e464ef20 2023-10-11T06:13:53.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbf7dffb640 1 -- 172.21.15.84:0/1661059908 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbf90044040 con 0x5576e464ef20 2023-10-11T06:13:53.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbfa807c740 10 client.6165 fetch_fsmap finished waiting for FSMap version 24 2023-10-11T06:13:53.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbfa807c740 10 client.6165 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T06:13:53.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbfa807c740 20 client.6165 populate_metadata read hostname 'smithi084' 2023-10-11T06:13:53.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbfa807c740 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e4673780 0x5576e4678f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbfa807c740 4 client.6165 mds_command: new command op to 6027 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-11T06:13:53.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 --> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576e45aab50 con 0x5576e4673780 2023-10-11T06:13:53.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.989+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e4673780 0x5576e4678f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.990+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e4673780 0x5576e4678f40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ec2e48eff71418 server_cookie=daf61c53bba69d5f in_seq=0 out_seq=0 2023-10-11T06:13:53.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.990+0000 7fbf7dffb640 10 client.6165 ms_handle_connect on v2:172.21.15.84:6835/2048038445 2023-10-11T06:13:53.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.990+0000 7fbf7dffb640 1 -- 172.21.15.84:0/1661059908 <== mds.0 v2:172.21.15.84:6835/2048038445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5576e45aab50 con 0x5576e4673780 2023-10-11T06:13:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.990+0000 7fbf7dffb640 10 client.6165 handle_command_reply: tid=0 2023-10-11T06:13:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.990+0000 7fbfa807c740 1 client.6165 shutdown 2023-10-11T06:13:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e464ef20 msgr2=0x5576e467a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e464ef20 0x5576e467a1f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009970 tx=0x7fbf90004730 comp rx=0 tx=0).stop 2023-10-11T06:13:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 shutdown_connections 2023-10-11T06:13:53.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 0x5576e4662480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e4664d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e4673780 0x5576e4678f40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 --2- 172.21.15.84:0/1661059908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e464ef20 0x5576e467a1f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbf7dffb640 0 client.6165 ms_handle_reset on v2:172.21.15.84:6835/2048038445 2023-10-11T06:13:53.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 >> 172.21.15.84:0/1661059908 conn(0x5576e4650cf0 msgr2=0x5576e467b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 shutdown_connections 2023-10-11T06:13:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 1 -- 172.21.15.84:0/1661059908 wait complete. 2023-10-11T06:13:53.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:52.991+0000 7fbfa807c740 20 client.6165 trim_cache size 0 max 16384 2023-10-11T06:13:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.007+0000 7fbfa807c740 1 Processor -- start 2023-10-11T06:13:53.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.007+0000 7fbfa807c740 1 -- start start 2023-10-11T06:13:53.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 0x5576e46b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e4664c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e46547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbfa807c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e431ff20 con 0x5576e46776d0 2023-10-11T06:13:53.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e467b010 con 0x5576e464ef20 2023-10-11T06:13:53.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576e469a120 con 0x5576e4677300 2023-10-11T06:13:53.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e46547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e46547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55070/0 (socket says 172.21.15.84:55070) 2023-10-11T06:13:53.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/2941544510 learned_addr learned my addr 172.21.15.84:0/2941544510 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:53.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/2941544510 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 msgr2=0x5576e4664c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 --2- 172.21.15.84:0/2941544510 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e4664c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/2941544510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 msgr2=0x5576e46b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 --2- 172.21.15.84:0/2941544510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 0x5576e46b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.008+0000 7fbf9f7fe640 1 -- 172.21.15.84:0/2941544510 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e45589e0 con 0x5576e46776d0 2023-10-11T06:13:53.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbf9f7fe640 1 --2- 172.21.15.84:0/2941544510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e46547e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbf9805f980 tx=0x7fbf980578f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dea9256aee854f server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:53.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbf957fa640 1 -- 172.21.15.84:0/2941544510 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9805bd00 con 0x5576e46776d0 2023-10-11T06:13:53.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbf957fa640 1 -- 172.21.15.84:0/2941544510 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9805feb0 con 0x5576e46776d0 2023-10-11T06:13:53.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbf957fa640 1 -- 172.21.15.84:0/2941544510 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980545e0 con 0x5576e46776d0 2023-10-11T06:13:53.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 -- 172.21.15.84:0/2941544510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 msgr2=0x5576e46547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2941544510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e46547e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbf9805f980 tx=0x7fbf980578f0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 -- 172.21.15.84:0/2941544510 shutdown_connections 2023-10-11T06:13:53.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2941544510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e46776d0 0x5576e46547e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2941544510 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e4677300 0x5576e4664c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2941544510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e464ef20 0x5576e46b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 -- 172.21.15.84:0/2941544510 >> 172.21.15.84:0/2941544510 conn(0x5576e4650cf0 msgr2=0x5576e464a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:53.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 -- 172.21.15.84:0/2941544510 shutdown_connections 2023-10-11T06:13:53.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.009+0000 7fbfa807c740 1 -- 172.21.15.84:0/2941544510 wait complete. 2023-10-11T06:13:53.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.010+0000 7fbfa807c740 1 -- start start 2023-10-11T06:13:53.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.010+0000 7fbfa807c740 1 Processor -- start 2023-10-11T06:13:53.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e464ef20 0x5576e4681790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e4677300 0x5576e4677bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbfa807c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 0x5576e4678100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbfa807c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e46b6710 con 0x5576e4677300 2023-10-11T06:13:53.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e4663950 con 0x5576e46776d0 2023-10-11T06:13:53.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbfa807c740 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576e4681cd0 con 0x5576e464ef20 2023-10-11T06:13:53.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbfa807c740 10 client.0 _ll_register_callbacks cb 0x7fbf88009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-11T06:13:53.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbf9e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e4677300 0x5576e4677bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbf9e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e4677300 0x5576e4677bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55078/0 (socket says 172.21.15.84:55078) 2023-10-11T06:13:53.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.011+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/2067522477 learned_addr learned my addr 172.21.15.84:0/2067522477 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:53.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/2067522477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e464ef20 msgr2=0x5576e4681790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e464ef20 0x5576e4681790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/2067522477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 msgr2=0x5576e4678100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 0x5576e4678100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf9e7fc640 1 -- 172.21.15.84:0/2067522477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa015eec0 con 0x5576e4677300 2023-10-11T06:13:53.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf9e7fc640 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e4677300 0x5576e4677bc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009780 tx=0x7fbf8800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2258d0bd607db429 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:53.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-10-11T06:13:53.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf7dffb640 1 -- 172.21.15.84:0/2067522477 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88023700 con 0x5576e4677300 2023-10-11T06:13:53.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf7dffb640 1 -- 172.21.15.84:0/2067522477 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf88020070 con 0x5576e4677300 2023-10-11T06:13:53.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf8800bc30 con 0x5576e4677300 2023-10-11T06:13:53.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.012+0000 7fbf7dffb640 1 -- 172.21.15.84:0/2067522477 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf88023860 con 0x5576e4677300 2023-10-11T06:13:53.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 10 client.6171 fetch_fsmap learned FSMap version 24 2023-10-11T06:13:53.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5576e457def0 con 0x5576e4677300 2023-10-11T06:13:53.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbf7dffb640 1 -- 172.21.15.84:0/2067522477 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8800bc30 con 0x5576e4677300 2023-10-11T06:13:53.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbf7dffb640 1 -- 172.21.15.84:0/2067522477 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbf88022070 con 0x5576e4677300 2023-10-11T06:13:53.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 10 client.6171 fetch_fsmap finished waiting for FSMap version 24 2023-10-11T06:13:53.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 10 client.6171 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-11T06:13:53.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 20 client.6171 populate_metadata read hostname 'smithi084' 2023-10-11T06:13:53.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e470bb60 0x5576e470bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 4 client.6171 mds_command: new command op to 6027 tid=0[{"prefix": "status"}] 2023-10-11T06:13:53.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 --> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5576e45b1080 con 0x5576e470bb60 2023-10-11T06:13:53.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.013+0000 7fbf9effd640 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e470bb60 0x5576e470bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbf9effd640 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e470bb60 0x5576e470bf30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71293f06d16a497 server_cookie=3ebb9c3da32fa792 in_seq=0 out_seq=0 2023-10-11T06:13:53.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbf7dffb640 10 client.6171 ms_handle_connect on v2:172.21.15.84:6835/2048038445 2023-10-11T06:13:53.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbf7dffb640 1 -- 172.21.15.84:0/2067522477 <== mds.0 v2:172.21.15.84:6835/2048038445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5576e45b1080 con 0x5576e470bb60 2023-10-11T06:13:53.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbf7dffb640 10 client.6171 handle_command_reply: tid=0 2023-10-11T06:13:53.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 client.6171 shutdown 2023-10-11T06:13:53.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e4677300 msgr2=0x5576e4677bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e4677300 0x5576e4677bc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009780 tx=0x7fbf8800e000 comp rx=0 tx=0).stop 2023-10-11T06:13:53.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 shutdown_connections 2023-10-11T06:13:53.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576e46776d0 0x5576e4678100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:6835/2048038445,v1:172.21.15.84:6837/2048038445] conn(0x5576e470bb60 0x5576e470bf30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5576e4677300 0x5576e4677bc0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 --2- 172.21.15.84:0/2067522477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5576e464ef20 0x5576e4681790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbf7dffb640 0 client.6171 ms_handle_reset on v2:172.21.15.84:6835/2048038445 2023-10-11T06:13:53.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 >> 172.21.15.84:0/2067522477 conn(0x5576e4650cf0 msgr2=0x5576e469cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:53.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 shutdown_connections 2023-10-11T06:13:53.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.014+0000 7fbfa807c740 1 -- 172.21.15.84:0/2067522477 wait complete. 2023-10-11T06:13:53.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbfa807c740 20 client.6171 trim_cache size 0 max 16384 2023-10-11T06:13:53.029 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:13:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c84dc637-9184-44da-bf67-6cb4224adc5a", 2023-10-11T06:13:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-10-11T06:13:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6027, 2023-10-11T06:13:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-10-11T06:13:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-10-11T06:13:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-10-11T06:13:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.6226131129999999, 2023-10-11T06:13:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 24, 2023-10-11T06:13:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 54, 2023-10-11T06:13:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 54, 2023-10-11T06:13:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.003379408000001 2023-10-11T06:13:53.032 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:13:53.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbf8006a830 msgr2=0x7fbf8006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbf8006a830 0x7fbf8006cce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbf90035000 tx=0x7fbf90030000 comp rx=0 tx=0).stop 2023-10-11T06:13:53.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00d2630 msgr2=0x7fbfa015e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00d2630 0x7fbfa015e980 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b990 tx=0x7fbf9800be50 comp rx=0 tx=0).stop 2023-10-11T06:13:53.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 shutdown_connections 2023-10-11T06:13:53.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbf8006a830 0x7fbf8006cce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa00d2630 0x7fbfa015e980 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfa00d0300 0x7fbfa0150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 --2- 172.21.15.84:0/3618385903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbfa00cf750 0x7fbfa015d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 >> 172.21.15.84:0/3618385903 conn(0x7fbfa0071cc0 msgr2=0x7fbfa00b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 shutdown_connections 2023-10-11T06:13:53.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.015+0000 7fbf9ffff640 1 -- 172.21.15.84:0/3618385903 wait complete. 2023-10-11T06:13:53.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:13:53.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.214+0000 7f62e6029640 1 Processor -- start 2023-10-11T06:13:53.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.214+0000 7f62e6029640 1 -- start start 2023-10-11T06:13:53.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.216+0000 7f62e6029640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 0x7f62e00cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.216+0000 7f62e6029640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e015fd70 con 0x7f62e015c240 2023-10-11T06:13:53.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.216+0000 7f62e6029640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e006d8a0 con 0x7f62e006d420 2023-10-11T06:13:53.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.216+0000 7f62e6029640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006e440 con 0x7f62e015be70 2023-10-11T06:13:53.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.216+0000 7f62e4826640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f62e015c240 0x7f62e0160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54536/0 (socket says 172.21.15.84:54536) 2023-10-11T06:13:53.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.216+0000 7f62e4826640 1 -- 172.21.15.84:0/310796170 learned_addr learned my addr 172.21.15.84:0/310796170 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:13:53.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62e5027640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 0x7f62e00cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356272308 0 0) 0x7f62e015fd70 con 0x7f62e015c240 2023-10-11T06:13:53.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62cc003520 con 0x7f62e015c240 2023-10-11T06:13:53.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1505812423 0 0) 0x7f62cc003520 con 0x7f62e015c240 2023-10-11T06:13:53.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 msgr2=0x7f62e00cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 0x7f62e00cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 >> v1:172.21.15.99:6789/0 conn(0x7f62e006d420 legacy=0x7f62e0070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e006dba0 con 0x7f62e015c240 2023-10-11T06:13:53.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f62d0001670 con 0x7f62e015c240 2023-10-11T06:13:53.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 >> v1:172.21.15.84:6789/0 conn(0x7f62e015c240 legacy=0x7f62e0160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62cc0039a0 0x7f62cc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 0x7f62cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62cc004d00 0x7f62cc005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006e440 con 0x7f62cc0039a0 2023-10-11T06:13:53.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc00bb10 con 0x7f62cc004d00 2023-10-11T06:13:53.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.217+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc00bca0 con 0x7f62e015be70 2023-10-11T06:13:53.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e5027640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62cc0039a0 0x7f62cc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e4826640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 0x7f62cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e5027640 1 -- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 msgr2=0x7f62cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e5027640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 0x7f62cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e5027640 1 -- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62cc004d00 msgr2=0x7f62cc005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:53.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e5027640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62cc004d00 0x7f62cc005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e5027640 1 -- 172.21.15.84:0/310796170 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc003520 con 0x7f62cc0039a0 2023-10-11T06:13:53.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.218+0000 7f62e4826640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62e015be70 0x7f62cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:53.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.219+0000 7f62e5027640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62cc0039a0 0x7f62cc003db0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009ac0 tx=0x7f62cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7a415fcacfc0b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:53.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.219+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc02c6b0 con 0x7f62cc0039a0 2023-10-11T06:13:53.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.219+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc02c810 con 0x7f62cc0039a0 2023-10-11T06:13:53.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.219+0000 7f62d7fff640 1 -- 172.21.15.84:0/310796170 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc02cba0 con 0x7f62cc0039a0 2023-10-11T06:13:53.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.220+0000 7f62e6029640 1 -- 172.21.15.84:0/310796170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62cc0039a0 msgr2=0x7f62cc003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.220+0000 7f62e6029640 1 --2- 172.21.15.84:0/310796170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62cc0039a0 0x7f62cc003db0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009ac0 tx=0x7f62cc0057f0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.220+0000 7f62e5828640 1 -- 172.21.15.84:0/310796170 reap_dead start 2023-10-11T06:13:53.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.220+0000 7f62e6029640 1 -- 172.21.15.84:0/310796170 shutdown_connections 2023-10-11T06:13:53.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.220+0000 7f62e6029640 1 -- 172.21.15.84:0/310796170 >> 172.21.15.84:0/310796170 conn(0x7f62e00cec30 msgr2=0x7f62e00cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:53.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.220+0000 7f62e6029640 1 -- 172.21.15.84:0/310796170 shutdown_connections 2023-10-11T06:13:53.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.220+0000 7f62e6029640 1 -- 172.21.15.84:0/310796170 wait complete. 2023-10-11T06:13:53.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.222+0000 7f62e6029640 1 Processor -- start 2023-10-11T06:13:53.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.222+0000 7f62e6029640 1 -- start start 2023-10-11T06:13:53.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e6029640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62cc0039a0 0x7f62e00c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e6029640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62e00b26e0 0x7f62e015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e6029640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e01f10a0 0x7f62e00c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e6029640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e015d040 con 0x7f62e01f10a0 2023-10-11T06:13:53.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e6029640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e015f550 con 0x7f62e00b26e0 2023-10-11T06:13:53.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e6029640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e015c710 con 0x7f62cc0039a0 2023-10-11T06:13:53.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e01f10a0 0x7f62e00c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e01f10a0 0x7f62e00c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55096/0 (socket says 172.21.15.84:55096) 2023-10-11T06:13:53.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 -- 172.21.15.84:0/247348547 learned_addr learned my addr 172.21.15.84:0/247348547 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:53.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e4826640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62e00b26e0 0x7f62e015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5027640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62cc0039a0 0x7f62e00c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 -- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62cc0039a0 msgr2=0x7f62e00c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62cc0039a0 0x7f62e00c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 -- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62e00b26e0 msgr2=0x7f62e015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62e00b26e0 0x7f62e015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.223+0000 7f62e5828640 1 -- 172.21.15.84:0/247348547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e015c9f0 con 0x7f62e01f10a0 2023-10-11T06:13:53.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.224+0000 7f62e5828640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e01f10a0 0x7f62e00c5d00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cf830 tx=0x7f62d800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:53.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.224+0000 7f62d5ffb640 1 -- 172.21.15.84:0/247348547 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d800cad0 con 0x7f62e01f10a0 2023-10-11T06:13:53.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.224+0000 7f62d5ffb640 1 -- 172.21.15.84:0/247348547 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d800cc50 con 0x7f62e01f10a0 2023-10-11T06:13:53.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.224+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e015ccd0 con 0x7f62e01f10a0 2023-10-11T06:13:53.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.224+0000 7f62d5ffb640 1 -- 172.21.15.84:0/247348547 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8003970 con 0x7f62e01f10a0 2023-10-11T06:13:53.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.224+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e015f220 con 0x7f62e01f10a0 2023-10-11T06:13:53.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.225+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e015d1b0 con 0x7f62e01f10a0 2023-10-11T06:13:53.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.226+0000 7f62d5ffb640 1 -- 172.21.15.84:0/247348547 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f62d80070d0 con 0x7f62e01f10a0 2023-10-11T06:13:53.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.228+0000 7f62d5ffb640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62c406a860 0x7f62c406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:53.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.228+0000 7f62d5ffb640 1 -- 172.21.15.84:0/247348547 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62d8087b90 con 0x7f62e01f10a0 2023-10-11T06:13:53.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.228+0000 7f62e5027640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62c406a860 0x7f62c406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:53.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.229+0000 7f62e5027640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62c406a860 0x7f62c406cd10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f62dc036000 tx=0x7f62dc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:53.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.232+0000 7f62d5ffb640 1 -- 172.21.15.84:0/247348547 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f62d8052f60 con 0x7f62e01f10a0 2023-10-11T06:13:53.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.454+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62e004e890 con 0x7f62e01f10a0 2023-10-11T06:13:53.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.455+0000 7f62d5ffb640 1 -- 172.21.15.84:0/247348547 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62d8056580 con 0x7f62e01f10a0 2023-10-11T06:13:53.458 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:13:53.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.457+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62c406a860 msgr2=0x7f62c406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.457+0000 7f62e6029640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62c406a860 0x7f62c406cd10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f62dc036000 tx=0x7f62dc030000 comp rx=0 tx=0).stop 2023-10-11T06:13:53.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.457+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e01f10a0 msgr2=0x7f62e00c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:53.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.457+0000 7f62e6029640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e01f10a0 0x7f62e00c5d00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cf830 tx=0x7f62d800c7c0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 shutdown_connections 2023-10-11T06:13:53.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f62c406a860 0x7f62c406cd10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e01f10a0 0x7f62e00c5d00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62e00b26e0 0x7f62e015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 --2- 172.21.15.84:0/247348547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62cc0039a0 0x7f62e00c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:53.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 >> 172.21.15.84:0/247348547 conn(0x7f62e00cec30 msgr2=0x7f62e00cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:53.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 shutdown_connections 2023-10-11T06:13:53.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:53.458+0000 7f62e6029640 1 -- 172.21.15.84:0/247348547 wait complete. 2023-10-11T06:13:53.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:13:53.477 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:13:53.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:13:53.478 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T06:13:53.478 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:53.478 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:13:53.479 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:13:53.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:13:53.479 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:53.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:53.480 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:13:53.533 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:13:53.533 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:13:53.533 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:13:53.533 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:53.534 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:13:53.534 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:53.534 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:53.534 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:53.535 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:13:53.535 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:53.535 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:53.535 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:13:53.535 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:13:53.536 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:53.536 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:13:53.536 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:53.536 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:13:53.536 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42243sec preferred_lft 42243sec 2023-10-11T06:13:53.537 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:13:53.537 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:53.537 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:13:53.537 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff 2023-10-11T06:13:53.537 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:13:53.538 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:53.538 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1c33:d5ff:fea1:21f9/64 scope link 2023-10-11T06:13:53.538 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:53.538 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:13:53.538 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:13:53.539 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::d4ff:bcff:feac:c798/64 scope link 2023-10-11T06:13:53.539 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:53.539 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:13:53.539 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:13:53.539 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4088:59ff:feb7:538e/64 scope link 2023-10-11T06:13:53.540 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:13:53.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:53.541 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:13:53.630 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:13:53.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:53.635 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:13:53.689 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:13:53.690 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:13:53.690 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:13:53.690 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:13:53.690 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:13:53.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:53.692 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:13:53.692 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:13:53.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:13:53.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:13:53.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:13:53.694 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:13:53.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:53.875 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:13:53.927 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T06:13:53.927 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:13:53.928 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:53.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:53.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-11T06:13:54.021 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T06:13:54.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:13:54.021 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:54.076 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:13:54.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:13:54.077 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:54.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:54.222 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:54.223 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:54.317 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:54.321 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:54.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:54.322 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:54.376 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:13:54.376 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T06:13:54.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:54.468 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:54.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:54.538 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:54.541 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:54.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:54.542 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:54.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-10-11T06:13:54.575+0000 7f71e8462480 -1 init, newargv = 0x55fc638e7ef0 newargc=15 2023-10-11T06:13:54.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[107719]: starting ceph client 2023-10-11T06:13:54.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[107719]: starting fuse 2023-10-11T06:13:55.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:13:55.792 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:13:55.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:13:55.884 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:13:55.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:13:55.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:55.889 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:13:55.945 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:13:55.945 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-10-11T06:13:55.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:13:55.947 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:13:55.947 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:13:55.947 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:13:55.948 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:13:55.948 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:55.949 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:13:55.949 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T06:13:55.950 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:13:55.950 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T06:13:55.950 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:55.951 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:13:55.951 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:13:55.952 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:13:55.953 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:13:55.953 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:55.954 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:13:55.954 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:13:55.954 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:13:55.955 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:13:55.955 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:13:55.956 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:13:55.956 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:13:55.956 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:13:55.957 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:13:55.957 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-10-11T06:13:55.957 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:13:56.137 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.107719.asok 2023-10-11T06:13:56.320 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107719.asok 2023-10-11T06:13:56.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:56.321 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107719.asok status 2023-10-11T06:13:56.564 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:13:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:13:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:13:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:13:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-10-11T06:13:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:13:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T06:13:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "107719", 2023-10-11T06:13:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:13:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:13:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:13:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6183, 2023-10-11T06:13:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:13:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:13:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:13:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 6183 2023-10-11T06:13:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:13:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:13:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:13:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1944664161 2023-10-11T06:13:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:13:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:13:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:13:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:13:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1944664161 2023-10-11T06:13:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:13:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.6183 192.168.0.1:0/1944664161", 2023-10-11T06:13:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/1944664161", 2023-10-11T06:13:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:13:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 24, 2023-10-11T06:13:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 54, 2023-10-11T06:13:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:13:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:13:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:13:56.572 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:13:56.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:56.578 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:13:56.579 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:56.634 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:13:56.634 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:56.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:56.635 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:13:56.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:13:56.737 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2023-10-11T06:13:56.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2023-10-11T06:13:57.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:13:57.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.011+0000 7fe48a685640 1 Processor -- start 2023-10-11T06:13:57.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.011+0000 7fe48a685640 1 -- start start 2023-10-11T06:13:57.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:13:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.012+0000 7fe48a685640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe484159e30 0x7fe4840afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.012+0000 7fe48a685640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840b1ed0 con 0x7fe484071440 2023-10-11T06:13:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe48a685640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48415e6c0 con 0x7fe4840b3370 2023-10-11T06:13:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe48a685640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48415a200 con 0x7fe484159e30 2023-10-11T06:13:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe489683640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe484159e30 0x7fe4840afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe489683640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe484159e30 0x7fe4840afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55102/0 (socket says 172.21.15.84:55102) 2023-10-11T06:13:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe489683640 1 -- 172.21.15.84:0/629020063 learned_addr learned my addr 172.21.15.84:0/629020063 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe488e82640 1 --1- 172.21.15.84:0/629020063 >> v1:172.21.15.84:6789/0 conn(0x7fe484071440 0x7fe48415d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54552/0 (socket says 172.21.15.84:54552) 2023-10-11T06:13:57.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe489683640 1 -- 172.21.15.84:0/629020063 >> v1:172.21.15.99:6789/0 conn(0x7fe4840b3370 legacy=0x7fe48415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe489683640 1 -- 172.21.15.84:0/629020063 >> v1:172.21.15.84:6789/0 conn(0x7fe484071440 legacy=0x7fe48415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.013+0000 7fe489683640 1 -- 172.21.15.84:0/629020063 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840718c0 con 0x7fe484159e30 2023-10-11T06:13:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.015+0000 7fe489683640 1 --2- 172.21.15.84:0/629020063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe484159e30 0x7fe4840afd80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe480009870 tx=0x7fe48002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75d6d3dcfeb1733 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.015+0000 7fe47bfff640 1 -- 172.21.15.84:0/629020063 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480030860 con 0x7fe484159e30 2023-10-11T06:13:57.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.015+0000 7fe47bfff640 1 -- 172.21.15.84:0/629020063 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4800309c0 con 0x7fe484159e30 2023-10-11T06:13:57.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.015+0000 7fe47bfff640 1 -- 172.21.15.84:0/629020063 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480030d50 con 0x7fe484159e30 2023-10-11T06:13:57.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.016+0000 7fe48a685640 1 -- 172.21.15.84:0/629020063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe484159e30 msgr2=0x7fe4840afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.016+0000 7fe48a685640 1 --2- 172.21.15.84:0/629020063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe484159e30 0x7fe4840afd80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe480009870 tx=0x7fe48002b9a0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.016+0000 7fe48a685640 1 -- 172.21.15.84:0/629020063 shutdown_connections 2023-10-11T06:13:57.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.016+0000 7fe48a685640 1 --2- 172.21.15.84:0/629020063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe484159e30 0x7fe4840afd80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.016+0000 7fe48a685640 1 -- 172.21.15.84:0/629020063 >> 172.21.15.84:0/629020063 conn(0x7fe484069c70 msgr2=0x7fe48406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:57.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.016+0000 7fe48a685640 1 -- 172.21.15.84:0/629020063 shutdown_connections 2023-10-11T06:13:57.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.016+0000 7fe48a685640 1 -- 172.21.15.84:0/629020063 wait complete. 2023-10-11T06:13:57.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.018+0000 7fe48a685640 1 Processor -- start 2023-10-11T06:13:57.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.018+0000 7fe48a685640 1 -- start start 2023-10-11T06:13:57.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe48a685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4840b3370 0x7fe4841533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe48a685640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe484159e30 0x7fe4840ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe48a685640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4840ac740 0x7fe4840b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe48a685640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840aec30 con 0x7fe4840b3370 2023-10-11T06:13:57.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe48a685640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840acbb0 con 0x7fe484159e30 2023-10-11T06:13:57.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe48a685640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841f19f0 con 0x7fe4840ac740 2023-10-11T06:13:57.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe489683640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4840b3370 0x7fe4841533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe489683640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4840b3370 0x7fe4841533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55106/0 (socket says 172.21.15.84:55106) 2023-10-11T06:13:57.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.019+0000 7fe489683640 1 -- 172.21.15.84:0/3944118225 learned_addr learned my addr 172.21.15.84:0/3944118225 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:57.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.020+0000 7fe488e82640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe484159e30 0x7fe4840ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.020+0000 7fe489683640 1 -- 172.21.15.84:0/3944118225 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4840ac740 msgr2=0x7fe4840b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.020+0000 7fe489683640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4840ac740 0x7fe4840b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.020+0000 7fe489683640 1 -- 172.21.15.84:0/3944118225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe484159e30 msgr2=0x7fe4840ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.020+0000 7fe489683640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe484159e30 0x7fe4840ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.020+0000 7fe489683640 1 -- 172.21.15.84:0/3944118225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4800094d0 con 0x7fe4840b3370 2023-10-11T06:13:57.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.020+0000 7fe489683640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4840b3370 0x7fe4841533a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe480000c00 tx=0x7fe480004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:57.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.021+0000 7fe479ffb640 1 -- 172.21.15.84:0/3944118225 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480030810 con 0x7fe4840b3370 2023-10-11T06:13:57.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.021+0000 7fe479ffb640 1 -- 172.21.15.84:0/3944118225 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480039d90 con 0x7fe4840b3370 2023-10-11T06:13:57.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.021+0000 7fe479ffb640 1 -- 172.21.15.84:0/3944118225 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480043650 con 0x7fe4840b3370 2023-10-11T06:13:57.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.021+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4840add60 con 0x7fe4840b3370 2023-10-11T06:13:57.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.021+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4840ae290 con 0x7fe4840b3370 2023-10-11T06:13:57.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.022+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4840b1ae0 con 0x7fe4840b3370 2023-10-11T06:13:57.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.023+0000 7fe479ffb640 1 -- 172.21.15.84:0/3944118225 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe48002f3b0 con 0x7fe4840b3370 2023-10-11T06:13:57.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.025+0000 7fe479ffb640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe46806a830 0x7fe46806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.025+0000 7fe479ffb640 1 -- 172.21.15.84:0/3944118225 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4800ab3f0 con 0x7fe4840b3370 2023-10-11T06:13:57.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.025+0000 7fe488e82640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe46806a830 0x7fe46806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.026+0000 7fe488e82640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe46806a830 0x7fe46806cce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe474007fd0 tx=0x7fe474009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:57.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.030+0000 7fe479ffb640 1 -- 172.21.15.84:0/3944118225 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe480076700 con 0x7fe4840b3370 2023-10-11T06:13:57.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.295+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7fe48404e890 con 0x7fe4840b3370 2023-10-11T06:13:57.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.302+0000 7fe479ffb640 1 -- 172.21.15.84:0/3944118225 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v14) v1 ==== 183+0+0 (secure 0 0 0) 0x7fe480079d20 con 0x7fe4840b3370 2023-10-11T06:13:57.305 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:13:57.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe46806a830 msgr2=0x7fe46806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe46806a830 0x7fe46806cce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe474007fd0 tx=0x7fe474009040 comp rx=0 tx=0).stop 2023-10-11T06:13:57.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4840b3370 msgr2=0x7fe4841533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4840b3370 0x7fe4841533a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe480000c00 tx=0x7fe480004730 comp rx=0 tx=0).stop 2023-10-11T06:13:57.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 shutdown_connections 2023-10-11T06:13:57.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe46806a830 0x7fe46806cce0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4840ac740 0x7fe4840b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe484159e30 0x7fe4840ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 --2- 172.21.15.84:0/3944118225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4840b3370 0x7fe4841533a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 >> 172.21.15.84:0/3944118225 conn(0x7fe484069c70 msgr2=0x7fe48406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:57.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 shutdown_connections 2023-10-11T06:13:57.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.305+0000 7fe48a685640 1 -- 172.21.15.84:0/3944118225 wait complete. 2023-10-11T06:13:57.327 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:13:57.327 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:13:57.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:13:57.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.633+0000 7f4c2278b640 1 Processor -- start 2023-10-11T06:13:57.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.633+0000 7f4c2278b640 1 -- start start 2023-10-11T06:13:57.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c2278b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 0x7f4c1c159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c2278b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4c1c0c8c50 0x7f4c1c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c2278b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4c1c0b05c0 0x7f4c1c15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c2278b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c070070 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c2278b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b0a20 con 0x7f4c1c0b05c0 2023-10-11T06:13:57.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c2278b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c9670 con 0x7f4c1c159250 2023-10-11T06:13:57.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c13fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4c1c0c8c50 0x7f4c1c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c13fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4c1c0c8c50 0x7f4c1c0b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55108/0 (socket says 172.21.15.84:55108) 2023-10-11T06:13:57.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.635+0000 7f4c13fff640 1 -- 172.21.15.84:0/349149936 learned_addr learned my addr 172.21.15.84:0/349149936 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:57.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c1bfff640 1 --2- 172.21.15.84:0/349149936 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 0x7f4c1c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c20d28640 1 --2- 172.21.15.84:0/349149936 >> v2:172.21.15.99:3300/0 conn(0x7f4c1c0b05c0 0x7f4c1c15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c13fff640 1 -- 172.21.15.84:0/349149936 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 msgr2=0x7f4c1c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:57.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c13fff640 1 --2- 172.21.15.84:0/349149936 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 0x7f4c1c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c13fff640 1 -- 172.21.15.84:0/349149936 >> v2:172.21.15.99:3300/0 conn(0x7f4c1c0b05c0 msgr2=0x7f4c1c15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:57.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c13fff640 1 --2- 172.21.15.84:0/349149936 >> v2:172.21.15.99:3300/0 conn(0x7f4c1c0b05c0 0x7f4c1c15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c13fff640 1 -- 172.21.15.84:0/349149936 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0c9950 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.636+0000 7f4c20d28640 1 --2- 172.21.15.84:0/349149936 >> v2:172.21.15.99:3300/0 conn(0x7f4c1c0b05c0 0x7f4c1c15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:57.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.637+0000 7f4c13fff640 1 --2- 172.21.15.84:0/349149936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0b0080 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0ae010 tx=0x7f4c0402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67682cc23305e864 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:57.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.637+0000 7f4c1b7fe640 1 -- 172.21.15.84:0/349149936 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0402ee90 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c1b7fe640 1 -- 172.21.15.84:0/349149936 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c04036440 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c1b7fe640 1 -- 172.21.15.84:0/349149936 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c040367d0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 -- 172.21.15.84:0/349149936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 msgr2=0x7f4c1c0b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 --2- 172.21.15.84:0/349149936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0b0080 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0ae010 tx=0x7f4c0402e0e0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 -- 172.21.15.84:0/349149936 shutdown_connections 2023-10-11T06:13:57.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 --2- 172.21.15.84:0/349149936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0b0080 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 --2- 172.21.15.84:0/349149936 >> v2:172.21.15.99:3300/0 conn(0x7f4c1c0b05c0 0x7f4c1c15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 --2- 172.21.15.84:0/349149936 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 0x7f4c1c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 -- 172.21.15.84:0/349149936 >> 172.21.15.84:0/349149936 conn(0x7f4c1c069c70 msgr2=0x7f4c1c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:57.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 -- 172.21.15.84:0/349149936 shutdown_connections 2023-10-11T06:13:57.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.638+0000 7f4c2278b640 1 -- 172.21.15.84:0/349149936 wait complete. 2023-10-11T06:13:57.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.640+0000 7f4c2278b640 1 Processor -- start 2023-10-11T06:13:57.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.640+0000 7f4c2278b640 1 -- start start 2023-10-11T06:13:57.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.641+0000 7f4c2278b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1c0b05c0 0x7f4c1c159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.641+0000 7f4c2278b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.641+0000 7f4c2278b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 0x7f4c1c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.641+0000 7f4c2278b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b35b0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.641+0000 7f4c2278b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c6400 con 0x7f4c1c0b05c0 2023-10-11T06:13:57.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.641+0000 7f4c2278b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b2f60 con 0x7f4c1c159250 2023-10-11T06:13:57.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.641+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55118/0 (socket says 172.21.15.84:55118) 2023-10-11T06:13:57.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 -- 172.21.15.84:0/108341749 learned_addr learned my addr 172.21.15.84:0/108341749 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:57.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 -- 172.21.15.84:0/108341749 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 msgr2=0x7f4c1c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 0x7f4c1c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 -- 172.21.15.84:0/108341749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1c0b05c0 msgr2=0x7f4c1c159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1c0b05c0 0x7f4c1c159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 -- 172.21.15.84:0/108341749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0400b770 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c13fff640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0cd5f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4c0402e120 tx=0x7f4c040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:57.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.642+0000 7f4c197fa640 1 -- 172.21.15.84:0/108341749 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04034850 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.643+0000 7f4c197fa640 1 -- 172.21.15.84:0/108341749 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c040349b0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.643+0000 7f4c197fa640 1 -- 172.21.15.84:0/108341749 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04034d60 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.643+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c0b3240 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.643+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c15a490 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.644+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c0b2dc0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.645+0000 7f4c197fa640 1 -- 172.21.15.84:0/108341749 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4c0403d040 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.646+0000 7f4c197fa640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4bfc06a830 0x7f4bfc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:57.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.646+0000 7f4c197fa640 1 -- 172.21.15.84:0/108341749 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c040adee0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.646+0000 7f4c1bfff640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4bfc06a830 0x7f4bfc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:57.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.647+0000 7f4c1bfff640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4bfc06a830 0x7f4bfc06cce0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006ea0 tx=0x7f4c0c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:57.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.650+0000 7f4c197fa640 1 -- 172.21.15.84:0/108341749 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4c1c0b2dc0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.840+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c1c0b30a0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.842+0000 7f4c197fa640 1 -- 172.21.15.84:0/108341749 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7f4c1c0b30a0 con 0x7f4c1c0c8c50 2023-10-11T06:13:57.845 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:13:57.845 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:13:57.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4bfc06a830 msgr2=0x7f4bfc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4bfc06a830 0x7f4bfc06cce0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c006ea0 tx=0x7f4c0c008040 comp rx=0 tx=0).stop 2023-10-11T06:13:57.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 msgr2=0x7f4c1c0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:57.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0cd5f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4c0402e120 tx=0x7f4c040046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 shutdown_connections 2023-10-11T06:13:57.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f4bfc06a830 0x7f4bfc06cce0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c1c159250 0x7f4c1c0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c1c0c8c50 0x7f4c1c0cd5f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 --2- 172.21.15.84:0/108341749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c1c0b05c0 0x7f4c1c159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:57.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 >> 172.21.15.84:0/108341749 conn(0x7f4c1c069c70 msgr2=0x7f4c1c1550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:57.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 shutdown_connections 2023-10-11T06:13:57.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:57.845+0000 7f4c2278b640 1 -- 172.21.15.84:0/108341749 wait complete. 2023-10-11T06:13:57.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:13:44.841435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:13:45.401366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:13:57.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:13:58.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.079+0000 7f2fef3a9640 1 Processor -- start 2023-10-11T06:13:58.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.079+0000 7f2fef3a9640 1 -- start start 2023-10-11T06:13:58.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.081+0000 7f2fef3a9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2fe815a0b0 0x7f2fe80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.081+0000 7f2fef3a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fe80c8d00 0x7f2fe80ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.081+0000 7f2fef3a9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe80ceb60 0x7f2fe806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.081+0000 7f2fef3a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe806e0f0 con 0x7f2fe80c8d00 2023-10-11T06:13:58.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.081+0000 7f2fef3a9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe80cbae0 con 0x7f2fe815a0b0 2023-10-11T06:13:58.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.081+0000 7f2fef3a9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe80cefc0 con 0x7f2fe80ceb60 2023-10-11T06:13:58.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.081+0000 7f2fec944640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fe80c8d00 0x7f2fe80ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fec944640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fe80c8d00 0x7f2fe80ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55126/0 (socket says 172.21.15.84:55126) 2023-10-11T06:13:58.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fec944640 1 -- 172.21.15.84:0/2265095768 learned_addr learned my addr 172.21.15.84:0/2265095768 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:58.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fed946640 1 --2- 172.21.15.84:0/2265095768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe80ceb60 0x7f2fe806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fed145640 1 --2- 172.21.15.84:0/2265095768 >> v2:172.21.15.99:3300/0 conn(0x7f2fe815a0b0 0x7f2fe80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fec944640 1 -- 172.21.15.84:0/2265095768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe80ceb60 msgr2=0x7f2fe806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:58.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fec944640 1 --2- 172.21.15.84:0/2265095768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe80ceb60 0x7f2fe806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fec944640 1 -- 172.21.15.84:0/2265095768 >> v2:172.21.15.99:3300/0 conn(0x7f2fe815a0b0 msgr2=0x7f2fe80cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:58.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fec944640 1 --2- 172.21.15.84:0/2265095768 >> v2:172.21.15.99:3300/0 conn(0x7f2fe815a0b0 0x7f2fe80cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.082+0000 7f2fec944640 1 -- 172.21.15.84:0/2265095768 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8071380 con 0x7f2fe80c8d00 2023-10-11T06:13:58.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.083+0000 7f2fec944640 1 --2- 172.21.15.84:0/2265095768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80ce620 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2fe806ad30 tx=0x7f2fd802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fc6b30c3d32593 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.083+0000 7f2fdffff640 1 -- 172.21.15.84:0/2265095768 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8031740 con 0x7f2fe80c8d00 2023-10-11T06:13:58.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fdffff640 1 -- 172.21.15.84:0/2265095768 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd80318a0 con 0x7f2fe80c8d00 2023-10-11T06:13:58.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fdffff640 1 -- 172.21.15.84:0/2265095768 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8031c30 con 0x7f2fe80c8d00 2023-10-11T06:13:58.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 -- 172.21.15.84:0/2265095768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80c8d00 msgr2=0x7f2fe80ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/2265095768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80ce620 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2fe806ad30 tx=0x7f2fd802b7e0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 -- 172.21.15.84:0/2265095768 shutdown_connections 2023-10-11T06:13:58.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/2265095768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80ce620 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/2265095768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe80ceb60 0x7f2fe806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/2265095768 >> v2:172.21.15.99:3300/0 conn(0x7f2fe815a0b0 0x7f2fe80cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 -- 172.21.15.84:0/2265095768 >> 172.21.15.84:0/2265095768 conn(0x7f2fe8069c70 msgr2=0x7f2fe806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:58.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 -- 172.21.15.84:0/2265095768 shutdown_connections 2023-10-11T06:13:58.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.084+0000 7f2fef3a9640 1 -- 172.21.15.84:0/2265095768 wait complete. 2023-10-11T06:13:58.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.086+0000 7f2fef3a9640 1 Processor -- start 2023-10-11T06:13:58.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.086+0000 7f2fef3a9640 1 -- start start 2023-10-11T06:13:58.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fef3a9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fef3a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80ceb60 0x7f2fe815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fef3a9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe815a0b0 0x7f2fe814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fef3a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe814aa20 con 0x7f2fe80ceb60 2023-10-11T06:13:58.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fef3a9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe815d290 con 0x7f2fe80c8d00 2023-10-11T06:13:58.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fef3a9640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8151510 con 0x7f2fe815a0b0 2023-10-11T06:13:58.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fec944640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80ceb60 0x7f2fe815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fec944640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80ceb60 0x7f2fe815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55132/0 (socket says 172.21.15.84:55132) 2023-10-11T06:13:58.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.087+0000 7f2fec944640 1 -- 172.21.15.84:0/3424704605 learned_addr learned my addr 172.21.15.84:0/3424704605 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fec944640 1 -- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe815a0b0 msgr2=0x7f2fe814fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fec944640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe815a0b0 0x7f2fe814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fec944640 1 -- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe80c8d00 msgr2=0x7f2fe80c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fed145640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fec944640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fec944640 1 -- 172.21.15.84:0/3424704605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd80094d0 con 0x7f2fe80ceb60 2023-10-11T06:13:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fed145640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fec944640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80ceb60 0x7f2fe815cd50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.088+0000 7f2fddffb640 1 -- 172.21.15.84:0/3424704605 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8033c70 con 0x7f2fe80ceb60 2023-10-11T06:13:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.089+0000 7f2fddffb640 1 -- 172.21.15.84:0/3424704605 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8033dd0 con 0x7f2fe80ceb60 2023-10-11T06:13:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.089+0000 7f2fddffb640 1 -- 172.21.15.84:0/3424704605 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd803c640 con 0x7f2fe80ceb60 2023-10-11T06:13:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.089+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe8150490 con 0x7f2fe80ceb60 2023-10-11T06:13:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.089+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe8150a50 con 0x7f2fe80ceb60 2023-10-11T06:13:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.090+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe8073ed0 con 0x7f2fe80ceb60 2023-10-11T06:13:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.091+0000 7f2fddffb640 1 -- 172.21.15.84:0/3424704605 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2fd803a040 con 0x7f2fe80ceb60 2023-10-11T06:13:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.092+0000 7f2fddffb640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2fcc06a830 0x7f2fcc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.092+0000 7f2fddffb640 1 -- 172.21.15.84:0/3424704605 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fd80aa9e0 con 0x7f2fe80ceb60 2023-10-11T06:13:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.092+0000 7f2fed145640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2fcc06a830 0x7f2fcc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.092+0000 7f2fed145640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2fcc06a830 0x7f2fcc06cce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4006fd0 tx=0x7f2fe4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.096+0000 7f2fddffb640 1 -- 172.21.15.84:0/3424704605 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2fd8075e10 con 0x7f2fe80ceb60 2023-10-11T06:13:58.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.283+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2fe80c94e0 con 0x7f2fe80ceb60 2023-10-11T06:13:58.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.284+0000 7f2fddffb640 1 -- 172.21.15.84:0/3424704605 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2fd8038020 con 0x7f2fe80ceb60 2023-10-11T06:13:58.287 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:13:58.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.287+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2fcc06a830 msgr2=0x7f2fcc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.287+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2fcc06a830 0x7f2fcc06cce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4006fd0 tx=0x7f2fe4008040 comp rx=0 tx=0).stop 2023-10-11T06:13:58.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.287+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80ceb60 msgr2=0x7f2fe815cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.287+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80ceb60 0x7f2fe815cd50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8000c00 tx=0x7f2fd80046c0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 shutdown_connections 2023-10-11T06:13:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2fcc06a830 0x7f2fcc06cce0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2fe815a0b0 0x7f2fe814fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fe80ceb60 0x7f2fe815cd50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 --2- 172.21.15.84:0/3424704605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe80c8d00 0x7f2fe80c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 >> 172.21.15.84:0/3424704605 conn(0x7f2fe8069c70 msgr2=0x7f2fe80c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 shutdown_connections 2023-10-11T06:13:58.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.288+0000 7f2fef3a9640 1 -- 172.21.15.84:0/3424704605 wait complete. 2023-10-11T06:13:58.309 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:13:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.524+0000 7f5d750ca640 1 Processor -- start 2023-10-11T06:13:58.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.525+0000 7f5d750ca640 1 -- start start 2023-10-11T06:13:58.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.526+0000 7f5d750ca640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d700cfb20 0x7f5d7006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.526+0000 7f5d750ca640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d700ceda0 0x7f5d700d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.526+0000 7f5d750ca640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d70160890 con 0x7f5d7015c2b0 2023-10-11T06:13:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.526+0000 7f5d750ca640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d700cd820 con 0x7f5d700ceda0 2023-10-11T06:13:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.526+0000 7f5d750ca640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d70160dc0 con 0x7f5d700cfb20 2023-10-11T06:13:58.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.526+0000 7f5d6f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d700ceda0 0x7f5d700d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d700ceda0 0x7f5d700d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55140/0 (socket says 172.21.15.84:55140) 2023-10-11T06:13:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6f59e640 1 -- 172.21.15.84:0/3366218508 learned_addr learned my addr 172.21.15.84:0/3366218508 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:58.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6ed9d640 1 --1- 172.21.15.84:0/3366218508 >> v1:172.21.15.84:6789/0 conn(0x7f5d7015c2b0 0x7f5d700cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54556/0 (socket says 172.21.15.84:54556) 2023-10-11T06:13:58.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d67fff640 1 --2- 172.21.15.84:0/3366218508 >> v2:172.21.15.99:3300/0 conn(0x7f5d700cfb20 0x7f5d7006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6f59e640 1 -- 172.21.15.84:0/3366218508 >> v2:172.21.15.99:3300/0 conn(0x7f5d700cfb20 msgr2=0x7f5d7006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6f59e640 1 --2- 172.21.15.84:0/3366218508 >> v2:172.21.15.99:3300/0 conn(0x7f5d700cfb20 0x7f5d7006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6f59e640 1 -- 172.21.15.84:0/3366218508 >> v1:172.21.15.84:6789/0 conn(0x7f5d7015c2b0 legacy=0x7f5d700cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6f59e640 1 -- 172.21.15.84:0/3366218508 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d700cf1a0 con 0x7f5d700ceda0 2023-10-11T06:13:58.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.527+0000 7f5d6e59c640 1 -- 172.21.15.84:0/3366218508 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439030380 0 0) 0x7f5d70160890 con 0x7f5d7015c2b0 2023-10-11T06:13:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.528+0000 7f5d6f59e640 1 --2- 172.21.15.84:0/3366218508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d700d0880 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5d700cd050 tx=0x7f5d6802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cb2e4da36da7a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d6e59c640 1 -- 172.21.15.84:0/3366218508 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6802cb20 con 0x7f5d700ceda0 2023-10-11T06:13:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d6e59c640 1 -- 172.21.15.84:0/3366218508 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6802cc80 con 0x7f5d700ceda0 2023-10-11T06:13:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d6e59c640 1 -- 172.21.15.84:0/3366218508 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680343f0 con 0x7f5d700ceda0 2023-10-11T06:13:58.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d750ca640 1 -- 172.21.15.84:0/3366218508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 msgr2=0x7f5d700d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d750ca640 1 --2- 172.21.15.84:0/3366218508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d700d0880 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5d700cd050 tx=0x7f5d6802bf80 comp rx=0 tx=0).stop 2023-10-11T06:13:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d750ca640 1 -- 172.21.15.84:0/3366218508 shutdown_connections 2023-10-11T06:13:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d750ca640 1 --2- 172.21.15.84:0/3366218508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d700d0880 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d750ca640 1 --2- 172.21.15.84:0/3366218508 >> v2:172.21.15.99:3300/0 conn(0x7f5d700cfb20 0x7f5d7006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.529+0000 7f5d750ca640 1 -- 172.21.15.84:0/3366218508 >> 172.21.15.84:0/3366218508 conn(0x7f5d7006af40 msgr2=0x7f5d7006b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.530+0000 7f5d750ca640 1 -- 172.21.15.84:0/3366218508 shutdown_connections 2023-10-11T06:13:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.530+0000 7f5d750ca640 1 -- 172.21.15.84:0/3366218508 wait complete. 2023-10-11T06:13:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.532+0000 7f5d750ca640 1 Processor -- start 2023-10-11T06:13:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.532+0000 7f5d750ca640 1 -- start start 2023-10-11T06:13:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.533+0000 7f5d750ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d7015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.533+0000 7f5d750ca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d700cfb20 0x7f5d700c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d750ca640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d7015c2b0 0x7f5d701539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d750ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7015a6f0 con 0x7f5d700ceda0 2023-10-11T06:13:58.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d750ca640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d700b2f30 con 0x7f5d700cfb20 2023-10-11T06:13:58.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d750ca640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d700b2600 con 0x7f5d7015c2b0 2023-10-11T06:13:58.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d7015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d7015a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55144/0 (socket says 172.21.15.84:55144) 2023-10-11T06:13:58.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 -- 172.21.15.84:0/1936611730 learned_addr learned my addr 172.21.15.84:0/1936611730 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:58.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d67fff640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d700cfb20 0x7f5d700c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6f59e640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d7015c2b0 0x7f5d701539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 -- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d7015c2b0 msgr2=0x7f5d701539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d7015c2b0 0x7f5d701539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 -- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d700cfb20 msgr2=0x7f5d700c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d700cfb20 0x7f5d700c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6ed9d640 1 -- 172.21.15.84:0/1936611730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68009b30 con 0x7f5d700ceda0 2023-10-11T06:13:58.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.534+0000 7f5d6f59e640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d7015c2b0 0x7f5d701539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:58.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.535+0000 7f5d6ed9d640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d7015a060 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c4d0 tx=0x7f5d6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.535+0000 7f5d677fe640 1 -- 172.21.15.84:0/1936611730 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600074e0 con 0x7f5d700ceda0 2023-10-11T06:13:58.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.535+0000 7f5d677fe640 1 -- 172.21.15.84:0/1936611730 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60007660 con 0x7f5d700ceda0 2023-10-11T06:13:58.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.535+0000 7f5d677fe640 1 -- 172.21.15.84:0/1936611730 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60004a20 con 0x7f5d700ceda0 2023-10-11T06:13:58.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.535+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d700b28e0 con 0x7f5d700ceda0 2023-10-11T06:13:58.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.535+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d70159810 con 0x7f5d700ceda0 2023-10-11T06:13:58.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.536+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d70067090 con 0x7f5d700ceda0 2023-10-11T06:13:58.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.537+0000 7f5d677fe640 1 -- 172.21.15.84:0/1936611730 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5d60004b80 con 0x7f5d700ceda0 2023-10-11T06:13:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.538+0000 7f5d677fe640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d5006a830 0x7f5d5006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.538+0000 7f5d677fe640 1 -- 172.21.15.84:0/1936611730 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d600881e0 con 0x7f5d700ceda0 2023-10-11T06:13:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.539+0000 7f5d67fff640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d5006a830 0x7f5d5006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.539+0000 7f5d67fff640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d5006a830 0x7f5d5006cce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5d700b1c00 tx=0x7f5d58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.543+0000 7f5d677fe640 1 -- 172.21.15.84:0/1936611730 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5d60053450 con 0x7f5d700ceda0 2023-10-11T06:13:58.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.732+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d700c9890 con 0x7f5d700ceda0 2023-10-11T06:13:58.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.733+0000 7f5d677fe640 1 -- 172.21.15.84:0/1936611730 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d60056a70 con 0x7f5d700ceda0 2023-10-11T06:13:58.735 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:13:58.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.735+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d5006a830 msgr2=0x7f5d5006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.735+0000 7f5d750ca640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d5006a830 0x7f5d5006cce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5d700b1c00 tx=0x7f5d58008040 comp rx=0 tx=0).stop 2023-10-11T06:13:58.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.735+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 msgr2=0x7f5d7015a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.735+0000 7f5d750ca640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d7015a060 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c4d0 tx=0x7f5d6000c990 comp rx=0 tx=0).stop 2023-10-11T06:13:58.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 shutdown_connections 2023-10-11T06:13:58.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d5006a830 0x7f5d5006cce0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d7015c2b0 0x7f5d701539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d700cfb20 0x7f5d700c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 --2- 172.21.15.84:0/1936611730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d700ceda0 0x7f5d7015a060 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 >> 172.21.15.84:0/1936611730 conn(0x7f5d7006af40 msgr2=0x7f5d7015b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:58.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 shutdown_connections 2023-10-11T06:13:58.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.736+0000 7f5d750ca640 1 -- 172.21.15.84:0/1936611730 wait complete. 2023-10-11T06:13:58.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:13:58.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.969+0000 7f4055234640 1 Processor -- start 2023-10-11T06:13:58.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.969+0000 7f4055234640 1 -- start start 2023-10-11T06:13:58.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f4055234640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f405014ff30 0x7f40500c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f4055234640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4050150300 0x7f40500cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f4055234640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500c9b90 con 0x7f40500cc7c0 2023-10-11T06:13:58.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f4055234640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405015da10 con 0x7f405014ff30 2023-10-11T06:13:58.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f4055234640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405015ce00 con 0x7f4050150300 2023-10-11T06:13:58.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f404ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f405014ff30 0x7f40500c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f404ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f405014ff30 0x7f40500c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59108/0 (socket says 172.21.15.84:59108) 2023-10-11T06:13:58.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.971+0000 7f404ed9d640 1 -- 172.21.15.84:0/498582467 learned_addr learned my addr 172.21.15.84:0/498582467 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:58.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.972+0000 7f404ed9d640 1 -- 172.21.15.84:0/498582467 >> v2:172.21.15.99:3300/0 conn(0x7f4050150300 msgr2=0x7f40500cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:58.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.972+0000 7f4047fff640 1 --2- 172.21.15.84:0/498582467 >> v2:172.21.15.99:3300/0 conn(0x7f4050150300 0x7f40500cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.972+0000 7f404ed9d640 1 --2- 172.21.15.84:0/498582467 >> v2:172.21.15.99:3300/0 conn(0x7f4050150300 0x7f40500cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.972+0000 7f404ed9d640 1 -- 172.21.15.84:0/498582467 >> v1:172.21.15.84:6789/0 conn(0x7f40500cc7c0 legacy=0x7f40500ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.972+0000 7f404ed9d640 1 -- 172.21.15.84:0/498582467 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405015d0e0 con 0x7f405014ff30 2023-10-11T06:13:58.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.972+0000 7f4047fff640 1 --2- 172.21.15.84:0/498582467 >> v2:172.21.15.99:3300/0 conn(0x7f4050150300 0x7f40500cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:58.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.972+0000 7f404e59c640 1 -- 172.21.15.84:0/498582467 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594715348 0 0) 0x7f40500c9b90 con 0x7f40500cc7c0 2023-10-11T06:13:58.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.973+0000 7f404ed9d640 1 --2- 172.21.15.84:0/498582467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405014ff30 0x7f40500c9650 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f405006fb50 tx=0x7f404002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae4b3bace181613 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.973+0000 7f404e59c640 1 -- 172.21.15.84:0/498582467 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404002eea0 con 0x7f405014ff30 2023-10-11T06:13:58.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.973+0000 7f404e59c640 1 -- 172.21.15.84:0/498582467 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040036440 con 0x7f405014ff30 2023-10-11T06:13:58.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.973+0000 7f404e59c640 1 -- 172.21.15.84:0/498582467 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400367d0 con 0x7f405014ff30 2023-10-11T06:13:58.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 -- 172.21.15.84:0/498582467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405014ff30 msgr2=0x7f40500c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 --2- 172.21.15.84:0/498582467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405014ff30 0x7f40500c9650 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f405006fb50 tx=0x7f404002e0f0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 -- 172.21.15.84:0/498582467 shutdown_connections 2023-10-11T06:13:58.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 --2- 172.21.15.84:0/498582467 >> v2:172.21.15.99:3300/0 conn(0x7f4050150300 0x7f40500cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 --2- 172.21.15.84:0/498582467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405014ff30 0x7f40500c9650 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 -- 172.21.15.84:0/498582467 >> 172.21.15.84:0/498582467 conn(0x7f4050069c70 msgr2=0x7f405006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:58.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 -- 172.21.15.84:0/498582467 shutdown_connections 2023-10-11T06:13:58.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.974+0000 7f4055234640 1 -- 172.21.15.84:0/498582467 wait complete. 2023-10-11T06:13:58.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.976+0000 7f4055234640 1 Processor -- start 2023-10-11T06:13:58.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.976+0000 7f4055234640 1 -- start start 2023-10-11T06:13:58.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f4055234640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f405014ff30 0x7f4050143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f4055234640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050150300 0x7f40500c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f4055234640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40500c68c0 0x7f4050158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f4055234640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c5820 con 0x7f40500c68c0 2023-10-11T06:13:58.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f4055234640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c5b00 con 0x7f4050150300 2023-10-11T06:13:58.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f4055234640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c5de0 con 0x7f405014ff30 2023-10-11T06:13:58.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f404f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40500c68c0 0x7f4050158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f404f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40500c68c0 0x7f4050158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59114/0 (socket says 172.21.15.84:59114) 2023-10-11T06:13:58.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.977+0000 7f404f59e640 1 -- 172.21.15.84:0/3855721707 learned_addr learned my addr 172.21.15.84:0/3855721707 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:58.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f4047fff640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050150300 0x7f40500c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f404ed9d640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f405014ff30 0x7f4050143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f404f59e640 1 -- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f405014ff30 msgr2=0x7f4050143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f404f59e640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f405014ff30 0x7f4050143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f404f59e640 1 -- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050150300 msgr2=0x7f40500c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:58.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f404f59e640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050150300 0x7f40500c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:58.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f404f59e640 1 -- 172.21.15.84:0/3855721707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404000b770 con 0x7f40500c68c0 2023-10-11T06:13:58.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f4047fff640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050150300 0x7f40500c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:58.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f404f59e640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40500c68c0 0x7f4050158250 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f404800c7c0 tx=0x7f404800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.978+0000 7f40477fe640 1 -- 172.21.15.84:0/3855721707 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048003da0 con 0x7f40500c68c0 2023-10-11T06:13:58.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.979+0000 7f40477fe640 1 -- 172.21.15.84:0/3855721707 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4048005070 con 0x7f40500c68c0 2023-10-11T06:13:58.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.979+0000 7f40477fe640 1 -- 172.21.15.84:0/3855721707 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048005420 con 0x7f40500c68c0 2023-10-11T06:13:58.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.979+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500c60c0 con 0x7f40500c68c0 2023-10-11T06:13:58.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.979+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4050143660 con 0x7f40500c68c0 2023-10-11T06:13:58.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.980+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405006ab90 con 0x7f40500c68c0 2023-10-11T06:13:58.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.981+0000 7f40477fe640 1 -- 172.21.15.84:0/3855721707 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4048005580 con 0x7f40500c68c0 2023-10-11T06:13:58.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.982+0000 7f40477fe640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f403006a830 0x7f403006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:58.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.982+0000 7f40477fe640 1 -- 172.21.15.84:0/3855721707 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4048088900 con 0x7f40500c68c0 2023-10-11T06:13:58.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.982+0000 7f404ed9d640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f403006a830 0x7f403006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:58.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.983+0000 7f404ed9d640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f403006a830 0x7f403006cce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4040038000 tx=0x7f4040008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:58.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:58.986+0000 7f40477fe640 1 -- 172.21.15.84:0/3855721707 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4048053ae0 con 0x7f40500c68c0 2023-10-11T06:13:59.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.172+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4050073ed0 con 0x7f40500c68c0 2023-10-11T06:13:59.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.173+0000 7f40477fe640 1 -- 172.21.15.84:0/3855721707 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4048017020 con 0x7f40500c68c0 2023-10-11T06:13:59.176 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:13:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f403006a830 msgr2=0x7f403006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f403006a830 0x7f403006cce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4040038000 tx=0x7f4040008040 comp rx=0 tx=0).stop 2023-10-11T06:13:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40500c68c0 msgr2=0x7f4050158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40500c68c0 0x7f4050158250 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f404800c7c0 tx=0x7f404800cc80 comp rx=0 tx=0).stop 2023-10-11T06:13:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 shutdown_connections 2023-10-11T06:13:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f403006a830 0x7f403006cce0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40500c68c0 0x7f4050158250 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050150300 0x7f40500c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 --2- 172.21.15.84:0/3855721707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f405014ff30 0x7f4050143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.175+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 >> 172.21.15.84:0/3855721707 conn(0x7f4050069c70 msgr2=0x7f40500ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.176+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 shutdown_connections 2023-10-11T06:13:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.176+0000 7f4055234640 1 -- 172.21.15.84:0/3855721707 wait complete. 2023-10-11T06:13:59.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:13:59.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.403+0000 7fb2eb349640 1 Processor -- start 2023-10-11T06:13:59.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.403+0000 7fb2eb349640 1 -- start start 2023-10-11T06:13:59.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.405+0000 7fb2eb349640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2e414ff30 0x7fb2e40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.405+0000 7fb2eb349640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2e4150300 0x7fb2e40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.405+0000 7fb2eb349640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40c9b90 con 0x7fb2e40cc7c0 2023-10-11T06:13:59.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.405+0000 7fb2eb349640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e415da10 con 0x7fb2e414ff30 2023-10-11T06:13:59.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.405+0000 7fb2eb349640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e415ce00 con 0x7fb2e4150300 2023-10-11T06:13:59.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e90e5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2e414ff30 0x7fb2e40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e90e5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2e414ff30 0x7fb2e40c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59124/0 (socket says 172.21.15.84:59124) 2023-10-11T06:13:59.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e90e5640 1 -- 172.21.15.84:0/31108324 learned_addr learned my addr 172.21.15.84:0/31108324 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:59.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e88e4640 1 --2- 172.21.15.84:0/31108324 >> v2:172.21.15.99:3300/0 conn(0x7fb2e4150300 0x7fb2e40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e90e5640 1 -- 172.21.15.84:0/31108324 >> v2:172.21.15.99:3300/0 conn(0x7fb2e4150300 msgr2=0x7fb2e40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:59.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e90e5640 1 --2- 172.21.15.84:0/31108324 >> v2:172.21.15.99:3300/0 conn(0x7fb2e4150300 0x7fb2e40cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e90e5640 1 -- 172.21.15.84:0/31108324 >> v1:172.21.15.84:6789/0 conn(0x7fb2e40cc7c0 legacy=0x7fb2e40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e90e5640 1 -- 172.21.15.84:0/31108324 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e415d0e0 con 0x7fb2e414ff30 2023-10-11T06:13:59.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.406+0000 7fb2e88e4640 1 --2- 172.21.15.84:0/31108324 >> v2:172.21.15.99:3300/0 conn(0x7fb2e4150300 0x7fb2e40cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:13:59.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.407+0000 7fb2e90e5640 1 --2- 172.21.15.84:0/31108324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e414ff30 0x7fb2e40c9650 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb2e406fb50 tx=0x7fb2e002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417470e88170e672 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:59.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.407+0000 7fb2dbfff640 1 -- 172.21.15.84:0/31108324 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0031880 con 0x7fb2e414ff30 2023-10-11T06:13:59.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2dbfff640 1 -- 172.21.15.84:0/31108324 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e00319e0 con 0x7fb2e414ff30 2023-10-11T06:13:59.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2dbfff640 1 -- 172.21.15.84:0/31108324 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0031d70 con 0x7fb2e414ff30 2023-10-11T06:13:59.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 -- 172.21.15.84:0/31108324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e414ff30 msgr2=0x7fb2e40c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 --2- 172.21.15.84:0/31108324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e414ff30 0x7fb2e40c9650 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb2e406fb50 tx=0x7fb2e002b920 comp rx=0 tx=0).stop 2023-10-11T06:13:59.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 -- 172.21.15.84:0/31108324 shutdown_connections 2023-10-11T06:13:59.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 --2- 172.21.15.84:0/31108324 >> v2:172.21.15.99:3300/0 conn(0x7fb2e4150300 0x7fb2e40cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 --2- 172.21.15.84:0/31108324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e414ff30 0x7fb2e40c9650 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 -- 172.21.15.84:0/31108324 >> 172.21.15.84:0/31108324 conn(0x7fb2e4069c70 msgr2=0x7fb2e406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:59.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 -- 172.21.15.84:0/31108324 shutdown_connections 2023-10-11T06:13:59.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.408+0000 7fb2eb349640 1 -- 172.21.15.84:0/31108324 wait complete. 2023-10-11T06:13:59.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.410+0000 7fb2eb349640 1 Processor -- start 2023-10-11T06:13:59.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.410+0000 7fb2eb349640 1 -- start start 2023-10-11T06:13:59.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2eb349640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e414ff30 0x7fb2e413df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2eb349640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2e4150300 0x7fb2e40c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2eb349640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e40c8280 0x7fb2e406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2eb349640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c76e0 con 0x7fb2e40c8280 2023-10-11T06:13:59.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2eb349640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b3f00 con 0x7fb2e414ff30 2023-10-11T06:13:59.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2eb349640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b2e60 con 0x7fb2e4150300 2023-10-11T06:13:59.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2e98e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e40c8280 0x7fb2e406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2e98e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e40c8280 0x7fb2e406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59126/0 (socket says 172.21.15.84:59126) 2023-10-11T06:13:59.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.411+0000 7fb2e98e6640 1 -- 172.21.15.84:0/3056646978 learned_addr learned my addr 172.21.15.84:0/3056646978 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:59.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2e88e4640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2e4150300 0x7fb2e40c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2e98e6640 1 -- 172.21.15.84:0/3056646978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2e4150300 msgr2=0x7fb2e40c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2e98e6640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2e4150300 0x7fb2e40c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2e98e6640 1 -- 172.21.15.84:0/3056646978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e414ff30 msgr2=0x7fb2e413df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2e98e6640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e414ff30 0x7fb2e413df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2e98e6640 1 -- 172.21.15.84:0/3056646978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e00094d0 con 0x7fb2e40c8280 2023-10-11T06:13:59.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2e98e6640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e40c8280 0x7fb2e406cc70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00c680 tx=0x7fb2dc00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:59.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2d9ffb640 1 -- 172.21.15.84:0/3056646978 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc003b70 con 0x7fb2e40c8280 2023-10-11T06:13:59.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2d9ffb640 1 -- 172.21.15.84:0/3056646978 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc003cd0 con 0x7fb2e40c8280 2023-10-11T06:13:59.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2d9ffb640 1 -- 172.21.15.84:0/3056646978 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc005310 con 0x7fb2e40c8280 2023-10-11T06:13:59.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e40b3140 con 0x7fb2e40c8280 2023-10-11T06:13:59.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.412+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40b3700 con 0x7fb2e40c8280 2023-10-11T06:13:59.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.413+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e406ab90 con 0x7fb2e40c8280 2023-10-11T06:13:59.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.414+0000 7fb2d9ffb640 1 -- 172.21.15.84:0/3056646978 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb2dc005470 con 0x7fb2e40c8280 2023-10-11T06:13:59.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.415+0000 7fb2d9ffb640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb2c806a830 0x7fb2c806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.415+0000 7fb2d9ffb640 1 -- 172.21.15.84:0/3056646978 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2dc088cf0 con 0x7fb2e40c8280 2023-10-11T06:13:59.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.416+0000 7fb2e90e5640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb2c806a830 0x7fb2c806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.416+0000 7fb2e90e5640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb2c806a830 0x7fb2c806cce0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0036000 tx=0x7fb2e0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:59.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.419+0000 7fb2d9ffb640 1 -- 172.21.15.84:0/3056646978 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb2dc08d070 con 0x7fb2e40c8280 2023-10-11T06:13:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.593+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2e4073ed0 con 0x7fb2e40c8280 2023-10-11T06:13:59.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.593+0000 7fb2d9ffb640 1 -- 172.21.15.84:0/3056646978 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2dc054000 con 0x7fb2e40c8280 2023-10-11T06:13:59.596 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:13:59.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb2c806a830 msgr2=0x7fb2c806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb2c806a830 0x7fb2c806cce0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0036000 tx=0x7fb2e0030000 comp rx=0 tx=0).stop 2023-10-11T06:13:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e40c8280 msgr2=0x7fb2e406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e40c8280 0x7fb2e406cc70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00c680 tx=0x7fb2dc00cb40 comp rx=0 tx=0).stop 2023-10-11T06:13:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 shutdown_connections 2023-10-11T06:13:59.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb2c806a830 0x7fb2c806cce0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2e40c8280 0x7fb2e406cc70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2e4150300 0x7fb2e40c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 --2- 172.21.15.84:0/3056646978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e414ff30 0x7fb2e413df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 >> 172.21.15.84:0/3056646978 conn(0x7fb2e4069c70 msgr2=0x7fb2e40ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 shutdown_connections 2023-10-11T06:13:59.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.596+0000 7fb2eb349640 1 -- 172.21.15.84:0/3056646978 wait complete. 2023-10-11T06:13:59.614 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:13:59.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.821+0000 7f1815c34640 1 Processor -- start 2023-10-11T06:13:59.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.822+0000 7f1815c34640 1 -- start start 2023-10-11T06:13:59.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.823+0000 7f1815c34640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f181015c290 0x7f18100c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.823+0000 7f1815c34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18100cb6c0 0x7f18100cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.823+0000 7f1815c34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181015fe40 con 0x7f18100c9190 2023-10-11T06:13:59.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.823+0000 7f1815c34640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18100cad70 con 0x7f18100cb6c0 2023-10-11T06:13:59.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.823+0000 7f1815c34640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181006ff30 con 0x7f181015c290 2023-10-11T06:13:59.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.823+0000 7f180ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18100cb6c0 0x7f18100cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18100cb6c0 0x7f18100cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59128/0 (socket says 172.21.15.84:59128) 2023-10-11T06:13:59.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180ffff640 1 -- 172.21.15.84:0/1862366944 learned_addr learned my addr 172.21.15.84:0/1862366944 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:59.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f1807fff640 1 --1- 172.21.15.84:0/1862366944 >> v1:172.21.15.84:6789/0 conn(0x7f18100c9190 0x7f18100cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34310/0 (socket says 172.21.15.84:34310) 2023-10-11T06:13:59.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180f7fe640 1 --2- 172.21.15.84:0/1862366944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f181015c290 0x7f18100c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180ffff640 1 -- 172.21.15.84:0/1862366944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f181015c290 msgr2=0x7f18100c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:13:59.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180ffff640 1 --2- 172.21.15.84:0/1862366944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f181015c290 0x7f18100c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180ffff640 1 -- 172.21.15.84:0/1862366944 >> v1:172.21.15.84:6789/0 conn(0x7f18100c9190 legacy=0x7f18100cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180ffff640 1 -- 172.21.15.84:0/1862366944 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18101600e0 con 0x7f18100cb6c0 2023-10-11T06:13:59.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.824+0000 7f180effd640 1 -- 172.21.15.84:0/1862366944 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225853412 0 0) 0x7f181015fe40 con 0x7f18100c9190 2023-10-11T06:13:59.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.825+0000 7f180ffff640 1 --2- 172.21.15.84:0/1862366944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100cb6c0 0x7f18100cc410 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1808009e50 tx=0x7f180802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c2891738667271 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:59.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.825+0000 7f180effd640 1 -- 172.21.15.84:0/1862366944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180802cb00 con 0x7f18100cb6c0 2023-10-11T06:13:59.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.825+0000 7f180effd640 1 -- 172.21.15.84:0/1862366944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180802cc60 con 0x7f18100cb6c0 2023-10-11T06:13:59.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.825+0000 7f180effd640 1 -- 172.21.15.84:0/1862366944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080343f0 con 0x7f18100cb6c0 2023-10-11T06:13:59.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 -- 172.21.15.84:0/1862366944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100cb6c0 msgr2=0x7f18100cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 --2- 172.21.15.84:0/1862366944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100cb6c0 0x7f18100cc410 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1808009e50 tx=0x7f180802bf60 comp rx=0 tx=0).stop 2023-10-11T06:13:59.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 -- 172.21.15.84:0/1862366944 shutdown_connections 2023-10-11T06:13:59.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 --2- 172.21.15.84:0/1862366944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100cb6c0 0x7f18100cc410 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 --2- 172.21.15.84:0/1862366944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f181015c290 0x7f18100c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 -- 172.21.15.84:0/1862366944 >> 172.21.15.84:0/1862366944 conn(0x7f1810069c70 msgr2=0x7f181006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:13:59.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 -- 172.21.15.84:0/1862366944 shutdown_connections 2023-10-11T06:13:59.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.826+0000 7f1815c34640 1 -- 172.21.15.84:0/1862366944 wait complete. 2023-10-11T06:13:59.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.828+0000 7f1815c34640 1 Processor -- start 2023-10-11T06:13:59.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.828+0000 7f1815c34640 1 -- start start 2023-10-11T06:13:59.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f1815c34640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100c9190 0x7f18100b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f1815c34640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18100cb6c0 0x7f18100c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f1815c34640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015c290 0x7f18100c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f1815c34640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18100b34e0 con 0x7f18100c9190 2023-10-11T06:13:59.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f1815c34640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18100b3e40 con 0x7f181015c290 2023-10-11T06:13:59.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f1815c34640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181014fe30 con 0x7f18100cb6c0 2023-10-11T06:13:59.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f180f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100c9190 0x7f18100b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f180f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100c9190 0x7f18100b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59132/0 (socket says 172.21.15.84:59132) 2023-10-11T06:13:59.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.829+0000 7f180f7fe640 1 -- 172.21.15.84:0/2551264040 learned_addr learned my addr 172.21.15.84:0/2551264040 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:13:59.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180f7fe640 1 -- 172.21.15.84:0/2551264040 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18100cb6c0 msgr2=0x7f18100c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180f7fe640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18100cb6c0 0x7f18100c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180f7fe640 1 -- 172.21.15.84:0/2551264040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015c290 msgr2=0x7f18100c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:13:59.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180f7fe640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015c290 0x7f18100c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:13:59.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180f7fe640 1 -- 172.21.15.84:0/2551264040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1808009b30 con 0x7f18100c9190 2023-10-11T06:13:59.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180f7fe640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100c9190 0x7f18100b3900 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f180000d800 tx=0x7f180000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:59.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180cff9640 1 -- 172.21.15.84:0/2551264040 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000096c0 con 0x7f18100c9190 2023-10-11T06:13:59.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180cff9640 1 -- 172.21.15.84:0/2551264040 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1800009820 con 0x7f18100c9190 2023-10-11T06:13:59.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f180cff9640 1 -- 172.21.15.84:0/2551264040 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800009bd0 con 0x7f18100c9190 2023-10-11T06:13:59.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18100b2860 con 0x7f18100c9190 2023-10-11T06:13:59.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.830+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18100b2e20 con 0x7f18100c9190 2023-10-11T06:13:59.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.831+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1810073ed0 con 0x7f18100c9190 2023-10-11T06:13:59.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.832+0000 7f180cff9640 1 -- 172.21.15.84:0/2551264040 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f1800009d30 con 0x7f18100c9190 2023-10-11T06:13:59.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.833+0000 7f180cff9640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f17f006a830 0x7f17f006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:13:59.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.833+0000 7f180cff9640 1 -- 172.21.15.84:0/2551264040 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18000897d0 con 0x7f18100c9190 2023-10-11T06:13:59.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.833+0000 7f1807fff640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f17f006a830 0x7f17f006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:13:59.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.834+0000 7f1807fff640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f17f006a830 0x7f17f006cce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17f8007fd0 tx=0x7f17f8007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:13:59.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:13:59.837+0000 7f180cff9640 1 -- 172.21.15.84:0/2551264040 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f1800054a40 con 0x7f18100c9190 2023-10-11T06:14:00.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.036+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f18100722b0 con 0x7f18100c9190 2023-10-11T06:14:00.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.036+0000 7f180cff9640 1 -- 172.21.15.84:0/2551264040 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1800058060 con 0x7f18100c9190 2023-10-11T06:14:00.039 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:14:00.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f17f006a830 msgr2=0x7f17f006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f17f006a830 0x7f17f006cce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17f8007fd0 tx=0x7f17f8007920 comp rx=0 tx=0).stop 2023-10-11T06:14:00.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100c9190 msgr2=0x7f18100b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100c9190 0x7f18100b3900 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f180000d800 tx=0x7f180000dcc0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 shutdown_connections 2023-10-11T06:14:00.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f17f006a830 0x7f17f006cce0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f181015c290 0x7f18100c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18100cb6c0 0x7f18100c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 --2- 172.21.15.84:0/2551264040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18100c9190 0x7f18100b3900 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 >> 172.21.15.84:0/2551264040 conn(0x7f1810069c70 msgr2=0x7f18100ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:00.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 shutdown_connections 2023-10-11T06:14:00.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.039+0000 7f1815c34640 1 -- 172.21.15.84:0/2551264040 wait complete. 2023-10-11T06:14:00.059 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:14:00.059 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:00.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:00.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.366+0000 7f0757d27640 1 Processor -- start 2023-10-11T06:14:00.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.366+0000 7f0757d27640 1 -- start start 2023-10-11T06:14:00.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.367+0000 7f0757d27640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 0x7f07500c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.367+0000 7f0757d27640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07500adb90 0x7f07500caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.367+0000 7f0757d27640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07501593f0 con 0x7f07500cd300 2023-10-11T06:14:00.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.367+0000 7f0757d27640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075015a340 con 0x7f07500adb90 2023-10-11T06:14:00.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.367+0000 7f0757d27640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07500ce4a0 con 0x7f07500ce0d0 2023-10-11T06:14:00.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f07562c4640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f07500cd300 0x7f07500b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34320/0 (socket says 172.21.15.84:34320) 2023-10-11T06:14:00.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f07562c4640 1 -- 172.21.15.84:0/51220706 learned_addr learned my addr 172.21.15.84:0/51220706 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:00.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0755ac3640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 0x7f07500c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783511713 0 0) 0x7f07501593f0 con 0x7f07500cd300 2023-10-11T06:14:00.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f073c003520 con 0x7f07500cd300 2023-10-11T06:14:00.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f07552c2640 1 --2- 172.21.15.84:0/51220706 >> v2:172.21.15.99:3300/0 conn(0x7f07500adb90 0x7f07500caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012784408 0 0) 0x7f073c003520 con 0x7f07500cd300 2023-10-11T06:14:00.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 msgr2=0x7f07500c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:00.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 0x7f07500c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 >> v2:172.21.15.99:3300/0 conn(0x7f07500adb90 msgr2=0x7f07500caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:00.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 --2- 172.21.15.84:0/51220706 >> v2:172.21.15.99:3300/0 conn(0x7f07500adb90 0x7f07500caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07500ce5e0 con 0x7f07500cd300 2023-10-11T06:14:00.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f07480019f0 con 0x7f07500cd300 2023-10-11T06:14:00.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.368+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 >> v1:172.21.15.84:6789/0 conn(0x7f07500cd300 legacy=0x7f07500b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0754ac1640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f073c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0754ac1640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c0047c0 0x7f073c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0754ac1640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 0x7f073c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075015a340 con 0x7f073c003970 2023-10-11T06:14:00.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07500ce4a0 con 0x7f073c0047c0 2023-10-11T06:14:00.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c00ba30 con 0x7f07500ce0d0 2023-10-11T06:14:00.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0755ac3640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f073c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f07552c2640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c0047c0 0x7f073c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0755ac3640 1 -- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 msgr2=0x7f073c005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:00.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f07562c4640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 0x7f073c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0755ac3640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 0x7f073c005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0755ac3640 1 -- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c0047c0 msgr2=0x7f073c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:00.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0755ac3640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c0047c0 0x7f073c004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f0755ac3640 1 -- 172.21.15.84:0/51220706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c003520 con 0x7f073c003970 2023-10-11T06:14:00.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f07552c2640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c0047c0 0x7f073c004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:00.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.369+0000 7f07562c4640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07500ce0d0 0x7f073c005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:00.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.370+0000 7f0755ac3640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f073c003d80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f074c00ba10 tx=0x7f073c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6122a1eccce07fbe server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:00.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c005b90 con 0x7f073c003970 2023-10-11T06:14:00.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c005cf0 con 0x7f073c003970 2023-10-11T06:14:00.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0754ac1640 1 -- 172.21.15.84:0/51220706 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c006270 con 0x7f073c003970 2023-10-11T06:14:00.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0757d27640 1 -- 172.21.15.84:0/51220706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 msgr2=0x7f073c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0757d27640 1 --2- 172.21.15.84:0/51220706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f073c003d80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f074c00ba10 tx=0x7f073c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f07562c4640 1 -- 172.21.15.84:0/51220706 reap_dead start 2023-10-11T06:14:00.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0757d27640 1 -- 172.21.15.84:0/51220706 shutdown_connections 2023-10-11T06:14:00.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0757d27640 1 -- 172.21.15.84:0/51220706 >> 172.21.15.84:0/51220706 conn(0x7f0750069c70 msgr2=0x7f075006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:00.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0757d27640 1 -- 172.21.15.84:0/51220706 shutdown_connections 2023-10-11T06:14:00.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.371+0000 7f0757d27640 1 -- 172.21.15.84:0/51220706 wait complete. 2023-10-11T06:14:00.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.373+0000 7f0757d27640 1 Processor -- start 2023-10-11T06:14:00.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.373+0000 7f0757d27640 1 -- start start 2023-10-11T06:14:00.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0757d27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f0750146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0757d27640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0750157b40 0x7f0750157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0757d27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07501501b0 0x7f0750150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0757d27640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07501584f0 con 0x7f073c003970 2023-10-11T06:14:00.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0757d27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0750146b40 con 0x7f07501501b0 2023-10-11T06:14:00.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0757d27640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0750150b60 con 0x7f0750157b40 2023-10-11T06:14:00.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0755ac3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f0750146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0755ac3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f0750146280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59144/0 (socket says 172.21.15.84:59144) 2023-10-11T06:14:00.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.374+0000 7f0755ac3640 1 -- 172.21.15.84:0/513282497 learned_addr learned my addr 172.21.15.84:0/513282497 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:00.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f07562c4640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07501501b0 0x7f0750150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f0755ac3640 1 -- 172.21.15.84:0/513282497 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0750157b40 msgr2=0x7f0750157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f0755ac3640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0750157b40 0x7f0750157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f0755ac3640 1 -- 172.21.15.84:0/513282497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07501501b0 msgr2=0x7f0750150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f0755ac3640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07501501b0 0x7f0750150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f0755ac3640 1 -- 172.21.15.84:0/513282497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750150e30 con 0x7f073c003970 2023-10-11T06:14:00.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f0755ac3640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f0750146280 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f074c037040 tx=0x7f074c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:00.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.375+0000 7f07467fc640 1 -- 172.21.15.84:0/513282497 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c003e50 con 0x7f073c003970 2023-10-11T06:14:00.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.376+0000 7f07467fc640 1 -- 172.21.15.84:0/513282497 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c0044a0 con 0x7f073c003970 2023-10-11T06:14:00.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.376+0000 7f07467fc640 1 -- 172.21.15.84:0/513282497 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c006290 con 0x7f073c003970 2023-10-11T06:14:00.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.376+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0750151110 con 0x7f073c003970 2023-10-11T06:14:00.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.376+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0750164fa0 con 0x7f073c003970 2023-10-11T06:14:00.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.377+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0750151630 con 0x7f073c003970 2023-10-11T06:14:00.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.378+0000 7f07467fc640 1 -- 172.21.15.84:0/513282497 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f074c030070 con 0x7f073c003970 2023-10-11T06:14:00.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.379+0000 7f07467fc640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f073406a860 0x7f073406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.379+0000 7f07467fc640 1 -- 172.21.15.84:0/513282497 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f074c0aaff0 con 0x7f073c003970 2023-10-11T06:14:00.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.379+0000 7f07552c2640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f073406a860 0x7f073406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.380+0000 7f07552c2640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f073406a860 0x7f073406cd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f075015c640 tx=0x7f07400092f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:00.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.384+0000 7f07467fc640 1 -- 172.21.15.84:0/513282497 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f074c0ddba0 con 0x7f073c003970 2023-10-11T06:14:00.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.574+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0750165130 con 0x7f073c003970 2023-10-11T06:14:00.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.576+0000 7f07467fc640 1 -- 172.21.15.84:0/513282497 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7f074c042090 con 0x7f073c003970 2023-10-11T06:14:00.579 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:00.581 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:00.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.579+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f073406a860 msgr2=0x7f073406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.579+0000 7f0757d27640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f073406a860 0x7f073406cd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f075015c640 tx=0x7f07400092f0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.579+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 msgr2=0x7f0750146280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.579+0000 7f0757d27640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f0750146280 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f074c037040 tx=0x7f074c004730 comp rx=0 tx=0).stop 2023-10-11T06:14:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 shutdown_connections 2023-10-11T06:14:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f073406a860 0x7f073406cd10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07501501b0 0x7f0750150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0750157b40 0x7f0750157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 --2- 172.21.15.84:0/513282497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f073c003970 0x7f0750146280 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 >> 172.21.15.84:0/513282497 conn(0x7f0750069c70 msgr2=0x7f075015ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:00.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 shutdown_connections 2023-10-11T06:14:00.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.580+0000 7f0757d27640 1 -- 172.21.15.84:0/513282497 wait complete. 2023-10-11T06:14:00.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:13:44.841435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:13:45.401366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:14:00.601 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:14:00.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.810+0000 7fddbad8e640 1 Processor -- start 2023-10-11T06:14:00.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.810+0000 7fddbad8e640 1 -- start start 2023-10-11T06:14:00.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.812+0000 7fddbad8e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fddb4150300 0x7fddb40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.812+0000 7fddbad8e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb40c9b90 con 0x7fddb40ce510 2023-10-11T06:14:00.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.812+0000 7fddbad8e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb415da10 con 0x7fddb414ff30 2023-10-11T06:14:00.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.812+0000 7fddbad8e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb40cc990 con 0x7fddb4150300 2023-10-11T06:14:00.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.812+0000 7fddb932b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fddb40ce510 0x7fddb40ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34332/0 (socket says 172.21.15.84:34332) 2023-10-11T06:14:00.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.812+0000 7fddb932b640 1 -- 172.21.15.84:0/2976088350 learned_addr learned my addr 172.21.15.84:0/2976088350 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:00.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.812+0000 7fddb3fff640 1 --2- 172.21.15.84:0/2976088350 >> v2:172.21.15.99:3300/0 conn(0x7fddb4150300 0x7fddb40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023356017 0 0) 0x7fddb40c9b90 con 0x7fddb40ce510 2023-10-11T06:14:00.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd9c003520 con 0x7fddb40ce510 2023-10-11T06:14:00.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008911419 0 0) 0x7fdd9c003520 con 0x7fddb40ce510 2023-10-11T06:14:00.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 >> v2:172.21.15.99:3300/0 conn(0x7fddb4150300 msgr2=0x7fddb40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:00.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 --2- 172.21.15.84:0/2976088350 >> v2:172.21.15.99:3300/0 conn(0x7fddb4150300 0x7fddb40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 >> v1:172.21.15.99:6789/0 conn(0x7fddb414ff30 legacy=0x7fddb40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb415ce00 con 0x7fddb40ce510 2023-10-11T06:14:00.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fddac0019f0 con 0x7fddb40ce510 2023-10-11T06:14:00.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 >> v1:172.21.15.84:6789/0 conn(0x7fddb40ce510 legacy=0x7fddb40ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fdd9c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd9c0047c0 0x7fdd9c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd9c005170 0x7fdd9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40cc990 con 0x7fdd9c003970 2023-10-11T06:14:00.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c00c080 con 0x7fdd9c0047c0 2023-10-11T06:14:00.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.813+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c00e450 con 0x7fdd9c005170 2023-10-11T06:14:00.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb8b2a640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fdd9c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb3fff640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd9c0047c0 0x7fdd9c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb932b640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd9c005170 0x7fdd9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb8b2a640 1 -- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd9c005170 msgr2=0x7fdd9c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:00.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb8b2a640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd9c005170 0x7fdd9c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb8b2a640 1 -- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd9c0047c0 msgr2=0x7fdd9c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:00.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb8b2a640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd9c0047c0 0x7fdd9c004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb8b2a640 1 -- 172.21.15.84:0/2976088350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c003520 con 0x7fdd9c003970 2023-10-11T06:14:00.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb932b640 1 -- 172.21.15.84:0/2976088350 reap_dead start 2023-10-11T06:14:00.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.814+0000 7fddb3fff640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd9c0047c0 0x7fdd9c004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.815+0000 7fddb8b2a640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fdd9c003d80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001750 tx=0x7fdd9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c717024a54b25b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda40052d0 con 0x7fdd9c003970 2023-10-11T06:14:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda402e050 con 0x7fdd9c003970 2023-10-11T06:14:00.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddb37fe640 1 -- 172.21.15.84:0/2976088350 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4003830 con 0x7fdd9c003970 2023-10-11T06:14:00.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddbad8e640 1 -- 172.21.15.84:0/2976088350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 msgr2=0x7fdd9c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fdd9c003d80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001750 tx=0x7fdd9c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddbad8e640 1 -- 172.21.15.84:0/2976088350 shutdown_connections 2023-10-11T06:14:00.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2976088350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fdd9c003d80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddbad8e640 1 -- 172.21.15.84:0/2976088350 >> 172.21.15.84:0/2976088350 conn(0x7fddb4069c70 msgr2=0x7fddb406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:00.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddbad8e640 1 -- 172.21.15.84:0/2976088350 shutdown_connections 2023-10-11T06:14:00.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.816+0000 7fddbad8e640 1 -- 172.21.15.84:0/2976088350 wait complete. 2023-10-11T06:14:00.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.818+0000 7fddbad8e640 1 Processor -- start 2023-10-11T06:14:00.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.818+0000 7fddbad8e640 1 -- start start 2023-10-11T06:14:00.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddbad8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fddb413d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddbad8e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb40c71c0 0x7fddb414e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddbad8e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb406ffa0 0x7fddb4070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddbad8e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb4148a70 con 0x7fdd9c003970 2023-10-11T06:14:00.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddbad8e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb4070950 con 0x7fddb406ffa0 2023-10-11T06:14:00.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddbad8e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40c8360 con 0x7fddb40c71c0 2023-10-11T06:14:00.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddb8b2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fddb413d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddb8b2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fddb413d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59162/0 (socket says 172.21.15.84:59162) 2023-10-11T06:14:00.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.819+0000 7fddb8b2a640 1 -- 172.21.15.84:0/2729400769 learned_addr learned my addr 172.21.15.84:0/2729400769 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:00.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb3fff640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb40c71c0 0x7fddb414e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb932b640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb406ffa0 0x7fddb4070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb8b2a640 1 -- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb40c71c0 msgr2=0x7fddb414e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb8b2a640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb40c71c0 0x7fddb414e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb8b2a640 1 -- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb406ffa0 msgr2=0x7fddb4070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:00.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb8b2a640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb406ffa0 0x7fddb4070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:00.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb8b2a640 1 -- 172.21.15.84:0/2729400769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb413d600 con 0x7fdd9c003970 2023-10-11T06:14:00.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb932b640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb406ffa0 0x7fddb4070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:00.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.820+0000 7fddb8b2a640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fddb413d0c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdda4005040 tx=0x7fdda4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:00.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.821+0000 7fddb17fa640 1 -- 172.21.15.84:0/2729400769 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda40076c0 con 0x7fdd9c003970 2023-10-11T06:14:00.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.821+0000 7fddb17fa640 1 -- 172.21.15.84:0/2729400769 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda4007a30 con 0x7fdd9c003970 2023-10-11T06:14:00.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.821+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb40c7750 con 0x7fdd9c003970 2023-10-11T06:14:00.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.821+0000 7fddb17fa640 1 -- 172.21.15.84:0/2729400769 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4006be0 con 0x7fdd9c003970 2023-10-11T06:14:00.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.821+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb40c7d10 con 0x7fdd9c003970 2023-10-11T06:14:00.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.822+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb406ab90 con 0x7fdd9c003970 2023-10-11T06:14:00.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.823+0000 7fddb17fa640 1 -- 172.21.15.84:0/2729400769 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdda4006d40 con 0x7fdd9c003970 2023-10-11T06:14:00.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.824+0000 7fddb17fa640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda006a830 0x7fdda006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:00.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.824+0000 7fddb17fa640 1 -- 172.21.15.84:0/2729400769 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdda40ab020 con 0x7fdd9c003970 2023-10-11T06:14:00.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.824+0000 7fddb3fff640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda006a830 0x7fdda006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:00.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.825+0000 7fddb3fff640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda006a830 0x7fdda006cce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fddb40c9f10 tx=0x7fdda8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:00.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:00.828+0000 7fddb17fa640 1 -- 172.21.15.84:0/2729400769 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdda40762a0 con 0x7fdd9c003970 2023-10-11T06:14:01.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.026+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddb4073ed0 con 0x7fdd9c003970 2023-10-11T06:14:01.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.026+0000 7fddb17fa640 1 -- 172.21.15.84:0/2729400769 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdda404b0a0 con 0x7fdd9c003970 2023-10-11T06:14:01.029 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:14:01.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda006a830 msgr2=0x7fdda006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda006a830 0x7fdda006cce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fddb40c9f10 tx=0x7fdda8008040 comp rx=0 tx=0).stop 2023-10-11T06:14:01.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 msgr2=0x7fddb413d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fddb413d0c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdda4005040 tx=0x7fdda4001120 comp rx=0 tx=0).stop 2023-10-11T06:14:01.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 shutdown_connections 2023-10-11T06:14:01.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdda006a830 0x7fdda006cce0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddb406ffa0 0x7fddb4070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fddb40c71c0 0x7fddb414e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 --2- 172.21.15.84:0/2729400769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9c003970 0x7fddb413d0c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 >> 172.21.15.84:0/2729400769 conn(0x7fddb4069c70 msgr2=0x7fddb40cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:01.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.029+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 shutdown_connections 2023-10-11T06:14:01.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.030+0000 7fddbad8e640 1 -- 172.21.15.84:0/2729400769 wait complete. 2023-10-11T06:14:01.051 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:14:01.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.265+0000 7fb46fdd0640 1 Processor -- start 2023-10-11T06:14:01.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.265+0000 7fb46fdd0640 1 -- start start 2023-10-11T06:14:01.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.267+0000 7fb46fdd0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb46815a0b0 0x7fb4680cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.267+0000 7fb46fdd0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb468071380 con 0x7fb4680c8d00 2023-10-11T06:14:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.267+0000 7fb46fdd0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46806de10 con 0x7fb4680ced30 2023-10-11T06:14:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.267+0000 7fb46fdd0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46806e0f0 con 0x7fb46815a0b0 2023-10-11T06:14:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.267+0000 7fb46db6c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb46815a0b0 0x7fb4680cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.267+0000 7fb46db6c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb46815a0b0 0x7fb4680cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59174/0 (socket says 172.21.15.84:59174) 2023-10-11T06:14:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.267+0000 7fb46db6c640 1 -- 172.21.15.84:0/3309096999 learned_addr learned my addr 172.21.15.84:0/3309096999 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:01.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.268+0000 7fb46db6c640 1 -- 172.21.15.84:0/3309096999 >> v1:172.21.15.99:6789/0 conn(0x7fb4680ced30 legacy=0x7fb46806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:01.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.268+0000 7fb46db6c640 1 -- 172.21.15.84:0/3309096999 >> v1:172.21.15.84:6789/0 conn(0x7fb4680c8d00 legacy=0x7fb4680ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.268+0000 7fb46db6c640 1 -- 172.21.15.84:0/3309096999 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468070dc0 con 0x7fb46815a0b0 2023-10-11T06:14:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.269+0000 7fb46db6c640 1 --2- 172.21.15.84:0/3309096999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb4680cb5a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb464009870 tx=0x7fb46402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf35056807541b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.269+0000 7fb46cb6a640 1 -- 172.21.15.84:0/3309096999 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4640317e0 con 0x7fb46815a0b0 2023-10-11T06:14:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.269+0000 7fb46cb6a640 1 -- 172.21.15.84:0/3309096999 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb464031940 con 0x7fb46815a0b0 2023-10-11T06:14:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.269+0000 7fb46cb6a640 1 -- 172.21.15.84:0/3309096999 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb464031cd0 con 0x7fb46815a0b0 2023-10-11T06:14:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.270+0000 7fb46fdd0640 1 -- 172.21.15.84:0/3309096999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 msgr2=0x7fb4680cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.270+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/3309096999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb4680cb5a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb464009870 tx=0x7fb46402b880 comp rx=0 tx=0).stop 2023-10-11T06:14:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.270+0000 7fb46fdd0640 1 -- 172.21.15.84:0/3309096999 shutdown_connections 2023-10-11T06:14:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.270+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/3309096999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb4680cb5a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.270+0000 7fb46fdd0640 1 -- 172.21.15.84:0/3309096999 >> 172.21.15.84:0/3309096999 conn(0x7fb468069c70 msgr2=0x7fb46806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.270+0000 7fb46fdd0640 1 -- 172.21.15.84:0/3309096999 shutdown_connections 2023-10-11T06:14:01.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.270+0000 7fb46fdd0640 1 -- 172.21.15.84:0/3309096999 wait complete. 2023-10-11T06:14:01.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.272+0000 7fb46fdd0640 1 Processor -- start 2023-10-11T06:14:01.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.272+0000 7fb46fdd0640 1 -- start start 2023-10-11T06:14:01.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46fdd0640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4680c8d00 0x7fb46815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46fdd0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4680ced30 0x7fb46815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46fdd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb46815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46fdd0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46815bc70 con 0x7fb46815a0b0 2023-10-11T06:14:01.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46fdd0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680b30f0 con 0x7fb4680ced30 2023-10-11T06:14:01.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46fdd0640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46815b060 con 0x7fb4680c8d00 2023-10-11T06:14:01.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46e36d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb46815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46e36d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb46815a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59176/0 (socket says 172.21.15.84:59176) 2023-10-11T06:14:01.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46e36d640 1 -- 172.21.15.84:0/175215893 learned_addr learned my addr 172.21.15.84:0/175215893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:01.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.273+0000 7fb46db6c640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4680c8d00 0x7fb46815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46d36b640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4680ced30 0x7fb46815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46e36d640 1 -- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4680c8d00 msgr2=0x7fb46815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46e36d640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4680c8d00 0x7fb46815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46e36d640 1 -- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4680ced30 msgr2=0x7fb46815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46e36d640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4680ced30 0x7fb46815a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46e36d640 1 -- 172.21.15.84:0/175215893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640094d0 con 0x7fb46815a0b0 2023-10-11T06:14:01.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46e36d640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb46815a9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c7b0 tx=0x7fb46000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:01.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb45e7fc640 1 -- 172.21.15.84:0/175215893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460003d80 con 0x7fb46815a0b0 2023-10-11T06:14:01.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb45e7fc640 1 -- 172.21.15.84:0/175215893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460005070 con 0x7fb46815a0b0 2023-10-11T06:14:01.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb45e7fc640 1 -- 172.21.15.84:0/175215893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460005420 con 0x7fb46815a0b0 2023-10-11T06:14:01.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46815b340 con 0x7fb46815a0b0 2023-10-11T06:14:01.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.274+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46815b900 con 0x7fb46815a0b0 2023-10-11T06:14:01.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.275+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46815be20 con 0x7fb46815a0b0 2023-10-11T06:14:01.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.276+0000 7fb45e7fc640 1 -- 172.21.15.84:0/175215893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb460005580 con 0x7fb46815a0b0 2023-10-11T06:14:01.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.278+0000 7fb45e7fc640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.278+0000 7fb45e7fc640 1 -- 172.21.15.84:0/175215893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb460088580 con 0x7fb46815a0b0 2023-10-11T06:14:01.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.278+0000 7fb46db6c640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.279+0000 7fb46db6c640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb464036000 tx=0x7fb46403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:01.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.283+0000 7fb45e7fc640 1 -- 172.21.15.84:0/175215893 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb46815be20 con 0x7fb46815a0b0 2023-10-11T06:14:01.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.473+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb46815be20 con 0x7fb46815a0b0 2023-10-11T06:14:01.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.474+0000 7fb45e7fc640 1 -- 172.21.15.84:0/175215893 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb46815be20 con 0x7fb46815a0b0 2023-10-11T06:14:01.477 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:14:01.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 msgr2=0x7fb44c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb464036000 tx=0x7fb46403d040 comp rx=0 tx=0).stop 2023-10-11T06:14:01.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 msgr2=0x7fb46815a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb46815a9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c7b0 tx=0x7fb46000cc70 comp rx=0 tx=0).stop 2023-10-11T06:14:01.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 shutdown_connections 2023-10-11T06:14:01.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb44c06a830 0x7fb44c06cce0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb46815a0b0 0x7fb46815a9c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4680ced30 0x7fb46815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 --2- 172.21.15.84:0/175215893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4680c8d00 0x7fb46815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 >> 172.21.15.84:0/175215893 conn(0x7fb468069c70 msgr2=0x7fb46806d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 shutdown_connections 2023-10-11T06:14:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.477+0000 7fb46fdd0640 1 -- 172.21.15.84:0/175215893 wait complete. 2023-10-11T06:14:01.497 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:14:01.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.718+0000 7fc931821640 1 Processor -- start 2023-10-11T06:14:01.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.718+0000 7fc931821640 1 -- start start 2023-10-11T06:14:01.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.719+0000 7fc931821640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc92c15a0b0 0x7fc92c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.719+0000 7fc931821640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.719+0000 7fc931821640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc92c0ceb60 0x7fc92c06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.719+0000 7fc931821640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c06e0f0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.719+0000 7fc931821640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0cbae0 con 0x7fc92c0ceb60 2023-10-11T06:14:01.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.719+0000 7fc931821640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0cefc0 con 0x7fc92c0c8d00 2023-10-11T06:14:01.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.719+0000 7fc92affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc92c15a0b0 0x7fc92c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc92c15a0b0 0x7fc92c0cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59182/0 (socket says 172.21.15.84:59182) 2023-10-11T06:14:01.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92affd640 1 -- 172.21.15.84:0/2158578262 learned_addr learned my addr 172.21.15.84:0/2158578262 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:01.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92b7fe640 1 --2- 172.21.15.84:0/2158578262 >> v2:172.21.15.99:3300/0 conn(0x7fc92c0ceb60 0x7fc92c06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc923fff640 1 --2- 172.21.15.84:0/2158578262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92affd640 1 -- 172.21.15.84:0/2158578262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 msgr2=0x7fc92c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:01.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92affd640 1 --2- 172.21.15.84:0/2158578262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92affd640 1 -- 172.21.15.84:0/2158578262 >> v2:172.21.15.99:3300/0 conn(0x7fc92c0ceb60 msgr2=0x7fc92c06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:01.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92affd640 1 --2- 172.21.15.84:0/2158578262 >> v2:172.21.15.99:3300/0 conn(0x7fc92c0ceb60 0x7fc92c06c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc92affd640 1 -- 172.21.15.84:0/2158578262 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c071380 con 0x7fc92c15a0b0 2023-10-11T06:14:01.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.720+0000 7fc923fff640 1 --2- 172.21.15.84:0/2158578262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:01.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.722+0000 7fc92affd640 1 --2- 172.21.15.84:0/2158578262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0cb5a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06e2f0 tx=0x7fc91c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a420c5763791df server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:01.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.722+0000 7fc92a7fc640 1 -- 172.21.15.84:0/2158578262 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c031740 con 0x7fc92c15a0b0 2023-10-11T06:14:01.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.722+0000 7fc92a7fc640 1 -- 172.21.15.84:0/2158578262 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c0318a0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.722+0000 7fc92a7fc640 1 -- 172.21.15.84:0/2158578262 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c031c30 con 0x7fc92c15a0b0 2023-10-11T06:14:01.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 -- 172.21.15.84:0/2158578262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 msgr2=0x7fc92c0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 --2- 172.21.15.84:0/2158578262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0cb5a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06e2f0 tx=0x7fc91c02b7e0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 -- 172.21.15.84:0/2158578262 shutdown_connections 2023-10-11T06:14:01.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 --2- 172.21.15.84:0/2158578262 >> v2:172.21.15.99:3300/0 conn(0x7fc92c0ceb60 0x7fc92c06c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 --2- 172.21.15.84:0/2158578262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 --2- 172.21.15.84:0/2158578262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0cb5a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 -- 172.21.15.84:0/2158578262 >> 172.21.15.84:0/2158578262 conn(0x7fc92c069c70 msgr2=0x7fc92c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 -- 172.21.15.84:0/2158578262 shutdown_connections 2023-10-11T06:14:01.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.723+0000 7fc931821640 1 -- 172.21.15.84:0/2158578262 wait complete. 2023-10-11T06:14:01.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.725+0000 7fc931821640 1 Processor -- start 2023-10-11T06:14:01.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.725+0000 7fc931821640 1 -- start start 2023-10-11T06:14:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc931821640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c1ef510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc931821640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c0ceb60 0x7fc92c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc931821640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc931821640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1efa50 con 0x7fc92c15a0b0 2023-10-11T06:14:01.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc931821640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c15ba10 con 0x7fc92c0ceb60 2023-10-11T06:14:01.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc931821640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1f0650 con 0x7fc92c0c8d00 2023-10-11T06:14:01.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc92b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc92b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59184/0 (socket says 172.21.15.84:59184) 2023-10-11T06:14:01.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.726+0000 7fc92b7fe640 1 -- 172.21.15.84:0/3978143874 learned_addr learned my addr 172.21.15.84:0/3978143874 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:01.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc923fff640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c0ceb60 0x7fc92c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc92affd640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c1ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc92b7fe640 1 -- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 msgr2=0x7fc92c1ef510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc92b7fe640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c1ef510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc92b7fe640 1 -- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c0ceb60 msgr2=0x7fc92c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc92b7fe640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c0ceb60 0x7fc92c15b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc92b7fe640 1 -- 172.21.15.84:0/3978143874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0094d0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc92b7fe640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0c5780 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fc924007e70 tx=0x7fc92400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc928ff9640 1 -- 172.21.15.84:0/3978143874 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924014620 con 0x7fc92c15a0b0 2023-10-11T06:14:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc928ff9640 1 -- 172.21.15.84:0/3978143874 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9240147a0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc928ff9640 1 -- 172.21.15.84:0/3978143874 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924014b50 con 0x7fc92c15a0b0 2023-10-11T06:14:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c0c5cc0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.727+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1f0bd0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.728+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c073ed0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.729+0000 7fc928ff9640 1 -- 172.21.15.84:0/3978143874 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc924014cb0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.731+0000 7fc928ff9640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc90c06a830 0x7fc90c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:01.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.731+0000 7fc928ff9640 1 -- 172.21.15.84:0/3978143874 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc924089950 con 0x7fc92c15a0b0 2023-10-11T06:14:01.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.731+0000 7fc92affd640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc90c06a830 0x7fc90c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:01.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.732+0000 7fc92affd640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc90c06a830 0x7fc90c06cce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc91c035000 tx=0x7fc91c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:01.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.736+0000 7fc928ff9640 1 -- 172.21.15.84:0/3978143874 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc924054bb0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.923+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc92c0c5540 con 0x7fc92c15a0b0 2023-10-11T06:14:01.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.924+0000 7fc928ff9640 1 -- 172.21.15.84:0/3978143874 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc9240581d0 con 0x7fc92c15a0b0 2023-10-11T06:14:01.927 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:14:01.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.926+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc90c06a830 msgr2=0x7fc90c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.926+0000 7fc931821640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc90c06a830 0x7fc90c06cce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc91c035000 tx=0x7fc91c008030 comp rx=0 tx=0).stop 2023-10-11T06:14:01.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.926+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 msgr2=0x7fc92c0c5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:01.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.926+0000 7fc931821640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0c5780 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fc924007e70 tx=0x7fc92400e4b0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 shutdown_connections 2023-10-11T06:14:01.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc90c06a830 0x7fc90c06cce0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc92c15a0b0 0x7fc92c0c5780 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c0ceb60 0x7fc92c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 --2- 172.21.15.84:0/3978143874 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c0c8d00 0x7fc92c1ef510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 >> 172.21.15.84:0/3978143874 conn(0x7fc92c069c70 msgr2=0x7fc92c0c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:01.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 shutdown_connections 2023-10-11T06:14:01.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:01.927+0000 7fc931821640 1 -- 172.21.15.84:0/3978143874 wait complete. 2023-10-11T06:14:01.951 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:14:02.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.160+0000 7fbac4c7a640 1 Processor -- start 2023-10-11T06:14:02.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.160+0000 7fbac4c7a640 1 -- start start 2023-10-11T06:14:02.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbac4c7a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbac00ced30 0x7fbac00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbac4c7a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00c8d00 0x7fbac006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbac4c7a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac0071380 con 0x7fbac015a0b0 2023-10-11T06:14:02.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbac4c7a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac006e0f0 con 0x7fbac00ced30 2023-10-11T06:14:02.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbac4c7a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00cbae0 con 0x7fbac00c8d00 2023-10-11T06:14:02.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbabdd9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbac00ced30 0x7fbac00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbabdd9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbac00ced30 0x7fbac00cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59186/0 (socket says 172.21.15.84:59186) 2023-10-11T06:14:02.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.162+0000 7fbabdd9b640 1 -- 172.21.15.84:0/2287811668 learned_addr learned my addr 172.21.15.84:0/2287811668 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:02.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.163+0000 7fbabed9d640 1 --2- 172.21.15.84:0/2287811668 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00c8d00 0x7fbac006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.163+0000 7fbabdd9b640 1 -- 172.21.15.84:0/2287811668 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00c8d00 msgr2=0x7fbac006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:02.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.163+0000 7fbabdd9b640 1 --2- 172.21.15.84:0/2287811668 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00c8d00 0x7fbac006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.163+0000 7fbabdd9b640 1 -- 172.21.15.84:0/2287811668 >> v1:172.21.15.99:6789/0 conn(0x7fbac015a0b0 legacy=0x7fbac00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:02.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.163+0000 7fbabdd9b640 1 -- 172.21.15.84:0/2287811668 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac006de10 con 0x7fbac00ced30 2023-10-11T06:14:02.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.163+0000 7fbabed9d640 1 --2- 172.21.15.84:0/2287811668 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00c8d00 0x7fbac006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:02.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.164+0000 7fbabdd9b640 1 --2- 172.21.15.84:0/2287811668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00ced30 0x7fbac00cb5a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbab40098a0 tx=0x7fbab402b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7116c5ec57def81c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:02.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.164+0000 7fbabd59a640 1 -- 172.21.15.84:0/2287811668 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4033620 con 0x7fbac00ced30 2023-10-11T06:14:02.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.164+0000 7fbabd59a640 1 -- 172.21.15.84:0/2287811668 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab4033780 con 0x7fbac00ced30 2023-10-11T06:14:02.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.164+0000 7fbabd59a640 1 -- 172.21.15.84:0/2287811668 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4033b10 con 0x7fbac00ced30 2023-10-11T06:14:02.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2287811668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00ced30 msgr2=0x7fbac00cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2287811668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00ced30 0x7fbac00cb5a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbab40098a0 tx=0x7fbab402b7d0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2287811668 shutdown_connections 2023-10-11T06:14:02.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2287811668 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00c8d00 0x7fbac006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2287811668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00ced30 0x7fbac00cb5a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2287811668 >> 172.21.15.84:0/2287811668 conn(0x7fbac0069c70 msgr2=0x7fbac006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:02.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2287811668 shutdown_connections 2023-10-11T06:14:02.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.165+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2287811668 wait complete. 2023-10-11T06:14:02.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.167+0000 7fbac4c7a640 1 Processor -- start 2023-10-11T06:14:02.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.167+0000 7fbac4c7a640 1 -- start start 2023-10-11T06:14:02.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbac4c7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00c8d00 0x7fbac00c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbac4c7a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00ced30 0x7fbac00b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbac4c7a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac015a0b0 0x7fbac0156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbac4c7a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00c7060 con 0x7fbac00c8d00 2023-10-11T06:14:02.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbac4c7a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00c7340 con 0x7fbac015a0b0 2023-10-11T06:14:02.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbac4c7a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00c7620 con 0x7fbac00ced30 2023-10-11T06:14:02.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00c8d00 0x7fbac00c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00c8d00 0x7fbac00c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59194/0 (socket says 172.21.15.84:59194) 2023-10-11T06:14:02.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 -- 172.21.15.84:0/2618313547 learned_addr learned my addr 172.21.15.84:0/2618313547 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:02.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 -- 172.21.15.84:0/2618313547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00ced30 msgr2=0x7fbac00b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00ced30 0x7fbac00b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 -- 172.21.15.84:0/2618313547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac015a0b0 msgr2=0x7fbac0156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac015a0b0 0x7fbac0156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.168+0000 7fbabe59c640 1 -- 172.21.15.84:0/2618313547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40094d0 con 0x7fbac00c8d00 2023-10-11T06:14:02.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.169+0000 7fbabe59c640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00c8d00 0x7fbac00c8300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbab000b3f0 tx=0x7fbab000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:02.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.169+0000 7fbaaeffd640 1 -- 172.21.15.84:0/2618313547 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0011830 con 0x7fbac00c8d00 2023-10-11T06:14:02.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.169+0000 7fbaaeffd640 1 -- 172.21.15.84:0/2618313547 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab0011990 con 0x7fbac00c8d00 2023-10-11T06:14:02.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.169+0000 7fbaaeffd640 1 -- 172.21.15.84:0/2618313547 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0011d40 con 0x7fbac00c8d00 2023-10-11T06:14:02.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.169+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac00c7900 con 0x7fbac00c8d00 2023-10-11T06:14:02.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.169+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac0156bb0 con 0x7fbac00c8d00 2023-10-11T06:14:02.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.170+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac00b18a0 con 0x7fbac00c8d00 2023-10-11T06:14:02.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.171+0000 7fbaaeffd640 1 -- 172.21.15.84:0/2618313547 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fbab001a400 con 0x7fbac00c8d00 2023-10-11T06:14:02.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.172+0000 7fbaaeffd640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbaa006a830 0x7fbaa006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.172+0000 7fbaaeffd640 1 -- 172.21.15.84:0/2618313547 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbab0088ec0 con 0x7fbac00c8d00 2023-10-11T06:14:02.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.173+0000 7fbabdd9b640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbaa006a830 0x7fbaa006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.173+0000 7fbabdd9b640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbaa006a830 0x7fbaa006cce0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbab4035000 tx=0x7fbab4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:02.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.176+0000 7fbaaeffd640 1 -- 172.21.15.84:0/2618313547 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fbac00b18a0 con 0x7fbac00c8d00 2023-10-11T06:14:02.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.367+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbac00b18a0 con 0x7fbac00c8d00 2023-10-11T06:14:02.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.368+0000 7fbaaeffd640 1 -- 172.21.15.84:0/2618313547 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbab00150a0 con 0x7fbac00c8d00 2023-10-11T06:14:02.369 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:14:02.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbaa006a830 msgr2=0x7fbaa006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbaa006a830 0x7fbaa006cce0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbab4035000 tx=0x7fbab4030000 comp rx=0 tx=0).stop 2023-10-11T06:14:02.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00c8d00 msgr2=0x7fbac00c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00c8d00 0x7fbac00c8300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbab000b3f0 tx=0x7fbab000b8b0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 shutdown_connections 2023-10-11T06:14:02.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fbaa006a830 0x7fbaa006cce0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbac015a0b0 0x7fbac0156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbac00ced30 0x7fbac00b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 --2- 172.21.15.84:0/2618313547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbac00c8d00 0x7fbac00c8300 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 >> 172.21.15.84:0/2618313547 conn(0x7fbac0069c70 msgr2=0x7fbac014ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:02.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 shutdown_connections 2023-10-11T06:14:02.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.371+0000 7fbac4c7a640 1 -- 172.21.15.84:0/2618313547 wait complete. 2023-10-11T06:14:02.391 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:14:02.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.602+0000 7f012d79f640 1 Processor -- start 2023-10-11T06:14:02.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.602+0000 7f012d79f640 1 -- start start 2023-10-11T06:14:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.603+0000 7f012d79f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280adb90 0x7f01280caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.603+0000 7f012d79f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f01280cd300 0x7f01280cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.603+0000 7f012d79f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01280cb960 con 0x7f01280ce0d0 2023-10-11T06:14:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.603+0000 7f012d79f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012815a340 con 0x7f01280cd300 2023-10-11T06:14:02.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.603+0000 7f012d79f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01281593f0 con 0x7f01280adb90 2023-10-11T06:14:02.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f01277fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f01280cd300 0x7f01280cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f0126ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f01280ce0d0 0x7f01280c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34346/0 (socket says 172.21.15.84:34346) 2023-10-11T06:14:02.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f0126ffd640 1 -- 172.21.15.84:0/2272620507 learned_addr learned my addr 172.21.15.84:0/2272620507 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:02.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f011ffff640 1 --2- 172.21.15.84:0/2272620507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280adb90 0x7f01280caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f01277fe640 1 -- 172.21.15.84:0/2272620507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280adb90 msgr2=0x7f01280caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:02.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f01277fe640 1 --2- 172.21.15.84:0/2272620507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280adb90 0x7f01280caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f01277fe640 1 -- 172.21.15.84:0/2272620507 >> v1:172.21.15.84:6789/0 conn(0x7f01280ce0d0 legacy=0x7f01280c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.604+0000 7f01277fe640 1 -- 172.21.15.84:0/2272620507 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01281596c0 con 0x7f01280cd300 2023-10-11T06:14:02.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.605+0000 7f01277fe640 1 --2- 172.21.15.84:0/2272620507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280cd300 0x7f01280cb420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0120009d90 tx=0x7f012002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cdb2a24de1e74a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:02.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.605+0000 7f01267fc640 1 -- 172.21.15.84:0/2272620507 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012002cc60 con 0x7f01280cd300 2023-10-11T06:14:02.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f01267fc640 1 -- 172.21.15.84:0/2272620507 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012002cdc0 con 0x7f01280cd300 2023-10-11T06:14:02.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f01267fc640 1 -- 172.21.15.84:0/2272620507 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120034660 con 0x7f01280cd300 2023-10-11T06:14:02.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 -- 172.21.15.84:0/2272620507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280cd300 msgr2=0x7f01280cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 --2- 172.21.15.84:0/2272620507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280cd300 0x7f01280cb420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0120009d90 tx=0x7f012002beb0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 -- 172.21.15.84:0/2272620507 shutdown_connections 2023-10-11T06:14:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 --2- 172.21.15.84:0/2272620507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280cd300 0x7f01280cb420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 --2- 172.21.15.84:0/2272620507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280adb90 0x7f01280caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 -- 172.21.15.84:0/2272620507 >> 172.21.15.84:0/2272620507 conn(0x7f0128069c70 msgr2=0x7f012806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:02.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 -- 172.21.15.84:0/2272620507 shutdown_connections 2023-10-11T06:14:02.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.606+0000 7f012d79f640 1 -- 172.21.15.84:0/2272620507 wait complete. 2023-10-11T06:14:02.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.608+0000 7f012d79f640 1 Processor -- start 2023-10-11T06:14:02.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.608+0000 7f012d79f640 1 -- start start 2023-10-11T06:14:02.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f012d79f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280adb90 0x7f01280ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f012d79f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01280cd300 0x7f01280c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f012d79f640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280ce0d0 0x7f0128146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f012d79f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0128148a30 con 0x7f01280adb90 2023-10-11T06:14:02.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f012d79f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01280c86c0 con 0x7f01280cd300 2023-10-11T06:14:02.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f012d79f640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01281481d0 con 0x7f01280ce0d0 2023-10-11T06:14:02.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f0126ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280adb90 0x7f01280ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f0126ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280adb90 0x7f01280ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59218/0 (socket says 172.21.15.84:59218) 2023-10-11T06:14:02.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.609+0000 7f0126ffd640 1 -- 172.21.15.84:0/4096462768 learned_addr learned my addr 172.21.15.84:0/4096462768 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f0126ffd640 1 -- 172.21.15.84:0/4096462768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280ce0d0 msgr2=0x7f0128146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f0126ffd640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280ce0d0 0x7f0128146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f0126ffd640 1 -- 172.21.15.84:0/4096462768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01280cd300 msgr2=0x7f01280c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f0126ffd640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01280cd300 0x7f01280c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f0126ffd640 1 -- 172.21.15.84:0/4096462768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120009a70 con 0x7f01280adb90 2023-10-11T06:14:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f0126ffd640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280adb90 0x7f01280ac490 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f011800c480 tx=0x7f011800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:02.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f011f7fe640 1 -- 172.21.15.84:0/4096462768 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118007480 con 0x7f01280adb90 2023-10-11T06:14:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f011f7fe640 1 -- 172.21.15.84:0/4096462768 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180075e0 con 0x7f01280adb90 2023-10-11T06:14:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f011f7fe640 1 -- 172.21.15.84:0/4096462768 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180059a0 con 0x7f01280adb90 2023-10-11T06:14:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.610+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0128148470 con 0x7f01280adb90 2023-10-11T06:14:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.611+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0128156c00 con 0x7f01280adb90 2023-10-11T06:14:02.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.612+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0128156a60 con 0x7f01280adb90 2023-10-11T06:14:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.613+0000 7f011f7fe640 1 -- 172.21.15.84:0/4096462768 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0118005b00 con 0x7f01280adb90 2023-10-11T06:14:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.614+0000 7f011f7fe640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f010806a830 0x7f010806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:02.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.614+0000 7f011f7fe640 1 -- 172.21.15.84:0/4096462768 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f01180885a0 con 0x7f01280adb90 2023-10-11T06:14:02.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.614+0000 7f011ffff640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f010806a830 0x7f010806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:02.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.615+0000 7f011ffff640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f010806a830 0x7f010806cce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0110006fd0 tx=0x7f0110008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:02.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.618+0000 7f011f7fe640 1 -- 172.21.15.84:0/4096462768 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0128156a60 con 0x7f01280adb90 2023-10-11T06:14:02.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.822+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0128156a60 con 0x7f01280adb90 2023-10-11T06:14:02.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.824+0000 7f011f7fe640 1 -- 172.21.15.84:0/4096462768 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0128156a60 con 0x7f01280adb90 2023-10-11T06:14:02.825 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:14:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f010806a830 msgr2=0x7f010806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f010806a830 0x7f010806cce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0110006fd0 tx=0x7f0110008040 comp rx=0 tx=0).stop 2023-10-11T06:14:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280adb90 msgr2=0x7f01280ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:02.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280adb90 0x7f01280ac490 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f011800c480 tx=0x7f011800c940 comp rx=0 tx=0).stop 2023-10-11T06:14:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 shutdown_connections 2023-10-11T06:14:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f010806a830 0x7f010806cce0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01280ce0d0 0x7f0128146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01280cd300 0x7f01280c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 --2- 172.21.15.84:0/4096462768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01280adb90 0x7f01280ac490 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:02.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.826+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 >> 172.21.15.84:0/4096462768 conn(0x7f0128069c70 msgr2=0x7f012806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:02.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.827+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 shutdown_connections 2023-10-11T06:14:02.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:02.827+0000 7f012d79f640 1 -- 172.21.15.84:0/4096462768 wait complete. 2023-10-11T06:14:02.845 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:14:02.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:02.846 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:02.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[107719]: fuse finished with error 0 and tester_r 0 2023-10-11T06:14:02.934 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:14:09.035 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:14:09.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:09.036 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:14:09.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:14:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.313+0000 7fc59787d640 1 Processor -- start 2023-10-11T06:14:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.314+0000 7fc59787d640 1 -- start start 2023-10-11T06:14:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc59787d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc59015c290 0x7fc5900af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc59787d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5900d2a60 con 0x7fc5900d0c20 2023-10-11T06:14:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc59787d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59006cd10 con 0x7fc5900d25e0 2023-10-11T06:14:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc59787d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5900afb60 con 0x7fc59015c290 2023-10-11T06:14:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc595619640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc59015c290 0x7fc5900af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc595619640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc59015c290 0x7fc5900af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53170/0 (socket says 172.21.15.84:53170) 2023-10-11T06:14:09.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc595619640 1 -- 172.21.15.84:0/2262869098 learned_addr learned my addr 172.21.15.84:0/2262869098 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:09.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.315+0000 7fc595e1a640 1 --1- 172.21.15.84:0/2262869098 >> v1:172.21.15.84:6789/0 conn(0x7fc5900d0c20 0x7fc5900b0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35888/0 (socket says 172.21.15.84:35888) 2023-10-11T06:14:09.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.316+0000 7fc595619640 1 -- 172.21.15.84:0/2262869098 >> v1:172.21.15.99:6789/0 conn(0x7fc5900d25e0 legacy=0x7fc5900d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.316+0000 7fc595619640 1 -- 172.21.15.84:0/2262869098 >> v1:172.21.15.84:6789/0 conn(0x7fc5900d0c20 legacy=0x7fc5900b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.316+0000 7fc595619640 1 -- 172.21.15.84:0/2262869098 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5900afe40 con 0x7fc59015c290 2023-10-11T06:14:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.317+0000 7fc595619640 1 --2- 172.21.15.84:0/2262869098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5900af4d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc580009870 tx=0x7fc58002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e08395ca3bbeed0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.317+0000 7fc58ffff640 1 -- 172.21.15.84:0/2262869098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580031870 con 0x7fc59015c290 2023-10-11T06:14:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.317+0000 7fc58ffff640 1 -- 172.21.15.84:0/2262869098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5800319d0 con 0x7fc59015c290 2023-10-11T06:14:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.317+0000 7fc58ffff640 1 -- 172.21.15.84:0/2262869098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580031d60 con 0x7fc59015c290 2023-10-11T06:14:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.318+0000 7fc59787d640 1 -- 172.21.15.84:0/2262869098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 msgr2=0x7fc5900af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.318+0000 7fc59787d640 1 --2- 172.21.15.84:0/2262869098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5900af4d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc580009870 tx=0x7fc58002b910 comp rx=0 tx=0).stop 2023-10-11T06:14:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.318+0000 7fc59787d640 1 -- 172.21.15.84:0/2262869098 shutdown_connections 2023-10-11T06:14:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.318+0000 7fc59787d640 1 --2- 172.21.15.84:0/2262869098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5900af4d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.318+0000 7fc59787d640 1 -- 172.21.15.84:0/2262869098 >> 172.21.15.84:0/2262869098 conn(0x7fc590071cc0 msgr2=0x7fc59009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.318+0000 7fc59787d640 1 -- 172.21.15.84:0/2262869098 shutdown_connections 2023-10-11T06:14:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.318+0000 7fc59787d640 1 -- 172.21.15.84:0/2262869098 wait complete. 2023-10-11T06:14:09.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.320+0000 7fc59787d640 1 Processor -- start 2023-10-11T06:14:09.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.320+0000 7fc59787d640 1 -- start start 2023-10-11T06:14:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc59787d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5900d0c20 0x7fc5900b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc59787d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d25e0 0x7fc5901f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc59787d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5901f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc59787d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900b5b20 con 0x7fc59015c290 2023-10-11T06:14:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc59787d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900b7940 con 0x7fc5900d25e0 2023-10-11T06:14:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc59787d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900b7c20 con 0x7fc5900d0c20 2023-10-11T06:14:09.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc595e1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5901f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:09.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc595e1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5901f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53180/0 (socket says 172.21.15.84:53180) 2023-10-11T06:14:09.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc595e1a640 1 -- 172.21.15.84:0/3318709182 learned_addr learned my addr 172.21.15.84:0/3318709182 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:09.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc594e18640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d25e0 0x7fc5901f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:09.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.321+0000 7fc595619640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5900d0c20 0x7fc5900b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:09.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc595e1a640 1 -- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5900d0c20 msgr2=0x7fc5900b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:09.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc595e1a640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5900d0c20 0x7fc5900b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:09.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc595e1a640 1 -- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d25e0 msgr2=0x7fc5901f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:09.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc595e1a640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d25e0 0x7fc5901f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:09.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc595e1a640 1 -- 172.21.15.84:0/3318709182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5800094d0 con 0x7fc59015c290 2023-10-11T06:14:09.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc595619640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5900d0c20 0x7fc5900b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:09.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc595e1a640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5901f4d00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc58800c7a0 tx=0x7fc58800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:09.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc58dffb640 1 -- 172.21.15.84:0/3318709182 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58800a640 con 0x7fc59015c290 2023-10-11T06:14:09.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc58dffb640 1 -- 172.21.15.84:0/3318709182 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58800a7a0 con 0x7fc59015c290 2023-10-11T06:14:09.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc58dffb640 1 -- 172.21.15.84:0/3318709182 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58800ab10 con 0x7fc59015c290 2023-10-11T06:14:09.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5900b7f00 con 0x7fc59015c290 2023-10-11T06:14:09.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.322+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5900b84c0 con 0x7fc59015c290 2023-10-11T06:14:09.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.323+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5900b86e0 con 0x7fc59015c290 2023-10-11T06:14:09.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.324+0000 7fc58dffb640 1 -- 172.21.15.84:0/3318709182 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc58800ac70 con 0x7fc59015c290 2023-10-11T06:14:09.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.325+0000 7fc58dffb640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc57c06a830 0x7fc57c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:09.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.325+0000 7fc58dffb640 1 -- 172.21.15.84:0/3318709182 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5880886b0 con 0x7fc59015c290 2023-10-11T06:14:09.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.325+0000 7fc595619640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc57c06a830 0x7fc57c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:09.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.326+0000 7fc595619640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc57c06a830 0x7fc57c06cce0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc580036000 tx=0x7fc58003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:09.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.330+0000 7fc58dffb640 1 -- 172.21.15.84:0/3318709182 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc5880539d0 con 0x7fc59015c290 2023-10-11T06:14:09.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.563+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc5900b00f0 con 0x7fc59015c290 2023-10-11T06:14:09.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.563+0000 7fc58dffb640 1 -- 172.21.15.84:0/3318709182 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc588056ff0 con 0x7fc59015c290 2023-10-11T06:14:09.565 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:14:09.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc57c06a830 msgr2=0x7fc57c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:09.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc57c06a830 0x7fc57c06cce0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc580036000 tx=0x7fc58003c040 comp rx=0 tx=0).stop 2023-10-11T06:14:09.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 msgr2=0x7fc5901f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:09.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5901f4d00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc58800c7a0 tx=0x7fc58800cc60 comp rx=0 tx=0).stop 2023-10-11T06:14:09.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 shutdown_connections 2023-10-11T06:14:09.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc57c06a830 0x7fc57c06cce0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:09.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc59015c290 0x7fc5901f4d00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:09.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d25e0 0x7fc5901f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:09.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 --2- 172.21.15.84:0/3318709182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5900d0c20 0x7fc5900b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:09.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.566+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 >> 172.21.15.84:0/3318709182 conn(0x7fc590071cc0 msgr2=0x7fc5900b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:09.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.567+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 shutdown_connections 2023-10-11T06:14:09.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:09.567+0000 7fc59787d640 1 -- 172.21.15.84:0/3318709182 wait complete. 2023-10-11T06:14:09.587 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:14:09.587 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:14:09.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:14:09.588 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T06:14:09.588 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:09.588 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:14:09.589 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:14:09.589 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:14:09.589 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:14:09.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:09.590 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:14:09.642 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:14:09.642 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:14:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:14:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:14:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:09.643 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:14:09.643 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:14:09.644 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:14:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:09.644 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:14:09.644 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:14:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:09.645 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:14:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:14:09.645 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42227sec preferred_lft 42227sec 2023-10-11T06:14:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:14:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:09.646 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:14:09.646 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:14:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1c33:d5ff:fea1:21f9/64 scope link 2023-10-11T06:14:09.647 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:09.647 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:14:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:14:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::d4ff:bcff:feac:c798/64 scope link 2023-10-11T06:14:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:09.648 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:14:09.648 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:14:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4088:59ff:feb7:538e/64 scope link 2023-10-11T06:14:09.649 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:09.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:09.650 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:14:09.739 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:14:09.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:09.744 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:14:09.800 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:14:09.800 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:14:09.801 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:14:09.801 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:14:09.801 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:14:09.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:09.802 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:14:09.802 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:14:09.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:14:09.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:14:09.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:14:09.804 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:14:09.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:10.000 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:14:10.051 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T06:14:10.052 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:14:10.052 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:14:10.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:10.053 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-11T06:14:10.144 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T06:14:10.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:14:10.145 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:10.204 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:14:10.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:14:10.205 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:10.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:14:10.354 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:14:10.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:14:10.451 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:14:10.454 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:14:10.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:10.455 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:14:10.505 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:14:10.505 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T06:14:10.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:14:10.589 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:14:10.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:14:10.660 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:14:10.663 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:14:10.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:10.664 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:14:10.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-10-11T06:14:10.699+0000 7f5359ac9480 -1 init, newargv = 0x5576712a46f0 newargc=15 2023-10-11T06:14:10.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[108837]: starting ceph client 2023-10-11T06:14:10.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[108837]: starting fuse 2023-10-11T06:14:10.710 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:14:10.711 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-10-11T06:14:10.711 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:14:10.712 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:14:10.712 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:14:10.713 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:14:10.713 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:14:10.714 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:10.714 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:14:10.715 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T06:14:10.715 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:14:10.716 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T06:14:10.716 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:10.717 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:14:10.718 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:14:10.718 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:14:10.719 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:14:10.720 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:10.720 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:14:10.721 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:14:10.722 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:14:10.722 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:14:10.722 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:14:10.723 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:14:10.723 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:14:10.723 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:14:10.724 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:10.724 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-10-11T06:14:10.724 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:14:10.903 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.108837.asok 2023-10-11T06:14:11.085 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.108837.asok 2023-10-11T06:14:11.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:11.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.108837.asok status 2023-10-11T06:14:11.327 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:14:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:14:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:14:11.328 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-10-11T06:14:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:14:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T06:14:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "108837", 2023-10-11T06:14:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:14:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:11.329 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:14:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:14:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6297, 2023-10-11T06:14:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:14:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:14:11.330 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:14:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 6297 2023-10-11T06:14:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:14:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:14:11.331 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:14:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 134062707 2023-10-11T06:14:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:14:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:14:11.332 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:14:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:14:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 134062707 2023-10-11T06:14:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.6297 192.168.0.1:0/134062707", 2023-10-11T06:14:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/134062707", 2023-10-11T06:14:11.333 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:14:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 24, 2023-10-11T06:14:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 54, 2023-10-11T06:14:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:14:11.334 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:14:11.335 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:14:11.335 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:11.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:11.342 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:14:11.343 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:11.399 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:14:11.399 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:11.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:11.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:11.499 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:14:11.499 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:14:11.500 DEBUG:teuthology.orchestra.run.smithi084:> import errno 2023-10-11T06:14:11.500 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:11.500 DEBUG:teuthology.orchestra.run.smithi084:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-10-11T06:14:11.501 DEBUG:teuthology.orchestra.run.smithi084:> try: 2023-10-11T06:14:11.501 DEBUG:teuthology.orchestra.run.smithi084:> if False: 2023-10-11T06:14:11.501 DEBUG:teuthology.orchestra.run.smithi084:> ret = os.read(fd, 1024) 2023-10-11T06:14:11.501 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-10-11T06:14:11.502 DEBUG:teuthology.orchestra.run.smithi084:> os.write(fd, b'"'"'content'"'"') 2023-10-11T06:14:11.502 DEBUG:teuthology.orchestra.run.smithi084:> except OSError as e: 2023-10-11T06:14:11.502 DEBUG:teuthology.orchestra.run.smithi084:> if e.errno != errno.EPERM: 2023-10-11T06:14:11.503 DEBUG:teuthology.orchestra.run.smithi084:> raise 2023-10-11T06:14:11.503 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-10-11T06:14:11.503 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("client does not check permission of data pool") 2023-10-11T06:14:11.504 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:14:11.816 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2023-10-11T06:14:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.088+0000 7f8e65015640 1 Processor -- start 2023-10-11T06:14:12.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.089+0000 7f8e65015640 1 -- start start 2023-10-11T06:14:12.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e65015640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e6015c3d0 0x7f8e600cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e65015640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8e600d05a0 0x7f8e600d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e65015640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6015c7a0 con 0x7f8e600d2620 2023-10-11T06:14:12.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e65015640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e600cef40 con 0x7f8e600d05a0 2023-10-11T06:14:12.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e65015640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e600cf1e0 con 0x7f8e6015c3d0 2023-10-11T06:14:12.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e5f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8e600d05a0 0x7f8e600d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e5f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8e600d05a0 0x7f8e600d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53184/0 (socket says 172.21.15.84:53184) 2023-10-11T06:14:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2274481967 learned_addr learned my addr 172.21.15.84:0/2274481967 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:12.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.090+0000 7f8e64814640 1 --1- 172.21.15.84:0/2274481967 >> v1:172.21.15.84:6789/0 conn(0x7f8e600d2620 0x7f8e600d2aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35892/0 (socket says 172.21.15.84:35892) 2023-10-11T06:14:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.091+0000 7f8e5ffff640 1 --2- 172.21.15.84:0/2274481967 >> v2:172.21.15.99:3300/0 conn(0x7f8e6015c3d0 0x7f8e600cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.091+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2274481967 >> v2:172.21.15.99:3300/0 conn(0x7f8e6015c3d0 msgr2=0x7f8e600cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:12.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.091+0000 7f8e5f7fe640 1 --2- 172.21.15.84:0/2274481967 >> v2:172.21.15.99:3300/0 conn(0x7f8e6015c3d0 0x7f8e600cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.091+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2274481967 >> v1:172.21.15.84:6789/0 conn(0x7f8e600d2620 legacy=0x7f8e600d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.091+0000 7f8e5effd640 1 -- 172.21.15.84:0/2274481967 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903363335 0 0) 0x7f8e6015c7a0 con 0x7f8e600d2620 2023-10-11T06:14:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.091+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2274481967 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600cf4c0 con 0x7f8e600d05a0 2023-10-11T06:14:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.091+0000 7f8e5ffff640 1 --2- 172.21.15.84:0/2274481967 >> v2:172.21.15.99:3300/0 conn(0x7f8e6015c3d0 0x7f8e600cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:12.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.092+0000 7f8e5f7fe640 1 --2- 172.21.15.84:0/2274481967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d05a0 0x7f8e600d0a10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8e6008b5c0 tx=0x7f8e4802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c55e930a729d9e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.092+0000 7f8e5effd640 1 -- 172.21.15.84:0/2274481967 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48004230 con 0x7f8e600d05a0 2023-10-11T06:14:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.092+0000 7f8e5effd640 1 -- 172.21.15.84:0/2274481967 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48004390 con 0x7f8e600d05a0 2023-10-11T06:14:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.092+0000 7f8e5effd640 1 -- 172.21.15.84:0/2274481967 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48030840 con 0x7f8e600d05a0 2023-10-11T06:14:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 -- 172.21.15.84:0/2274481967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d05a0 msgr2=0x7f8e600d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 --2- 172.21.15.84:0/2274481967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d05a0 0x7f8e600d0a10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8e6008b5c0 tx=0x7f8e4802b980 comp rx=0 tx=0).stop 2023-10-11T06:14:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 -- 172.21.15.84:0/2274481967 shutdown_connections 2023-10-11T06:14:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 --2- 172.21.15.84:0/2274481967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d05a0 0x7f8e600d0a10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 --2- 172.21.15.84:0/2274481967 >> v2:172.21.15.99:3300/0 conn(0x7f8e6015c3d0 0x7f8e600cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 -- 172.21.15.84:0/2274481967 >> 172.21.15.84:0/2274481967 conn(0x7f8e60089ff0 msgr2=0x7f8e6009bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:12.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 -- 172.21.15.84:0/2274481967 shutdown_connections 2023-10-11T06:14:12.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.093+0000 7f8e65015640 1 -- 172.21.15.84:0/2274481967 wait complete. 2023-10-11T06:14:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.095+0000 7f8e65015640 1 Processor -- start 2023-10-11T06:14:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.095+0000 7f8e65015640 1 -- start start 2023-10-11T06:14:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.096+0000 7f8e65015640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e600d05a0 0x7f8e6015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.096+0000 7f8e65015640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d2620 0x7f8e6009dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.096+0000 7f8e65015640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e6015c3d0 0x7f8e6009e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.096+0000 7f8e65015640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6015cef0 con 0x7f8e600d2620 2023-10-11T06:14:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.096+0000 7f8e65015640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6015d1d0 con 0x7f8e600d05a0 2023-10-11T06:14:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.096+0000 7f8e65015640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6015d4b0 con 0x7f8e6015c3d0 2023-10-11T06:14:12.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d2620 0x7f8e6009dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d2620 0x7f8e6009dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53190/0 (socket says 172.21.15.84:53190) 2023-10-11T06:14:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2604018651 learned_addr learned my addr 172.21.15.84:0/2604018651 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5ffff640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e600d05a0 0x7f8e6015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e64814640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e6015c3d0 0x7f8e6009e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e6015c3d0 msgr2=0x7f8e6009e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e6015c3d0 0x7f8e6009e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e600d05a0 msgr2=0x7f8e6015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e600d05a0 0x7f8e6015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5f7fe640 1 -- 172.21.15.84:0/2604018651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e48009460 con 0x7f8e600d2620 2023-10-11T06:14:12.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.097+0000 7f8e5ffff640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e600d05a0 0x7f8e6015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.098+0000 7f8e5f7fe640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d2620 0x7f8e6009dda0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8e4802b9c0 tx=0x7f8e480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.098+0000 7f8e5cff9640 1 -- 172.21.15.84:0/2604018651 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4803e070 con 0x7f8e600d2620 2023-10-11T06:14:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.098+0000 7f8e5cff9640 1 -- 172.21.15.84:0/2604018651 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48004500 con 0x7f8e600d2620 2023-10-11T06:14:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.098+0000 7f8e5cff9640 1 -- 172.21.15.84:0/2604018651 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48030e20 con 0x7f8e600d2620 2023-10-11T06:14:12.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.098+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6015d790 con 0x7f8e600d2620 2023-10-11T06:14:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.098+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6015aef0 con 0x7f8e600d2620 2023-10-11T06:14:12.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.099+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6015dcb0 con 0x7f8e600d2620 2023-10-11T06:14:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.100+0000 7f8e5cff9640 1 -- 172.21.15.84:0/2604018651 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8e48003610 con 0x7f8e600d2620 2023-10-11T06:14:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.101+0000 7f8e5cff9640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e4006a830 0x7f8e4006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.101+0000 7f8e5cff9640 1 -- 172.21.15.84:0/2604018651 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e480ab110 con 0x7f8e600d2620 2023-10-11T06:14:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.101+0000 7f8e5ffff640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e4006a830 0x7f8e4006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.102+0000 7f8e5ffff640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e4006a830 0x7f8e4006cce0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8e50007f90 tx=0x7f8e500074e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:12.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.106+0000 7f8e5cff9640 1 -- 172.21.15.84:0/2604018651 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f8e48076390 con 0x7f8e600d2620 2023-10-11T06:14:12.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.364+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7f8e6004e890 con 0x7f8e600d2620 2023-10-11T06:14:12.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.370+0000 7f8e5cff9640 1 -- 172.21.15.84:0/2604018651 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 183+0+0 (secure 0 0 0) 0x7f8e480799b0 con 0x7f8e600d2620 2023-10-11T06:14:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:14:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e4006a830 msgr2=0x7f8e4006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e4006a830 0x7f8e4006cce0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8e50007f90 tx=0x7f8e500074e0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d2620 msgr2=0x7f8e6009dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d2620 0x7f8e6009dda0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8e4802b9c0 tx=0x7f8e480046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 shutdown_connections 2023-10-11T06:14:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f8e4006a830 0x7f8e4006cce0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e6015c3d0 0x7f8e6009e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e600d2620 0x7f8e6009dda0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 --2- 172.21.15.84:0/2604018651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e600d05a0 0x7f8e6015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 >> 172.21.15.84:0/2604018651 conn(0x7f8e60089ff0 msgr2=0x7f8e6015b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:12.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 shutdown_connections 2023-10-11T06:14:12.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.373+0000 7f8e65015640 1 -- 172.21.15.84:0/2604018651 wait complete. 2023-10-11T06:14:12.394 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:14:12.394 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:12.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:12.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.655+0000 7f4816034640 1 Processor -- start 2023-10-11T06:14:12.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.656+0000 7f4816034640 1 -- start start 2023-10-11T06:14:12.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.657+0000 7f4816034640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100adb90 0x7f48100caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.657+0000 7f4816034640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f48100cd300 0x7f48100cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.657+0000 7f4816034640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100cb960 con 0x7f48100ce0d0 2023-10-11T06:14:12.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.657+0000 7f4816034640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481015a340 con 0x7f48100cd300 2023-10-11T06:14:12.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.657+0000 7f4816034640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100ce4a0 con 0x7f48100adb90 2023-10-11T06:14:12.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f48100ce0d0 0x7f48100c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35906/0 (socket says 172.21.15.84:35906) 2023-10-11T06:14:12.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480f7fe640 1 -- 172.21.15.84:0/160866905 learned_addr learned my addr 172.21.15.84:0/160866905 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:12.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480ffff640 1 --2- 172.21.15.84:0/160866905 >> v2:172.21.15.84:3300/0 conn(0x7f48100cd300 0x7f48100cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f4806ffd640 1 --2- 172.21.15.84:0/160866905 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100adb90 0x7f48100caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480ffff640 1 -- 172.21.15.84:0/160866905 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100adb90 msgr2=0x7f48100caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:12.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480ffff640 1 --2- 172.21.15.84:0/160866905 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100adb90 0x7f48100caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480ffff640 1 -- 172.21.15.84:0/160866905 >> v1:172.21.15.84:6789/0 conn(0x7f48100ce0d0 legacy=0x7f48100c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480ffff640 1 -- 172.21.15.84:0/160866905 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48101593f0 con 0x7f48100cd300 2023-10-11T06:14:12.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.658+0000 7f480effd640 1 -- 172.21.15.84:0/160866905 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380714113 0 0) 0x7f48100cb960 con 0x7f48100ce0d0 2023-10-11T06:14:12.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.659+0000 7f480ffff640 1 --2- 172.21.15.84:0/160866905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100cb420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4808009dc0 tx=0x7f480802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6705d9dc462e3f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:12.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f480effd640 1 -- 172.21.15.84:0/160866905 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480802cc40 con 0x7f48100cd300 2023-10-11T06:14:12.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f480effd640 1 -- 172.21.15.84:0/160866905 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480802cda0 con 0x7f48100cd300 2023-10-11T06:14:12.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f480effd640 1 -- 172.21.15.84:0/160866905 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808034660 con 0x7f48100cd300 2023-10-11T06:14:12.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 -- 172.21.15.84:0/160866905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 msgr2=0x7f48100cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 --2- 172.21.15.84:0/160866905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100cb420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4808009dc0 tx=0x7f480802be90 comp rx=0 tx=0).stop 2023-10-11T06:14:12.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 -- 172.21.15.84:0/160866905 shutdown_connections 2023-10-11T06:14:12.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 --2- 172.21.15.84:0/160866905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100cb420 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 --2- 172.21.15.84:0/160866905 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100adb90 0x7f48100caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 -- 172.21.15.84:0/160866905 >> 172.21.15.84:0/160866905 conn(0x7f4810069c70 msgr2=0x7f481006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:12.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 -- 172.21.15.84:0/160866905 shutdown_connections 2023-10-11T06:14:12.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.660+0000 7f4816034640 1 -- 172.21.15.84:0/160866905 wait complete. 2023-10-11T06:14:12.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.662+0000 7f4816034640 1 Processor -- start 2023-10-11T06:14:12.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.662+0000 7f4816034640 1 -- start start 2023-10-11T06:14:12.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.663+0000 7f4816034640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48100adb90 0x7f48100b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.663+0000 7f4816034640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.663+0000 7f4816034640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100ce0d0 0x7f48100ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.663+0000 7f4816034640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100b0490 con 0x7f48100cd300 2023-10-11T06:14:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.663+0000 7f4816034640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100b0e60 con 0x7f48100adb90 2023-10-11T06:14:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.663+0000 7f4816034640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48101f2430 con 0x7f48100ce0d0 2023-10-11T06:14:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53196/0 (socket says 172.21.15.84:53196) 2023-10-11T06:14:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 -- 172.21.15.84:0/3828093403 learned_addr learned my addr 172.21.15.84:0/3828093403 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:12.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f480ffff640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100ce0d0 0x7f48100ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f480f7fe640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48100adb90 0x7f48100b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 -- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100ce0d0 msgr2=0x7f48100ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100ce0d0 0x7f48100ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 -- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48100adb90 msgr2=0x7f48100b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48100adb90 0x7f48100b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f4806ffd640 1 -- 172.21.15.84:0/3828093403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808009a70 con 0x7f48100cd300 2023-10-11T06:14:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.664+0000 7f480f7fe640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48100adb90 0x7f48100b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.665+0000 7f4806ffd640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100c44f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f47f800d5f0 tx=0x7f47f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:12.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.665+0000 7f480cff9640 1 -- 172.21.15.84:0/3828093403 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80125f0 con 0x7f48100cd300 2023-10-11T06:14:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.665+0000 7f480cff9640 1 -- 172.21.15.84:0/3828093403 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f8012750 con 0x7f48100cd300 2023-10-11T06:14:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.665+0000 7f480cff9640 1 -- 172.21.15.84:0/3828093403 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8012b00 con 0x7f48100cd300 2023-10-11T06:14:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.665+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48101f2710 con 0x7f48100cd300 2023-10-11T06:14:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.665+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48100ae310 con 0x7f48100cd300 2023-10-11T06:14:12.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.666+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48100b0600 con 0x7f48100cd300 2023-10-11T06:14:12.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.667+0000 7f480cff9640 1 -- 172.21.15.84:0/3828093403 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f47f8012c60 con 0x7f48100cd300 2023-10-11T06:14:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.668+0000 7f480cff9640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f47f006a830 0x7f47f006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.668+0000 7f480cff9640 1 -- 172.21.15.84:0/3828093403 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47f8089500 con 0x7f48100cd300 2023-10-11T06:14:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.669+0000 7f480f7fe640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f47f006a830 0x7f47f006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.669+0000 7f480f7fe640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f47f006a830 0x7f47f006cce0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4800007fd0 tx=0x7f4800009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:12.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.673+0000 7f480cff9640 1 -- 172.21.15.84:0/3828093403 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f47f80546e0 con 0x7f48100cd300 2023-10-11T06:14:12.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.862+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4810159c50 con 0x7f48100cd300 2023-10-11T06:14:12.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.865+0000 7f480cff9640 1 -- 172.21.15.84:0/3828093403 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7f47f8057d00 con 0x7f48100cd300 2023-10-11T06:14:12.867 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:12.867 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:12.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.867+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f47f006a830 msgr2=0x7f47f006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.867+0000 7f4816034640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f47f006a830 0x7f47f006cce0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4800007fd0 tx=0x7f4800009040 comp rx=0 tx=0).stop 2023-10-11T06:14:12.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.867+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 msgr2=0x7f48100c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:12.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.867+0000 7f4816034640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100c44f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f47f800d5f0 tx=0x7f47f800dab0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 shutdown_connections 2023-10-11T06:14:12.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f47f006a830 0x7f47f006cce0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48100ce0d0 0x7f48100ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48100cd300 0x7f48100c44f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 --2- 172.21.15.84:0/3828093403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48100adb90 0x7f48100b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:12.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 >> 172.21.15.84:0/3828093403 conn(0x7f4810069c70 msgr2=0x7f481006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:12.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 shutdown_connections 2023-10-11T06:14:12.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:12.868+0000 7f4816034640 1 -- 172.21.15.84:0/3828093403 wait complete. 2023-10-11T06:14:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:13:44.841435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:13:45.401366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:14:12.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:14:13.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.103+0000 7f95e91dd640 1 Processor -- start 2023-10-11T06:14:13.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.103+0000 7f95e91dd640 1 -- start start 2023-10-11T06:14:13.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e91dd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f95e406ca10 0x7f95e4152d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e91dd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f95e40ce6b0 0x7f95e40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e91dd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e406dd40 con 0x7f95e406c640 2023-10-11T06:14:13.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e91dd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e415c480 con 0x7f95e40ce6b0 2023-10-11T06:14:13.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e91dd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e406e020 con 0x7f95e406ca10 2023-10-11T06:14:13.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e2d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f95e406c640 0x7f95e4159830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35910/0 (socket says 172.21.15.84:35910) 2023-10-11T06:14:13.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e2d9d640 1 -- 172.21.15.84:0/3369954813 learned_addr learned my addr 172.21.15.84:0/3369954813 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:13.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.105+0000 7f95e359e640 1 --2- 172.21.15.84:0/3369954813 >> v2:172.21.15.84:3300/0 conn(0x7f95e40ce6b0 0x7f95e40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95dbfff640 1 --2- 172.21.15.84:0/3369954813 >> v2:172.21.15.99:3300/0 conn(0x7f95e406ca10 0x7f95e4152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95e259c640 1 -- 172.21.15.84:0/3369954813 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892995771 0 0) 0x7f95e406dd40 con 0x7f95e406c640 2023-10-11T06:14:13.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95e259c640 1 -- 172.21.15.84:0/3369954813 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d0003520 con 0x7f95e406c640 2023-10-11T06:14:13.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95e359e640 1 -- 172.21.15.84:0/3369954813 >> v2:172.21.15.99:3300/0 conn(0x7f95e406ca10 msgr2=0x7f95e4152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:13.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95e359e640 1 --2- 172.21.15.84:0/3369954813 >> v2:172.21.15.99:3300/0 conn(0x7f95e406ca10 0x7f95e4152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95e259c640 1 -- 172.21.15.84:0/3369954813 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3940567791 0 0) 0x7f95d0003520 con 0x7f95e406c640 2023-10-11T06:14:13.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95e359e640 1 -- 172.21.15.84:0/3369954813 >> v1:172.21.15.84:6789/0 conn(0x7f95e406c640 legacy=0x7f95e4159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.106+0000 7f95e359e640 1 -- 172.21.15.84:0/3369954813 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40cc420 con 0x7f95e40ce6b0 2023-10-11T06:14:13.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.108+0000 7f95e359e640 1 --2- 172.21.15.84:0/3369954813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e40ce6b0 0x7f95e40caea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f95e406d780 tx=0x7f95dc02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90762980b967205e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.108+0000 7f95e259c640 1 -- 172.21.15.84:0/3369954813 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4001670 con 0x7f95e40ce6b0 2023-10-11T06:14:13.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.108+0000 7f95e259c640 1 -- 172.21.15.84:0/3369954813 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc02c8c0 con 0x7f95e40ce6b0 2023-10-11T06:14:13.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.108+0000 7f95e259c640 1 -- 172.21.15.84:0/3369954813 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc02cc50 con 0x7f95e40ce6b0 2023-10-11T06:14:13.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 -- 172.21.15.84:0/3369954813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e40ce6b0 msgr2=0x7f95e40caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 --2- 172.21.15.84:0/3369954813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e40ce6b0 0x7f95e40caea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f95e406d780 tx=0x7f95dc02bd90 comp rx=0 tx=0).stop 2023-10-11T06:14:13.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 -- 172.21.15.84:0/3369954813 shutdown_connections 2023-10-11T06:14:13.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 --2- 172.21.15.84:0/3369954813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e40ce6b0 0x7f95e40caea0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 --2- 172.21.15.84:0/3369954813 >> v2:172.21.15.99:3300/0 conn(0x7f95e406ca10 0x7f95e4152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 -- 172.21.15.84:0/3369954813 >> 172.21.15.84:0/3369954813 conn(0x7f95e4069c70 msgr2=0x7f95e406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:13.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 -- 172.21.15.84:0/3369954813 shutdown_connections 2023-10-11T06:14:13.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.109+0000 7f95e91dd640 1 -- 172.21.15.84:0/3369954813 wait complete. 2023-10-11T06:14:13.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.111+0000 7f95e91dd640 1 Processor -- start 2023-10-11T06:14:13.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.111+0000 7f95e91dd640 1 -- start start 2023-10-11T06:14:13.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e91dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e406c640 0x7f95e415f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e91dd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95e406ca10 0x7f95e414cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e91dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95e40ce6b0 0x7f95e40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e91dd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b3630 con 0x7f95e406c640 2023-10-11T06:14:13.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e91dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b3910 con 0x7f95e40ce6b0 2023-10-11T06:14:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e91dd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b3bf0 con 0x7f95e406ca10 2023-10-11T06:14:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e406c640 0x7f95e415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e406c640 0x7f95e415f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53204/0 (socket says 172.21.15.84:53204) 2023-10-11T06:14:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e2d9d640 1 -- 172.21.15.84:0/2006490125 learned_addr learned my addr 172.21.15.84:0/2006490125 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:13.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95e359e640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95e40ce6b0 0x7f95e40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.112+0000 7f95dbfff640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95e406ca10 0x7f95e414cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e2d9d640 1 -- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95e406ca10 msgr2=0x7f95e414cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e2d9d640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95e406ca10 0x7f95e414cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e2d9d640 1 -- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95e40ce6b0 msgr2=0x7f95e40b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e2d9d640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95e40ce6b0 0x7f95e40b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e2d9d640 1 -- 172.21.15.84:0/2006490125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc009a70 con 0x7f95e406c640 2023-10-11T06:14:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e359e640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95e40ce6b0 0x7f95e40b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e2d9d640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e406c640 0x7f95e415f150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f95d400c760 tx=0x7f95d400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95db7fe640 1 -- 172.21.15.84:0/2006490125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4005ad0 con 0x7f95e406c640 2023-10-11T06:14:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95db7fe640 1 -- 172.21.15.84:0/2006490125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4005c30 con 0x7f95e406c640 2023-10-11T06:14:13.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95db7fe640 1 -- 172.21.15.84:0/2006490125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4007070 con 0x7f95e406c640 2023-10-11T06:14:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e40b3ed0 con 0x7f95e406c640 2023-10-11T06:14:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.113+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e414d090 con 0x7f95e406c640 2023-10-11T06:14:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.115+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e40cc6e0 con 0x7f95e406c640 2023-10-11T06:14:13.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.115+0000 7f95db7fe640 1 -- 172.21.15.84:0/2006490125 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f95d40071d0 con 0x7f95e406c640 2023-10-11T06:14:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.116+0000 7f95db7fe640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95c406a860 0x7f95c406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.116+0000 7f95db7fe640 1 -- 172.21.15.84:0/2006490125 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f95d4088c60 con 0x7f95e406c640 2023-10-11T06:14:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.117+0000 7f95dbfff640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95c406a860 0x7f95c406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.117+0000 7f95dbfff640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95c406a860 0x7f95c406cd10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f95e40cb0d0 tx=0x7f95cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.121+0000 7f95db7fe640 1 -- 172.21.15.84:0/2006490125 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f95e40cc6e0 con 0x7f95e406c640 2023-10-11T06:14:13.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.305+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95e40cc6e0 con 0x7f95e406c640 2023-10-11T06:14:13.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.306+0000 7f95db7fe640 1 -- 172.21.15.84:0/2006490125 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95e40cc6e0 con 0x7f95e406c640 2023-10-11T06:14:13.307 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:14:13.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.308+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95c406a860 msgr2=0x7f95c406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.308+0000 7f95e91dd640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95c406a860 0x7f95c406cd10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f95e40cb0d0 tx=0x7f95cc008040 comp rx=0 tx=0).stop 2023-10-11T06:14:13.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e406c640 msgr2=0x7f95e415f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e406c640 0x7f95e415f150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f95d400c760 tx=0x7f95d400cc20 comp rx=0 tx=0).stop 2023-10-11T06:14:13.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 shutdown_connections 2023-10-11T06:14:13.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f95c406a860 0x7f95c406cd10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95e40ce6b0 0x7f95e40b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95e406ca10 0x7f95e414cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 --2- 172.21.15.84:0/2006490125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e406c640 0x7f95e415f150 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 >> 172.21.15.84:0/2006490125 conn(0x7f95e4069c70 msgr2=0x7f95e40ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:13.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 shutdown_connections 2023-10-11T06:14:13.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.309+0000 7f95e91dd640 1 -- 172.21.15.84:0/2006490125 wait complete. 2023-10-11T06:14:13.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:14:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.534+0000 7f61b14d5640 1 Processor -- start 2023-10-11T06:14:13.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.534+0000 7f61b14d5640 1 -- start start 2023-10-11T06:14:13.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61b14d5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f61ac15a0b0 0x7f61ac0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61b14d5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac071380 con 0x7f61ac0c8d00 2023-10-11T06:14:13.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61b14d5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ac06de10 con 0x7f61ac0ced30 2023-10-11T06:14:13.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61b14d5640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac06e0f0 con 0x7f61ac15a0b0 2023-10-11T06:14:13.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61a27fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61ac0c8d00 0x7f61ac0ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35922/0 (socket says 172.21.15.84:35922) 2023-10-11T06:14:13.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61a27fc640 1 -- 172.21.15.84:0/650134024 learned_addr learned my addr 172.21.15.84:0/650134024 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:13.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61aaffd640 1 --2- 172.21.15.84:0/650134024 >> v2:172.21.15.99:3300/0 conn(0x7f61ac15a0b0 0x7f61ac0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.536+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854847129 0 0) 0x7f61ac071380 con 0x7f61ac0c8d00 2023-10-11T06:14:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6198003520 con 0x7f61ac0c8d00 2023-10-11T06:14:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 410329921 0 0) 0x7f6198003520 con 0x7f61ac0c8d00 2023-10-11T06:14:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 >> v2:172.21.15.99:3300/0 conn(0x7f61ac15a0b0 msgr2=0x7f61ac0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 --2- 172.21.15.84:0/650134024 >> v2:172.21.15.99:3300/0 conn(0x7f61ac15a0b0 0x7f61ac0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 >> v1:172.21.15.99:6789/0 conn(0x7f61ac0ced30 legacy=0x7f61ac06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac070dc0 con 0x7f61ac0c8d00 2023-10-11T06:14:13.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f6194001670 con 0x7f61ac0c8d00 2023-10-11T06:14:13.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 >> v1:172.21.15.84:6789/0 conn(0x7f61ac0c8d00 legacy=0x7f61ac0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6198003970 0x7f6198003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f6198004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6198005290 0x7f619800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac06e0f0 con 0x7f61980047c0 2023-10-11T06:14:13.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619800c1c0 con 0x7f6198005290 2023-10-11T06:14:13.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619800c380 con 0x7f6198003970 2023-10-11T06:14:13.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.537+0000 7f61a27fc640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f6198004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.538+0000 7f61a27fc640 1 -- 172.21.15.84:0/650134024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6198003970 msgr2=0x7f6198003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:13.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.538+0000 7f61a27fc640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6198003970 0x7f6198003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.538+0000 7f61a27fc640 1 -- 172.21.15.84:0/650134024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6198005290 msgr2=0x7f619800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:13.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.538+0000 7f61a27fc640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6198005290 0x7f619800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.538+0000 7f61a27fc640 1 -- 172.21.15.84:0/650134024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198003520 con 0x7f61980047c0 2023-10-11T06:14:13.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.538+0000 7f61ab7fe640 1 -- 172.21.15.84:0/650134024 reap_dead start 2023-10-11T06:14:13.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.539+0000 7f61a27fc640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f6198004c30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f619400c240 tx=0x7f61980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a14256b58fa3692 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.539+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194005ad0 con 0x7f61980047c0 2023-10-11T06:14:13.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.539+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194005c30 con 0x7f61980047c0 2023-10-11T06:14:13.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.539+0000 7f61aa7fc640 1 -- 172.21.15.84:0/650134024 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619402caf0 con 0x7f61980047c0 2023-10-11T06:14:13.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.539+0000 7f61b14d5640 1 -- 172.21.15.84:0/650134024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 msgr2=0x7f6198004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.539+0000 7f61b14d5640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f6198004c30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f619400c240 tx=0x7f61980057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.540+0000 7f61b14d5640 1 -- 172.21.15.84:0/650134024 shutdown_connections 2023-10-11T06:14:13.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.540+0000 7f61b14d5640 1 --2- 172.21.15.84:0/650134024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f6198004c30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.540+0000 7f61b14d5640 1 -- 172.21.15.84:0/650134024 >> 172.21.15.84:0/650134024 conn(0x7f61ac069c70 msgr2=0x7f61ac06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:13.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.540+0000 7f61b14d5640 1 -- 172.21.15.84:0/650134024 shutdown_connections 2023-10-11T06:14:13.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.540+0000 7f61b14d5640 1 -- 172.21.15.84:0/650134024 wait complete. 2023-10-11T06:14:13.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.541+0000 7f61b14d5640 1 Processor -- start 2023-10-11T06:14:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.542+0000 7f61b14d5640 1 -- start start 2023-10-11T06:14:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.542+0000 7f61b14d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f61ac15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.542+0000 7f61b14d5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac14b310 0x7f61ac1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61b14d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cca40 0x7f61ac1f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61b14d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0cc620 con 0x7f61980047c0 2023-10-11T06:14:13.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61b14d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac14b780 con 0x7f61ac0cca40 2023-10-11T06:14:13.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61b14d5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0cc280 con 0x7f61ac14b310 2023-10-11T06:14:13.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f61ac15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f61ac15c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53224/0 (socket says 172.21.15.84:53224) 2023-10-11T06:14:13.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 -- 172.21.15.84:0/1925104962 learned_addr learned my addr 172.21.15.84:0/1925104962 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 -- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac14b310 msgr2=0x7f61ac1f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac14b310 0x7f61ac1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61ab7fe640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cca40 0x7f61ac1f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 -- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cca40 msgr2=0x7f61ac1f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cca40 0x7f61ac1f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 -- 172.21.15.84:0/1925104962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac1f26a0 con 0x7f61980047c0 2023-10-11T06:14:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61ab7fe640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cca40 0x7f61ac1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.543+0000 7f61aaffd640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f61ac15c5e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e7a0 tx=0x7f619c00e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.544+0000 7f61a3fff640 1 -- 172.21.15.84:0/1925104962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c013680 con 0x7f61980047c0 2023-10-11T06:14:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.544+0000 7f61a3fff640 1 -- 172.21.15.84:0/1925104962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c013800 con 0x7f61980047c0 2023-10-11T06:14:13.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.544+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac1f2980 con 0x7f61980047c0 2023-10-11T06:14:13.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.544+0000 7f61a3fff640 1 -- 172.21.15.84:0/1925104962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c013bb0 con 0x7f61980047c0 2023-10-11T06:14:13.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.544+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac1f2320 con 0x7f61980047c0 2023-10-11T06:14:13.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.545+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac073ed0 con 0x7f61980047c0 2023-10-11T06:14:13.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.546+0000 7f61a3fff640 1 -- 172.21.15.84:0/1925104962 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f619c013d10 con 0x7f61980047c0 2023-10-11T06:14:13.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.547+0000 7f61a3fff640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f618c06a860 0x7f618c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.547+0000 7f61a3fff640 1 -- 172.21.15.84:0/1925104962 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f619c088c00 con 0x7f61980047c0 2023-10-11T06:14:13.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.547+0000 7f61a27fc640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f618c06a860 0x7f618c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.548+0000 7f61a27fc640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f618c06a860 0x7f618c06cd10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6194034000 tx=0x7f6194039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.551+0000 7f61a3fff640 1 -- 172.21.15.84:0/1925104962 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f619c053e70 con 0x7f61980047c0 2023-10-11T06:14:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.739+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f61ac15bc10 con 0x7f61980047c0 2023-10-11T06:14:13.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.739+0000 7f61a3fff640 1 -- 172.21.15.84:0/1925104962 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f619c0210a0 con 0x7f61980047c0 2023-10-11T06:14:13.741 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:14:13.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f618c06a860 msgr2=0x7f618c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f618c06a860 0x7f618c06cd10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6194034000 tx=0x7f6194039040 comp rx=0 tx=0).stop 2023-10-11T06:14:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 msgr2=0x7f61ac15c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f61ac15c5e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e7a0 tx=0x7f619c00e7d0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 shutdown_connections 2023-10-11T06:14:13.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f618c06a860 0x7f618c06cd10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac0cca40 0x7f61ac1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f61ac14b310 0x7f61ac1f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 --2- 172.21.15.84:0/1925104962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61980047c0 0x7f61ac15c5e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 >> 172.21.15.84:0/1925104962 conn(0x7f61ac069c70 msgr2=0x7f61ac0c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:13.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 shutdown_connections 2023-10-11T06:14:13.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.742+0000 7f61b14d5640 1 -- 172.21.15.84:0/1925104962 wait complete. 2023-10-11T06:14:13.761 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:14:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.973+0000 7fc8ba4f7640 1 Processor -- start 2023-10-11T06:14:13.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.973+0000 7fc8ba4f7640 1 -- start start 2023-10-11T06:14:13.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.975+0000 7fc8ba4f7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc8b40d2f40 0x7fc8b40d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.975+0000 7fc8ba4f7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b40cf220 con 0x7fc8b415f7e0 2023-10-11T06:14:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.975+0000 7fc8ba4f7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b40cf500 con 0x7fc8b40d2ac0 2023-10-11T06:14:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.975+0000 7fc8ba4f7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40cf7e0 con 0x7fc8b40d2f40 2023-10-11T06:14:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.975+0000 7fc8b3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc8b415f7e0 0x7fc8b40cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35924/0 (socket says 172.21.15.84:35924) 2023-10-11T06:14:13.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b3fff640 1 -- 172.21.15.84:0/22300081 learned_addr learned my addr 172.21.15.84:0/22300081 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b8a94640 1 --2- 172.21.15.84:0/22300081 >> v2:172.21.15.99:3300/0 conn(0x7fc8b40d2f40 0x7fc8b40d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150390820 0 0) 0x7fc8b40cf220 con 0x7fc8b415f7e0 2023-10-11T06:14:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc89c003520 con 0x7fc8b415f7e0 2023-10-11T06:14:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504555481 0 0) 0x7fc89c003520 con 0x7fc8b415f7e0 2023-10-11T06:14:13.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 >> v2:172.21.15.99:3300/0 conn(0x7fc8b40d2f40 msgr2=0x7fc8b40d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 --2- 172.21.15.84:0/22300081 >> v2:172.21.15.99:3300/0 conn(0x7fc8b40d2f40 0x7fc8b40d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 >> v1:172.21.15.99:6789/0 conn(0x7fc8b40d2ac0 legacy=0x7fc8b40d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40cfac0 con 0x7fc8b415f7e0 2023-10-11T06:14:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fc8a4001670 con 0x7fc8b415f7e0 2023-10-11T06:14:13.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 >> v1:172.21.15.84:6789/0 conn(0x7fc8b415f7e0 legacy=0x7fc8b40cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c003970 0x7fc89c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc89c0047c0 0x7fc89c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c005170 0x7fc89c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.976+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40cf7e0 con 0x7fc89c005170 2023-10-11T06:14:13.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c00bff0 con 0x7fc89c003970 2023-10-11T06:14:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c00c1b0 con 0x7fc89c0047c0 2023-10-11T06:14:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b8a94640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c005170 0x7fc89c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b37fe640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc89c0047c0 0x7fc89c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b8a94640 1 -- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc89c0047c0 msgr2=0x7fc89c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:13.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b8a94640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc89c0047c0 0x7fc89c004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b3fff640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c003970 0x7fc89c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b8a94640 1 -- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c003970 msgr2=0x7fc89c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b8a94640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c003970 0x7fc89c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b8a94640 1 -- 172.21.15.84:0/22300081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c003520 con 0x7fc89c005170 2023-10-11T06:14:13.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b3fff640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c003970 0x7fc89c003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.977+0000 7fc8b37fe640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc89c0047c0 0x7fc89c004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.978+0000 7fc8b8a94640 1 -- 172.21.15.84:0/22300081 reap_dead start 2023-10-11T06:14:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.978+0000 7fc8b8a94640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c005170 0x7fc89c00b980 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009fc0 tx=0x7fc89c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cdc066c0f066f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac02cc30 con 0x7fc89c005170 2023-10-11T06:14:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac00f040 con 0x7fc89c005170 2023-10-11T06:14:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8b2ffd640 1 -- 172.21.15.84:0/22300081 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac003990 con 0x7fc89c005170 2023-10-11T06:14:13.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/22300081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c005170 msgr2=0x7fc89c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c005170 0x7fc89c00b980 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009fc0 tx=0x7fc89c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/22300081 shutdown_connections 2023-10-11T06:14:13.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/22300081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c005170 0x7fc89c00b980 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/22300081 >> 172.21.15.84:0/22300081 conn(0x7fc8b40713a0 msgr2=0x7fc8b40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.979+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/22300081 shutdown_connections 2023-10-11T06:14:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.980+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/22300081 wait complete. 2023-10-11T06:14:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.982+0000 7fc8ba4f7640 1 Processor -- start 2023-10-11T06:14:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.982+0000 7fc8ba4f7640 1 -- start start 2023-10-11T06:14:13.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8ba4f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c005170 0x7fc8b4153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8ba4f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b40cca10 0x7fc8b40b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8ba4f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8b40cd140 0x7fc8b40cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8ba4f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40cdaf0 con 0x7fc8b40cca10 2023-10-11T06:14:13.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8ba4f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b41544b0 con 0x7fc89c005170 2023-10-11T06:14:13.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8ba4f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4154a50 con 0x7fc8b40cd140 2023-10-11T06:14:13.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b40cca10 0x7fc8b40b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b40cca10 0x7fc8b40b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53248/0 (socket says 172.21.15.84:53248) 2023-10-11T06:14:13.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 -- 172.21.15.84:0/1336730239 learned_addr learned my addr 172.21.15.84:0/1336730239 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:13.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 -- 172.21.15.84:0/1336730239 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8b40cd140 msgr2=0x7fc8b40cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8b40cd140 0x7fc8b40cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 -- 172.21.15.84:0/1336730239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c005170 msgr2=0x7fc8b4153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:13.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c005170 0x7fc8b4153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:13.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.984+0000 7fc8b37fe640 1 -- 172.21.15.84:0/1336730239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4154d30 con 0x7fc8b40cca10 2023-10-11T06:14:13.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.985+0000 7fc8b37fe640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b40cca10 0x7fc8b40b6290 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800d560 tx=0x7fc8a800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.985+0000 7fc8b0ff9640 1 -- 172.21.15.84:0/1336730239 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8005480 con 0x7fc8b40cca10 2023-10-11T06:14:13.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.985+0000 7fc8b0ff9640 1 -- 172.21.15.84:0/1336730239 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a8003610 con 0x7fc8b40cca10 2023-10-11T06:14:13.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.985+0000 7fc8b0ff9640 1 -- 172.21.15.84:0/1336730239 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8003980 con 0x7fc8b40cca10 2023-10-11T06:14:13.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.985+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4155010 con 0x7fc8b40cca10 2023-10-11T06:14:13.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.985+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b41555d0 con 0x7fc8b40cca10 2023-10-11T06:14:13.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.986+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b4155760 con 0x7fc8b40cca10 2023-10-11T06:14:13.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.987+0000 7fc8b0ff9640 1 -- 172.21.15.84:0/1336730239 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc8a8003ae0 con 0x7fc8b40cca10 2023-10-11T06:14:13.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.988+0000 7fc8b0ff9640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc8a006a830 0x7fc8a006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:13.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.988+0000 7fc8b0ff9640 1 -- 172.21.15.84:0/1336730239 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8a8088f20 con 0x7fc8b40cca10 2023-10-11T06:14:13.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.988+0000 7fc8b3fff640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc8a006a830 0x7fc8a006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:13.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.989+0000 7fc8b3fff640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc8a006a830 0x7fc8a006cce0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40017a0 tx=0x7fc8a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:13.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:13.992+0000 7fc8b0ff9640 1 -- 172.21.15.84:0/1336730239 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc8a8054100 con 0x7fc8b40cca10 2023-10-11T06:14:14.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.185+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8b4073ed0 con 0x7fc8b40cca10 2023-10-11T06:14:14.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.186+0000 7fc8b0ff9640 1 -- 172.21.15.84:0/1336730239 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8a8001230 con 0x7fc8b40cca10 2023-10-11T06:14:14.187 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:14:14.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc8a006a830 msgr2=0x7fc8a006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc8a006a830 0x7fc8a006cce0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40017a0 tx=0x7fc8a4009040 comp rx=0 tx=0).stop 2023-10-11T06:14:14.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b40cca10 msgr2=0x7fc8b40b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b40cca10 0x7fc8b40b6290 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800d560 tx=0x7fc8a800da20 comp rx=0 tx=0).stop 2023-10-11T06:14:14.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 shutdown_connections 2023-10-11T06:14:14.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc8a006a830 0x7fc8a006cce0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8b40cd140 0x7fc8b40cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8b40cca10 0x7fc8b40b6290 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 --2- 172.21.15.84:0/1336730239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc89c005170 0x7fc8b4153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 >> 172.21.15.84:0/1336730239 conn(0x7fc8b40713a0 msgr2=0x7fc8b4160740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:14.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.188+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 shutdown_connections 2023-10-11T06:14:14.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.189+0000 7fc8ba4f7640 1 -- 172.21.15.84:0/1336730239 wait complete. 2023-10-11T06:14:14.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:14:14.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.418+0000 7ff481bd0640 1 Processor -- start 2023-10-11T06:14:14.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.418+0000 7ff481bd0640 1 -- start start 2023-10-11T06:14:14.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff481bd0640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c14ff30 0x7ff47c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff481bd0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff47c0ce510 0x7ff47c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff481bd0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c0c9b90 con 0x7ff47c150300 2023-10-11T06:14:14.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff481bd0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c15da10 con 0x7ff47c0ce510 2023-10-11T06:14:14.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff481bd0640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0ceef0 con 0x7ff47c14ff30 2023-10-11T06:14:14.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff47bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff47c0ce510 0x7ff47c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff472ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff47c150300 0x7ff47c0cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35936/0 (socket says 172.21.15.84:35936) 2023-10-11T06:14:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.420+0000 7ff472ffd640 1 -- 172.21.15.84:0/2486369594 learned_addr learned my addr 172.21.15.84:0/2486369594 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.421+0000 7ff47b7fe640 1 --2- 172.21.15.84:0/2486369594 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c14ff30 0x7ff47c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.421+0000 7ff47bfff640 1 -- 172.21.15.84:0/2486369594 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c14ff30 msgr2=0x7ff47c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.421+0000 7ff47bfff640 1 --2- 172.21.15.84:0/2486369594 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c14ff30 0x7ff47c0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.421+0000 7ff47bfff640 1 -- 172.21.15.84:0/2486369594 >> v1:172.21.15.84:6789/0 conn(0x7ff47c150300 legacy=0x7ff47c0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.421+0000 7ff47bfff640 1 -- 172.21.15.84:0/2486369594 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c0cc990 con 0x7ff47c0ce510 2023-10-11T06:14:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.421+0000 7ff47affd640 1 -- 172.21.15.84:0/2486369594 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914128253 0 0) 0x7ff47c0c9b90 con 0x7ff47c150300 2023-10-11T06:14:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.422+0000 7ff47bfff640 1 --2- 172.21.15.84:0/2486369594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c0ce510 0x7ff47c0ce9b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff474009d90 tx=0x7ff47402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d967a4d6e15527e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.422+0000 7ff47affd640 1 -- 172.21.15.84:0/2486369594 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47402cbc0 con 0x7ff47c0ce510 2023-10-11T06:14:14.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.422+0000 7ff47affd640 1 -- 172.21.15.84:0/2486369594 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47402cd20 con 0x7ff47c0ce510 2023-10-11T06:14:14.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.422+0000 7ff47affd640 1 -- 172.21.15.84:0/2486369594 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474034660 con 0x7ff47c0ce510 2023-10-11T06:14:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 -- 172.21.15.84:0/2486369594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c0ce510 msgr2=0x7ff47c0ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2486369594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c0ce510 0x7ff47c0ce9b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff474009d90 tx=0x7ff47402be10 comp rx=0 tx=0).stop 2023-10-11T06:14:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 -- 172.21.15.84:0/2486369594 shutdown_connections 2023-10-11T06:14:14.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2486369594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c0ce510 0x7ff47c0ce9b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2486369594 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c14ff30 0x7ff47c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 -- 172.21.15.84:0/2486369594 >> 172.21.15.84:0/2486369594 conn(0x7ff47c069c70 msgr2=0x7ff47c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:14.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 -- 172.21.15.84:0/2486369594 shutdown_connections 2023-10-11T06:14:14.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.423+0000 7ff481bd0640 1 -- 172.21.15.84:0/2486369594 wait complete. 2023-10-11T06:14:14.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.425+0000 7ff481bd0640 1 Processor -- start 2023-10-11T06:14:14.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.425+0000 7ff481bd0640 1 -- start start 2023-10-11T06:14:14.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff481bd0640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c0ce510 0x7ff47c1ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff481bd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c14ff30 0x7ff47c0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff481bd0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47c150300 0x7ff47c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff481bd0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c152850 con 0x7ff47c14ff30 2023-10-11T06:14:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff481bd0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1ef8e0 con 0x7ff47c150300 2023-10-11T06:14:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff481bd0640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c151bd0 con 0x7ff47c0ce510 2023-10-11T06:14:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c14ff30 0x7ff47c0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff472ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c14ff30 0x7ff47c0c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53264/0 (socket says 172.21.15.84:53264) 2023-10-11T06:14:14.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.426+0000 7ff472ffd640 1 -- 172.21.15.84:0/2320611210 learned_addr learned my addr 172.21.15.84:0/2320611210 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff47b7fe640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c0ce510 0x7ff47c1ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff472ffd640 1 -- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c0ce510 msgr2=0x7ff47c1ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff472ffd640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c0ce510 0x7ff47c1ef3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff47bfff640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47c150300 0x7ff47c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff472ffd640 1 -- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47c150300 msgr2=0x7ff47c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff472ffd640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47c150300 0x7ff47c0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff472ffd640 1 -- 172.21.15.84:0/2320611210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474009a70 con 0x7ff47c14ff30 2023-10-11T06:14:14.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff472ffd640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c14ff30 0x7ff47c0c6270 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c440 tx=0x7ff46400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:14.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff478ff9640 1 -- 172.21.15.84:0/2320611210 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640074e0 con 0x7ff47c14ff30 2023-10-11T06:14:14.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff478ff9640 1 -- 172.21.15.84:0/2320611210 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff464007640 con 0x7ff47c14ff30 2023-10-11T06:14:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff478ff9640 1 -- 172.21.15.84:0/2320611210 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464004a20 con 0x7ff47c14ff30 2023-10-11T06:14:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.427+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c151eb0 con 0x7ff47c14ff30 2023-10-11T06:14:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.428+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c152470 con 0x7ff47c14ff30 2023-10-11T06:14:14.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.429+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c152990 con 0x7ff47c14ff30 2023-10-11T06:14:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.430+0000 7ff478ff9640 1 -- 172.21.15.84:0/2320611210 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff464004b80 con 0x7ff47c14ff30 2023-10-11T06:14:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.431+0000 7ff478ff9640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff45c06a830 0x7ff45c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.431+0000 7ff478ff9640 1 -- 172.21.15.84:0/2320611210 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4640881b0 con 0x7ff47c14ff30 2023-10-11T06:14:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.431+0000 7ff47b7fe640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff45c06a830 0x7ff45c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.432+0000 7ff47b7fe640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff45c06a830 0x7ff45c06cce0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff46c006fd0 tx=0x7ff46c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:14.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.435+0000 7ff478ff9640 1 -- 172.21.15.84:0/2320611210 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff47c152990 con 0x7ff47c14ff30 2023-10-11T06:14:14.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.623+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff47c152990 con 0x7ff47c14ff30 2023-10-11T06:14:14.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.624+0000 7ff478ff9640 1 -- 172.21.15.84:0/2320611210 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff464018020 con 0x7ff47c14ff30 2023-10-11T06:14:14.626 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:14:14.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff45c06a830 msgr2=0x7ff45c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff45c06a830 0x7ff45c06cce0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff46c006fd0 tx=0x7ff46c008040 comp rx=0 tx=0).stop 2023-10-11T06:14:14.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c14ff30 msgr2=0x7ff47c0c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c14ff30 0x7ff47c0c6270 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c440 tx=0x7ff46400c900 comp rx=0 tx=0).stop 2023-10-11T06:14:14.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 shutdown_connections 2023-10-11T06:14:14.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff45c06a830 0x7ff45c06cce0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47c150300 0x7ff47c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff47c14ff30 0x7ff47c0c6270 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 --2- 172.21.15.84:0/2320611210 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff47c0ce510 0x7ff47c1ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 >> 172.21.15.84:0/2320611210 conn(0x7ff47c069c70 msgr2=0x7ff47c0c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:14.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 shutdown_connections 2023-10-11T06:14:14.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.627+0000 7ff481bd0640 1 -- 172.21.15.84:0/2320611210 wait complete. 2023-10-11T06:14:14.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:14:14.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.860+0000 7f29b4eb6640 1 Processor -- start 2023-10-11T06:14:14.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.860+0000 7f29b4eb6640 1 -- start start 2023-10-11T06:14:14.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29b4eb6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29b00ce0d0 0x7f29b00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29b4eb6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f29b00adb90 0x7f29b00caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29b4eb6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b01593f0 con 0x7f29b00cd300 2023-10-11T06:14:14.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29b4eb6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b015a340 con 0x7f29b00ce0d0 2023-10-11T06:14:14.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29b4eb6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b00ce4a0 con 0x7f29b00adb90 2023-10-11T06:14:14.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29ae59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29b00ce0d0 0x7f29b00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29ae59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29b00ce0d0 0x7f29b00c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53272/0 (socket says 172.21.15.84:53272) 2023-10-11T06:14:14.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29ae59c640 1 -- 172.21.15.84:0/3061594604 learned_addr learned my addr 172.21.15.84:0/3061594604 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:14.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29ae59c640 1 -- 172.21.15.84:0/3061594604 >> v2:172.21.15.99:3300/0 conn(0x7f29b00adb90 msgr2=0x7f29b00caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:14.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.862+0000 7f29a7fff640 1 --2- 172.21.15.84:0/3061594604 >> v2:172.21.15.99:3300/0 conn(0x7f29b00adb90 0x7f29b00caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.863+0000 7f29ae59c640 1 --2- 172.21.15.84:0/3061594604 >> v2:172.21.15.99:3300/0 conn(0x7f29b00adb90 0x7f29b00caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.863+0000 7f29ae59c640 1 -- 172.21.15.84:0/3061594604 >> v1:172.21.15.99:6789/0 conn(0x7f29b00cd300 legacy=0x7f29b00b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:14.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.863+0000 7f29ae59c640 1 -- 172.21.15.84:0/3061594604 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b01596c0 con 0x7f29b00ce0d0 2023-10-11T06:14:14.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.863+0000 7f29a7fff640 1 --2- 172.21.15.84:0/3061594604 >> v2:172.21.15.99:3300/0 conn(0x7f29b00adb90 0x7f29b00caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:14.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.864+0000 7f29ae59c640 1 --2- 172.21.15.84:0/3061594604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00ce0d0 0x7f29b00c8cc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f29b00cc8c0 tx=0x7f29a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e9f7342324c0a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:14.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.864+0000 7f29add9b640 1 -- 172.21.15.84:0/3061594604 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0004230 con 0x7f29b00ce0d0 2023-10-11T06:14:14.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.864+0000 7f29add9b640 1 -- 172.21.15.84:0/3061594604 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0004390 con 0x7f29b00ce0d0 2023-10-11T06:14:14.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.864+0000 7f29add9b640 1 -- 172.21.15.84:0/3061594604 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0031830 con 0x7f29b00ce0d0 2023-10-11T06:14:14.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 -- 172.21.15.84:0/3061594604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00ce0d0 msgr2=0x7f29b00c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/3061594604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00ce0d0 0x7f29b00c8cc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f29b00cc8c0 tx=0x7f29a002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 -- 172.21.15.84:0/3061594604 shutdown_connections 2023-10-11T06:14:14.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/3061594604 >> v2:172.21.15.99:3300/0 conn(0x7f29b00adb90 0x7f29b00caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/3061594604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00ce0d0 0x7f29b00c8cc0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 -- 172.21.15.84:0/3061594604 >> 172.21.15.84:0/3061594604 conn(0x7f29b0069c70 msgr2=0x7f29b006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:14.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 -- 172.21.15.84:0/3061594604 shutdown_connections 2023-10-11T06:14:14.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.865+0000 7f29b4eb6640 1 -- 172.21.15.84:0/3061594604 wait complete. 2023-10-11T06:14:14.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.867+0000 7f29b4eb6640 1 Processor -- start 2023-10-11T06:14:14.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.867+0000 7f29b4eb6640 1 -- start start 2023-10-11T06:14:14.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29b4eb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00adb90 0x7f29b00b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29b4eb6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29b00cd300 0x7f29b01f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29b4eb6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29b00ce0d0 0x7f29b01f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29b4eb6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00c56d0 con 0x7f29b00adb90 2023-10-11T06:14:14.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29b4eb6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0158610 con 0x7f29b00cd300 2023-10-11T06:14:14.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29b4eb6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00b0590 con 0x7f29b00ce0d0 2023-10-11T06:14:14.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29ae59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00adb90 0x7f29b00b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29ae59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00adb90 0x7f29b00b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53286/0 (socket says 172.21.15.84:53286) 2023-10-11T06:14:14.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29ae59c640 1 -- 172.21.15.84:0/1352790033 learned_addr learned my addr 172.21.15.84:0/1352790033 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:14.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29a7fff640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29b00cd300 0x7f29b01f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29aed9d640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29b00ce0d0 0x7f29b01f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29ae59c640 1 -- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29b00ce0d0 msgr2=0x7f29b01f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.868+0000 7f29ae59c640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29b00ce0d0 0x7f29b01f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29ae59c640 1 -- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29b00cd300 msgr2=0x7f29b01f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:14.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29ae59c640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29b00cd300 0x7f29b01f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:14.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29ae59c640 1 -- 172.21.15.84:0/1352790033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009460 con 0x7f29b00adb90 2023-10-11T06:14:14.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29a7fff640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29b00cd300 0x7f29b01f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:14.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29ae59c640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00adb90 0x7f29b00b0050 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000c00 tx=0x7f29a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:14.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29a6ffd640 1 -- 172.21.15.84:0/1352790033 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0031830 con 0x7f29b00adb90 2023-10-11T06:14:14.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29a6ffd640 1 -- 172.21.15.84:0/1352790033 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a003f070 con 0x7f29b00adb90 2023-10-11T06:14:14.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b00b0830 con 0x7f29b00adb90 2023-10-11T06:14:14.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29a6ffd640 1 -- 172.21.15.84:0/1352790033 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0031bd0 con 0x7f29b00adb90 2023-10-11T06:14:14.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.869+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b00c58b0 con 0x7f29b00adb90 2023-10-11T06:14:14.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.870+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b0073ed0 con 0x7f29b00adb90 2023-10-11T06:14:14.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.871+0000 7f29a6ffd640 1 -- 172.21.15.84:0/1352790033 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f29a0041030 con 0x7f29b00adb90 2023-10-11T06:14:14.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.872+0000 7f29a6ffd640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f299006a830 0x7f299006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:14.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.872+0000 7f29a6ffd640 1 -- 172.21.15.84:0/1352790033 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29a00ab320 con 0x7f29b00adb90 2023-10-11T06:14:14.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.872+0000 7f29a7fff640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f299006a830 0x7f299006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:14.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.873+0000 7f29a7fff640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f299006a830 0x7f299006cce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:14.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:14.876+0000 7f29a6ffd640 1 -- 172.21.15.84:0/1352790033 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f29a0076500 con 0x7f29b00adb90 2023-10-11T06:14:15.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.083+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29b0073ed0 con 0x7f29b00adb90 2023-10-11T06:14:15.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.084+0000 7f29a6ffd640 1 -- 172.21.15.84:0/1352790033 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f29a0031d30 con 0x7f29b00adb90 2023-10-11T06:14:15.085 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:14:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.086+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f299006a830 msgr2=0x7f299006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f299006a830 0x7f299006cce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998008040 comp rx=0 tx=0).stop 2023-10-11T06:14:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00adb90 msgr2=0x7f29b00b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00adb90 0x7f29b00b0050 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f29a0000c00 tx=0x7f29a00046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 shutdown_connections 2023-10-11T06:14:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f299006a830 0x7f299006cce0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29b00ce0d0 0x7f29b01f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29b00cd300 0x7f29b01f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 --2- 172.21.15.84:0/1352790033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29b00adb90 0x7f29b00b0050 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 >> 172.21.15.84:0/1352790033 conn(0x7f29b0069c70 msgr2=0x7f29b01588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 shutdown_connections 2023-10-11T06:14:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.087+0000 7f29b4eb6640 1 -- 172.21.15.84:0/1352790033 wait complete. 2023-10-11T06:14:15.106 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:14:15.107 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:15.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.413+0000 7f3749f5c640 1 Processor -- start 2023-10-11T06:14:15.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.413+0000 7f3749f5c640 1 -- start start 2023-10-11T06:14:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.414+0000 7f3749f5c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f37440d0c20 0x7f37440af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.414+0000 7f3749f5c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 0x7f37440b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.414+0000 7f3749f5c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440d2a40 con 0x7f374415c290 2023-10-11T06:14:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.414+0000 7f3749f5c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374406cd10 con 0x7f37440d0c20 2023-10-11T06:14:15.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.414+0000 7f3749f5c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37440afb60 con 0x7f37440d25e0 2023-10-11T06:14:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.415+0000 7f37437fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f374415c290 0x7f37440d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35940/0 (socket says 172.21.15.84:35940) 2023-10-11T06:14:15.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.415+0000 7f37437fe640 1 -- 172.21.15.84:0/2995611753 learned_addr learned my addr 172.21.15.84:0/2995611753 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:15.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.415+0000 7f3742ffd640 1 --2- 172.21.15.84:0/2995611753 >> v2:172.21.15.99:3300/0 conn(0x7f37440d0c20 0x7f37440af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.415+0000 7f3743fff640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 0x7f37440b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.415+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169638165 0 0) 0x7f37440d2a40 con 0x7f374415c290 2023-10-11T06:14:15.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.415+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f372c003520 con 0x7f374415c290 2023-10-11T06:14:15.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232620998 0 0) 0x7f372c003520 con 0x7f374415c290 2023-10-11T06:14:15.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 msgr2=0x7f37440b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:15.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 0x7f37440b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 >> v2:172.21.15.99:3300/0 conn(0x7f37440d0c20 msgr2=0x7f37440af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:15.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 --2- 172.21.15.84:0/2995611753 >> v2:172.21.15.99:3300/0 conn(0x7f37440d0c20 0x7f37440af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440afe40 con 0x7f374415c290 2023-10-11T06:14:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f3734001670 con 0x7f374415c290 2023-10-11T06:14:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f3743fff640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 0x7f37440b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:14:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 >> v1:172.21.15.84:6789/0 conn(0x7f374415c290 legacy=0x7f37440d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c003970 0x7f372c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 0x7f372c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f372c004d00 0x7f372c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374406cd10 con 0x7f372c004d00 2023-10-11T06:14:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440afb60 con 0x7f372c003970 2023-10-11T06:14:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372c00ba30 con 0x7f37440d25e0 2023-10-11T06:14:15.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.416+0000 7f3743fff640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f372c004d00 0x7f372c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.417+0000 7f3743fff640 1 -- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 msgr2=0x7f372c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.417+0000 7f3743fff640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f37440d25e0 0x7f372c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.417+0000 7f37437fe640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c003970 0x7f372c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.417+0000 7f3743fff640 1 -- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c003970 msgr2=0x7f372c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:15.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.417+0000 7f3743fff640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c003970 0x7f372c003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.417+0000 7f3743fff640 1 -- 172.21.15.84:0/2995611753 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c003520 con 0x7f372c004d00 2023-10-11T06:14:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.418+0000 7f3743fff640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f372c004d00 0x7f372c005170 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f373c0073e0 tx=0x7f372c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b121141f2101d46 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.418+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c02cd70 con 0x7f372c004d00 2023-10-11T06:14:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.418+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c034440 con 0x7f372c004d00 2023-10-11T06:14:15.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.418+0000 7f37427fc640 1 -- 172.21.15.84:0/2995611753 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c0347b0 con 0x7f372c004d00 2023-10-11T06:14:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.419+0000 7f3749f5c640 1 -- 172.21.15.84:0/2995611753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f372c004d00 msgr2=0x7f372c005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.419+0000 7f3749f5c640 1 --2- 172.21.15.84:0/2995611753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f372c004d00 0x7f372c005170 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f373c0073e0 tx=0x7f372c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.419+0000 7f3743fff640 1 -- 172.21.15.84:0/2995611753 reap_dead start 2023-10-11T06:14:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.419+0000 7f3749f5c640 1 -- 172.21.15.84:0/2995611753 shutdown_connections 2023-10-11T06:14:15.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.419+0000 7f3749f5c640 1 -- 172.21.15.84:0/2995611753 >> 172.21.15.84:0/2995611753 conn(0x7f3744071cc0 msgr2=0x7f374409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:15.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.419+0000 7f3749f5c640 1 -- 172.21.15.84:0/2995611753 shutdown_connections 2023-10-11T06:14:15.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.419+0000 7f3749f5c640 1 -- 172.21.15.84:0/2995611753 wait complete. 2023-10-11T06:14:15.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.421+0000 7f3749f5c640 1 Processor -- start 2023-10-11T06:14:15.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.421+0000 7f3749f5c640 1 -- start start 2023-10-11T06:14:15.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3749f5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c004d00 0x7f374414aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3749f5c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f374409c5e0 0x7f37440b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3749f5c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f374409db80 0x7f37440c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3749f5c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37440b81b0 con 0x7f374409db80 2023-10-11T06:14:15.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3749f5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374409cae0 con 0x7f372c004d00 2023-10-11T06:14:15.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3749f5c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374409cdc0 con 0x7f374409c5e0 2023-10-11T06:14:15.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f374409db80 0x7f37440c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f374409db80 0x7f37440c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53306/0 (socket says 172.21.15.84:53306) 2023-10-11T06:14:15.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 -- 172.21.15.84:0/1041818946 learned_addr learned my addr 172.21.15.84:0/1041818946 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:15.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f37437fe640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c004d00 0x7f374414aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3742ffd640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f374409c5e0 0x7f37440b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 -- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f374409c5e0 msgr2=0x7f37440b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f374409c5e0 0x7f37440b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 -- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c004d00 msgr2=0x7f374414aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c004d00 0x7f374414aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.422+0000 7f3743fff640 1 -- 172.21.15.84:0/1041818946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374409d0a0 con 0x7f374409db80 2023-10-11T06:14:15.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.423+0000 7f3743fff640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f374409db80 0x7f37440c81d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f373c0073e0 tx=0x7f373c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:15.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.423+0000 7f372bfff640 1 -- 172.21.15.84:0/1041818946 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c034cc0 con 0x7f374409db80 2023-10-11T06:14:15.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.423+0000 7f372bfff640 1 -- 172.21.15.84:0/1041818946 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c034e20 con 0x7f374409db80 2023-10-11T06:14:15.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.423+0000 7f372bfff640 1 -- 172.21.15.84:0/1041818946 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c003bb0 con 0x7f374409db80 2023-10-11T06:14:15.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.423+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374409d380 con 0x7f374409db80 2023-10-11T06:14:15.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.423+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37440a0520 con 0x7f374409db80 2023-10-11T06:14:15.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.424+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374409d8a0 con 0x7f374409db80 2023-10-11T06:14:15.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.425+0000 7f372bfff640 1 -- 172.21.15.84:0/1041818946 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f373c03e440 con 0x7f374409db80 2023-10-11T06:14:15.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.426+0000 7f372bfff640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f373006a860 0x7f373006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.426+0000 7f372bfff640 1 -- 172.21.15.84:0/1041818946 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f373c0aad60 con 0x7f374409db80 2023-10-11T06:14:15.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.427+0000 7f37437fe640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f373006a860 0x7f373006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.427+0000 7f37437fe640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f373006a860 0x7f373006cd10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3734001eb0 tx=0x7f3734009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:15.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.431+0000 7f372bfff640 1 -- 172.21.15.84:0/1041818946 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f374409d8a0 con 0x7f374409db80 2023-10-11T06:14:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.623+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f374409d8a0 con 0x7f374409db80 2023-10-11T06:14:15.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.626+0000 7f372bfff640 1 -- 172.21.15.84:0/1041818946 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7f374409d8a0 con 0x7f374409db80 2023-10-11T06:14:15.628 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:15.629 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:15.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f373006a860 msgr2=0x7f373006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f373006a860 0x7f373006cd10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3734001eb0 tx=0x7f3734009040 comp rx=0 tx=0).stop 2023-10-11T06:14:15.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f374409db80 msgr2=0x7f37440c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f374409db80 0x7f37440c81d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f373c0073e0 tx=0x7f373c004ab0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 shutdown_connections 2023-10-11T06:14:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f373006a860 0x7f373006cd10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f374409db80 0x7f37440c81d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f374409c5e0 0x7f37440b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 --2- 172.21.15.84:0/1041818946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f372c004d00 0x7f374414aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.629+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 >> 172.21.15.84:0/1041818946 conn(0x7f3744071cc0 msgr2=0x7f374409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.630+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 shutdown_connections 2023-10-11T06:14:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.630+0000 7f3749f5c640 1 -- 172.21.15.84:0/1041818946 wait complete. 2023-10-11T06:14:15.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:13:44.841435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:13:45.401366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:14:15.651 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:14:15.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.868+0000 7f5d32592640 1 Processor -- start 2023-10-11T06:14:15.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.868+0000 7f5d32592640 1 -- start start 2023-10-11T06:14:15.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d32592640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d2c14ff30 0x7f5d2c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d32592640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d2c150300 0x7f5d2c0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d32592640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c0cc7c0 0x7f5d2c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d32592640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c0c9b90 con 0x7f5d2c150300 2023-10-11T06:14:15.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d32592640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c15da10 con 0x7f5d2c14ff30 2023-10-11T06:14:15.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d32592640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c15ce00 con 0x7f5d2c0cc7c0 2023-10-11T06:14:15.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d23fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d2c150300 0x7f5d2c0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d23fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d2c150300 0x7f5d2c0cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53322/0 (socket says 172.21.15.84:53322) 2023-10-11T06:14:15.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d23fff640 1 -- 172.21.15.84:0/559841734 learned_addr learned my addr 172.21.15.84:0/559841734 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:15.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d2bfff640 1 --2- 172.21.15.84:0/559841734 >> v2:172.21.15.99:3300/0 conn(0x7f5d2c14ff30 0x7f5d2c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d30b2f640 1 --2- 172.21.15.84:0/559841734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c0cc7c0 0x7f5d2c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d23fff640 1 -- 172.21.15.84:0/559841734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c0cc7c0 msgr2=0x7f5d2c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:15.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d23fff640 1 --2- 172.21.15.84:0/559841734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c0cc7c0 0x7f5d2c0ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.870+0000 7f5d23fff640 1 -- 172.21.15.84:0/559841734 >> v2:172.21.15.99:3300/0 conn(0x7f5d2c14ff30 msgr2=0x7f5d2c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:15.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.871+0000 7f5d23fff640 1 --2- 172.21.15.84:0/559841734 >> v2:172.21.15.99:3300/0 conn(0x7f5d2c14ff30 0x7f5d2c0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.871+0000 7f5d23fff640 1 -- 172.21.15.84:0/559841734 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c15d0e0 con 0x7f5d2c150300 2023-10-11T06:14:15.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.871+0000 7f5d30b2f640 1 --2- 172.21.15.84:0/559841734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c0cc7c0 0x7f5d2c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:15.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.872+0000 7f5d23fff640 1 --2- 172.21.15.84:0/559841734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c0cc280 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c06e390 tx=0x7f5d1402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a0d99ee7deac7a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:15.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.872+0000 7f5d2b7fe640 1 -- 172.21.15.84:0/559841734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d140317e0 con 0x7f5d2c150300 2023-10-11T06:14:15.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.872+0000 7f5d2b7fe640 1 -- 172.21.15.84:0/559841734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14031940 con 0x7f5d2c150300 2023-10-11T06:14:15.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.872+0000 7f5d2b7fe640 1 -- 172.21.15.84:0/559841734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14031cd0 con 0x7f5d2c150300 2023-10-11T06:14:15.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 -- 172.21.15.84:0/559841734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 msgr2=0x7f5d2c0cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 --2- 172.21.15.84:0/559841734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c0cc280 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c06e390 tx=0x7f5d1402b880 comp rx=0 tx=0).stop 2023-10-11T06:14:15.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 -- 172.21.15.84:0/559841734 shutdown_connections 2023-10-11T06:14:15.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 --2- 172.21.15.84:0/559841734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c0cc280 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 --2- 172.21.15.84:0/559841734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c0cc7c0 0x7f5d2c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 --2- 172.21.15.84:0/559841734 >> v2:172.21.15.99:3300/0 conn(0x7f5d2c14ff30 0x7f5d2c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 -- 172.21.15.84:0/559841734 >> 172.21.15.84:0/559841734 conn(0x7f5d2c069c70 msgr2=0x7f5d2c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:15.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 -- 172.21.15.84:0/559841734 shutdown_connections 2023-10-11T06:14:15.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.873+0000 7f5d32592640 1 -- 172.21.15.84:0/559841734 wait complete. 2023-10-11T06:14:15.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.875+0000 7f5d32592640 1 Processor -- start 2023-10-11T06:14:15.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.875+0000 7f5d32592640 1 -- start start 2023-10-11T06:14:15.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d32592640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c14ff30 0x7f5d2c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d32592640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d32592640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c155c80 0x7f5d2c15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d32592640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c156100 con 0x7f5d2c150300 2023-10-11T06:14:15.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d32592640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c1efb10 con 0x7f5d2c155c80 2023-10-11T06:14:15.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d32592640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c15c6b0 con 0x7f5d2c14ff30 2023-10-11T06:14:15.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53338/0 (socket says 172.21.15.84:53338) 2023-10-11T06:14:15.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 -- 172.21.15.84:0/614363826 learned_addr learned my addr 172.21.15.84:0/614363826 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:15.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d2bfff640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c14ff30 0x7f5d2c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d30b2f640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c155c80 0x7f5d2c15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 -- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c14ff30 msgr2=0x7f5d2c14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c14ff30 0x7f5d2c14e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 -- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c155c80 msgr2=0x7f5d2c15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:15.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c155c80 0x7f5d2c15b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:15.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d23fff640 1 -- 172.21.15.84:0/614363826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d140094d0 con 0x7f5d2c150300 2023-10-11T06:14:15.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.877+0000 7f5d2bfff640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c14ff30 0x7f5d2c14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:15.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.878+0000 7f5d23fff640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c155740 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5d1402b8c0 tx=0x7f5d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:15.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.878+0000 7f5d297fa640 1 -- 172.21.15.84:0/614363826 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14032880 con 0x7f5d2c150300 2023-10-11T06:14:15.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.878+0000 7f5d297fa640 1 -- 172.21.15.84:0/614363826 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d140329e0 con 0x7f5d2c150300 2023-10-11T06:14:15.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.878+0000 7f5d297fa640 1 -- 172.21.15.84:0/614363826 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14032d90 con 0x7f5d2c150300 2023-10-11T06:14:15.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.878+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c1f0680 con 0x7f5d2c150300 2023-10-11T06:14:15.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.878+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c1f0b10 con 0x7f5d2c150300 2023-10-11T06:14:15.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.879+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c1f0810 con 0x7f5d2c150300 2023-10-11T06:14:15.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.880+0000 7f5d297fa640 1 -- 172.21.15.84:0/614363826 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f5d1404e450 con 0x7f5d2c150300 2023-10-11T06:14:15.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.881+0000 7f5d297fa640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d1006a860 0x7f5d1006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:15.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.881+0000 7f5d297fa640 1 -- 172.21.15.84:0/614363826 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d140ab1c0 con 0x7f5d2c150300 2023-10-11T06:14:15.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.881+0000 7f5d2bfff640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d1006a860 0x7f5d1006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:15.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.882+0000 7f5d2bfff640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d1006a860 0x7f5d1006cd10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006fd0 tx=0x7f5d1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:15.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:15.885+0000 7f5d297fa640 1 -- 172.21.15.84:0/614363826 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5d14076470 con 0x7f5d2c150300 2023-10-11T06:14:16.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.076+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d2c06ab90 con 0x7f5d2c150300 2023-10-11T06:14:16.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.077+0000 7f5d297fa640 1 -- 172.21.15.84:0/614363826 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d14079a90 con 0x7f5d2c150300 2023-10-11T06:14:16.078 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:14:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d1006a860 msgr2=0x7f5d1006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d1006a860 0x7f5d1006cd10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c006fd0 tx=0x7f5d1c008040 comp rx=0 tx=0).stop 2023-10-11T06:14:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 msgr2=0x7f5d2c155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c155740 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5d1402b8c0 tx=0x7f5d140046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 shutdown_connections 2023-10-11T06:14:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f5d1006a860 0x7f5d1006cd10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d2c155c80 0x7f5d2c15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d2c150300 0x7f5d2c155740 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 --2- 172.21.15.84:0/614363826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d2c14ff30 0x7f5d2c14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.079+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 >> 172.21.15.84:0/614363826 conn(0x7f5d2c069c70 msgr2=0x7f5d2c0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.080+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 shutdown_connections 2023-10-11T06:14:16.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.080+0000 7f5d32592640 1 -- 172.21.15.84:0/614363826 wait complete. 2023-10-11T06:14:16.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:14:16.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.316+0000 7ff915149640 1 Processor -- start 2023-10-11T06:14:16.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.317+0000 7ff915149640 1 -- start start 2023-10-11T06:14:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff915149640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 0x7ff9100d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff915149640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9100d0940 0x7ff9100d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff915149640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9100d3450 con 0x7ff91015f7e0 2023-10-11T06:14:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff915149640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9100cf220 con 0x7ff9100d0940 2023-10-11T06:14:16.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff915149640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100cf500 con 0x7ff9100d2ac0 2023-10-11T06:14:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff90f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9100d0940 0x7ff9100d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff90f59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9100d0940 0x7ff9100d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53352/0 (socket says 172.21.15.84:53352) 2023-10-11T06:14:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.318+0000 7ff90f59e640 1 -- 172.21.15.84:0/1715470722 learned_addr learned my addr 172.21.15.84:0/1715470722 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.319+0000 7ff90f59e640 1 -- 172.21.15.84:0/1715470722 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 msgr2=0x7ff9100d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.319+0000 7ff90f59e640 1 --2- 172.21.15.84:0/1715470722 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 0x7ff9100d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.319+0000 7ff90f59e640 1 -- 172.21.15.84:0/1715470722 >> v1:172.21.15.99:6789/0 conn(0x7ff91015f7e0 legacy=0x7ff9100cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.319+0000 7ff90f59e640 1 -- 172.21.15.84:0/1715470722 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9100cf7e0 con 0x7ff9100d0940 2023-10-11T06:14:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.320+0000 7ff90f59e640 1 --2- 172.21.15.84:0/1715470722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9100d0940 0x7ff9100d0da0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e50 tx=0x7ff90802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30489bc81cb514a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.320+0000 7ff90e59c640 1 -- 172.21.15.84:0/1715470722 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90802ca70 con 0x7ff9100d0940 2023-10-11T06:14:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.320+0000 7ff90e59c640 1 -- 172.21.15.84:0/1715470722 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90802cbd0 con 0x7ff9100d0940 2023-10-11T06:14:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.320+0000 7ff90e59c640 1 -- 172.21.15.84:0/1715470722 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080343f0 con 0x7ff9100d0940 2023-10-11T06:14:16.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 -- 172.21.15.84:0/1715470722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9100d0940 msgr2=0x7ff9100d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 --2- 172.21.15.84:0/1715470722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9100d0940 0x7ff9100d0da0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e50 tx=0x7ff90802bed0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 -- 172.21.15.84:0/1715470722 shutdown_connections 2023-10-11T06:14:16.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 --2- 172.21.15.84:0/1715470722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9100d0940 0x7ff9100d0da0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 --2- 172.21.15.84:0/1715470722 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 0x7ff9100d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 -- 172.21.15.84:0/1715470722 >> 172.21.15.84:0/1715470722 conn(0x7ff9100713a0 msgr2=0x7ff9100717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:16.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 -- 172.21.15.84:0/1715470722 shutdown_connections 2023-10-11T06:14:16.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.321+0000 7ff915149640 1 -- 172.21.15.84:0/1715470722 wait complete. 2023-10-11T06:14:16.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.323+0000 7ff915149640 1 Processor -- start 2023-10-11T06:14:16.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.323+0000 7ff915149640 1 -- start start 2023-10-11T06:14:16.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff915149640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9100d0940 0x7ff9100cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff915149640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 0x7ff91014a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff915149640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91015f7e0 0x7ff910155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff915149640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91015a2f0 con 0x7ff91015f7e0 2023-10-11T06:14:16.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff915149640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91015ae70 con 0x7ff9100d0940 2023-10-11T06:14:16.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff915149640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91015a540 con 0x7ff9100d2ac0 2023-10-11T06:14:16.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff90f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91015f7e0 0x7ff910155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:16.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff90f59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91015f7e0 0x7ff910155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53356/0 (socket says 172.21.15.84:53356) 2023-10-11T06:14:16.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.324+0000 7ff90f59e640 1 -- 172.21.15.84:0/74049030 learned_addr learned my addr 172.21.15.84:0/74049030 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:16.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff90f59e640 1 -- 172.21.15.84:0/74049030 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 msgr2=0x7ff91014a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff90f59e640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 0x7ff91014a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff90f59e640 1 -- 172.21.15.84:0/74049030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9100d0940 msgr2=0x7ff9100cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff90f59e640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9100d0940 0x7ff9100cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff90f59e640 1 -- 172.21.15.84:0/74049030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908009b30 con 0x7ff91015f7e0 2023-10-11T06:14:16.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff90f59e640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91015f7e0 0x7ff910155a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff908036040 tx=0x7ff908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:16.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff9077fe640 1 -- 172.21.15.84:0/74049030 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90803e450 con 0x7ff91015f7e0 2023-10-11T06:14:16.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff9077fe640 1 -- 172.21.15.84:0/74049030 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90803e5d0 con 0x7ff91015f7e0 2023-10-11T06:14:16.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff9077fe640 1 -- 172.21.15.84:0/74049030 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90803e980 con 0x7ff91015f7e0 2023-10-11T06:14:16.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.325+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91015a820 con 0x7ff91015f7e0 2023-10-11T06:14:16.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.326+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91014ae10 con 0x7ff91015f7e0 2023-10-11T06:14:16.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.327+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91015bb30 con 0x7ff91015f7e0 2023-10-11T06:14:16.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.327+0000 7ff9077fe640 1 -- 172.21.15.84:0/74049030 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff90803eae0 con 0x7ff91015f7e0 2023-10-11T06:14:16.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.328+0000 7ff9077fe640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8f006a860 0x7ff8f006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.329+0000 7ff9077fe640 1 -- 172.21.15.84:0/74049030 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9080aba40 con 0x7ff91015f7e0 2023-10-11T06:14:16.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.329+0000 7ff90ed9d640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8f006a860 0x7ff8f006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:16.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.329+0000 7ff90ed9d640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8f006a860 0x7ff8f006cd10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:16.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.333+0000 7ff9077fe640 1 -- 172.21.15.84:0/74049030 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff908076cb0 con 0x7ff91015f7e0 2023-10-11T06:14:16.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.520+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff9100d3450 con 0x7ff91015f7e0 2023-10-11T06:14:16.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.521+0000 7ff9077fe640 1 -- 172.21.15.84:0/74049030 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff9100d3450 con 0x7ff91015f7e0 2023-10-11T06:14:16.526 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:14:16.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.523+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8f006a860 msgr2=0x7ff8f006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.523+0000 7ff915149640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8f006a860 0x7ff8f006cd10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900006bc0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91015f7e0 msgr2=0x7ff910155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91015f7e0 0x7ff910155a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff908036040 tx=0x7ff908004a40 comp rx=0 tx=0).stop 2023-10-11T06:14:16.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 shutdown_connections 2023-10-11T06:14:16.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff8f006a860 0x7ff8f006cd10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91015f7e0 0x7ff910155a50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff9100d2ac0 0x7ff91014a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 --2- 172.21.15.84:0/74049030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9100d0940 0x7ff9100cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 >> 172.21.15.84:0/74049030 conn(0x7ff9100713a0 msgr2=0x7ff91015d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:16.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 shutdown_connections 2023-10-11T06:14:16.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.524+0000 7ff915149640 1 -- 172.21.15.84:0/74049030 wait complete. 2023-10-11T06:14:16.544 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:14:16.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.759+0000 7efe1ece7640 1 Processor -- start 2023-10-11T06:14:16.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.759+0000 7efe1ece7640 1 -- start start 2023-10-11T06:14:16.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.760+0000 7efe1ece7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efe1815f7e0 0x7efe180cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.760+0000 7efe1ece7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180cf220 con 0x7efe180cd6c0 2023-10-11T06:14:16.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ece7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180cf500 con 0x7efe180d31d0 2023-10-11T06:14:16.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ece7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe180cf7e0 con 0x7efe1815f7e0 2023-10-11T06:14:16.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ca83640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efe1815f7e0 0x7efe180cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:16.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ca83640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efe1815f7e0 0x7efe180cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53372/0 (socket says 172.21.15.84:53372) 2023-10-11T06:14:16.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ca83640 1 -- 172.21.15.84:0/3876352004 learned_addr learned my addr 172.21.15.84:0/3876352004 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:16.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ca83640 1 -- 172.21.15.84:0/3876352004 >> v1:172.21.15.99:6789/0 conn(0x7efe180d31d0 legacy=0x7efe180d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:16.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ca83640 1 -- 172.21.15.84:0/3876352004 >> v1:172.21.15.84:6789/0 conn(0x7efe180cd6c0 legacy=0x7efe180d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.761+0000 7efe1ca83640 1 -- 172.21.15.84:0/3876352004 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180cfac0 con 0x7efe1815f7e0 2023-10-11T06:14:16.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.763+0000 7efe1ca83640 1 --2- 172.21.15.84:0/3876352004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1815f7e0 0x7efe180cd180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7efe08009e50 tx=0x7efe0802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a32f58b6674929 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:16.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.763+0000 7efe177fe640 1 -- 172.21.15.84:0/3876352004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0802cb10 con 0x7efe1815f7e0 2023-10-11T06:14:16.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.763+0000 7efe177fe640 1 -- 172.21.15.84:0/3876352004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0802cc70 con 0x7efe1815f7e0 2023-10-11T06:14:16.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.763+0000 7efe177fe640 1 -- 172.21.15.84:0/3876352004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080343f0 con 0x7efe1815f7e0 2023-10-11T06:14:16.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.763+0000 7efe1ece7640 1 -- 172.21.15.84:0/3876352004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1815f7e0 msgr2=0x7efe180cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.763+0000 7efe1ece7640 1 --2- 172.21.15.84:0/3876352004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1815f7e0 0x7efe180cd180 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7efe08009e50 tx=0x7efe0802bf70 comp rx=0 tx=0).stop 2023-10-11T06:14:16.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.764+0000 7efe1ece7640 1 -- 172.21.15.84:0/3876352004 shutdown_connections 2023-10-11T06:14:16.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.764+0000 7efe1ece7640 1 --2- 172.21.15.84:0/3876352004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1815f7e0 0x7efe180cd180 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.764+0000 7efe1ece7640 1 -- 172.21.15.84:0/3876352004 >> 172.21.15.84:0/3876352004 conn(0x7efe180713a0 msgr2=0x7efe180717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:16.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.764+0000 7efe1ece7640 1 -- 172.21.15.84:0/3876352004 shutdown_connections 2023-10-11T06:14:16.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.764+0000 7efe1ece7640 1 -- 172.21.15.84:0/3876352004 wait complete. 2023-10-11T06:14:16.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.765+0000 7efe1ece7640 1 Processor -- start 2023-10-11T06:14:16.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.766+0000 7efe1ece7640 1 -- start start 2023-10-11T06:14:16.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.766+0000 7efe1ece7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe180cd6c0 0x7efe181557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.766+0000 7efe1ece7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe180d31d0 0x7efe180c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ece7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1815f7e0 0x7efe18159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ece7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c6b00 con 0x7efe180cd6c0 2023-10-11T06:14:16.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ece7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180cac10 con 0x7efe1815f7e0 2023-10-11T06:14:16.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ece7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180ca650 con 0x7efe180d31d0 2023-10-11T06:14:16.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe180cd6c0 0x7efe181557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:16.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe180cd6c0 0x7efe181557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53388/0 (socket says 172.21.15.84:53388) 2023-10-11T06:14:16.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 -- 172.21.15.84:0/1410825389 learned_addr learned my addr 172.21.15.84:0/1410825389 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:16.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 -- 172.21.15.84:0/1410825389 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe180d31d0 msgr2=0x7efe180c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe180d31d0 0x7efe180c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 -- 172.21.15.84:0/1410825389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1815f7e0 msgr2=0x7efe18159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1815f7e0 0x7efe18159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 -- 172.21.15.84:0/1410825389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe08009b30 con 0x7efe180cd6c0 2023-10-11T06:14:16.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.767+0000 7efe1ca83640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe180cd6c0 0x7efe181557d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7efe08009e50 tx=0x7efe08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:16.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.768+0000 7efe157fa640 1 -- 172.21.15.84:0/1410825389 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0803e450 con 0x7efe180cd6c0 2023-10-11T06:14:16.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.768+0000 7efe157fa640 1 -- 172.21.15.84:0/1410825389 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0803e5f0 con 0x7efe180cd6c0 2023-10-11T06:14:16.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.768+0000 7efe157fa640 1 -- 172.21.15.84:0/1410825389 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0803e9a0 con 0x7efe180cd6c0 2023-10-11T06:14:16.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.768+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe180ca930 con 0x7efe180cd6c0 2023-10-11T06:14:16.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.768+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe180cb910 con 0x7efe180cd6c0 2023-10-11T06:14:16.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.769+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe180cbaa0 con 0x7efe180cd6c0 2023-10-11T06:14:16.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.770+0000 7efe157fa640 1 -- 172.21.15.84:0/1410825389 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7efe0803eb00 con 0x7efe180cd6c0 2023-10-11T06:14:16.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.771+0000 7efe157fa640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efdf806a830 0x7efdf806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:16.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.771+0000 7efe157fa640 1 -- 172.21.15.84:0/1410825389 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe080abef0 con 0x7efe180cd6c0 2023-10-11T06:14:16.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.771+0000 7efe17fff640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efdf806a830 0x7efdf806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:16.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.772+0000 7efe17fff640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efdf806a830 0x7efdf806cce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe00000f60 tx=0x7efe000079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:16.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.775+0000 7efe157fa640 1 -- 172.21.15.84:0/1410825389 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7efe08077280 con 0x7efe180cd6c0 2023-10-11T06:14:16.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.966+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe180c6970 con 0x7efe180cd6c0 2023-10-11T06:14:16.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.967+0000 7efe157fa640 1 -- 172.21.15.84:0/1410825389 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe0803edb0 con 0x7efe180cd6c0 2023-10-11T06:14:16.969 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:14:16.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.969+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efdf806a830 msgr2=0x7efdf806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.969+0000 7efe1ece7640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efdf806a830 0x7efdf806cce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe00000f60 tx=0x7efe000079b0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.969+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe180cd6c0 msgr2=0x7efe181557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:16.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.969+0000 7efe1ece7640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe180cd6c0 0x7efe181557d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7efe08009e50 tx=0x7efe08004a40 comp rx=0 tx=0).stop 2023-10-11T06:14:16.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 shutdown_connections 2023-10-11T06:14:16.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7efdf806a830 0x7efdf806cce0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1815f7e0 0x7efe18159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe180d31d0 0x7efe180c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 --2- 172.21.15.84:0/1410825389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe180cd6c0 0x7efe181557d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:16.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 >> 172.21.15.84:0/1410825389 conn(0x7efe180713a0 msgr2=0x7efe18071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:16.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 shutdown_connections 2023-10-11T06:14:16.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:16.970+0000 7efe1ece7640 1 -- 172.21.15.84:0/1410825389 wait complete. 2023-10-11T06:14:16.990 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:14:17.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.204+0000 7fb60838e640 1 Processor -- start 2023-10-11T06:14:17.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.205+0000 7fb60838e640 1 -- start start 2023-10-11T06:14:17.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.206+0000 7fb60838e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb60015a0b0 0x7fb6000cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.206+0000 7fb60838e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb6000c8d00 0x7fb6000ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.206+0000 7fb60838e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000cefc0 con 0x7fb6000ceb60 2023-10-11T06:14:17.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.206+0000 7fb60838e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60006e0f0 con 0x7fb6000c8d00 2023-10-11T06:14:17.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.206+0000 7fb60838e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000cbae0 con 0x7fb60015a0b0 2023-10-11T06:14:17.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb605929640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb6000c8d00 0x7fb6000ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb605929640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb6000c8d00 0x7fb6000ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53404/0 (socket says 172.21.15.84:53404) 2023-10-11T06:14:17.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb605929640 1 -- 172.21.15.84:0/4012732460 learned_addr learned my addr 172.21.15.84:0/4012732460 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb60692b640 1 --1- 172.21.15.84:0/4012732460 >> v1:172.21.15.84:6789/0 conn(0x7fb6000ceb60 0x7fb60006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35950/0 (socket says 172.21.15.84:35950) 2023-10-11T06:14:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb60612a640 1 --2- 172.21.15.84:0/4012732460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb60015a0b0 0x7fb6000cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb605929640 1 -- 172.21.15.84:0/4012732460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb60015a0b0 msgr2=0x7fb6000cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb605929640 1 --2- 172.21.15.84:0/4012732460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb60015a0b0 0x7fb6000cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb605929640 1 -- 172.21.15.84:0/4012732460 >> v1:172.21.15.84:6789/0 conn(0x7fb6000ceb60 legacy=0x7fb60006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.207+0000 7fb605929640 1 -- 172.21.15.84:0/4012732460 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb600071380 con 0x7fb6000c8d00 2023-10-11T06:14:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.208+0000 7fb605929640 1 --2- 172.21.15.84:0/4012732460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6000c8d00 0x7fb6000ce620 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00098a0 tx=0x7fb5f002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa2a618e0053d84 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb605128640 1 -- 172.21.15.84:0/4012732460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00317c0 con 0x7fb6000c8d00 2023-10-11T06:14:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb605128640 1 -- 172.21.15.84:0/4012732460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0031920 con 0x7fb6000c8d00 2023-10-11T06:14:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb605128640 1 -- 172.21.15.84:0/4012732460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0031cb0 con 0x7fb6000c8d00 2023-10-11T06:14:17.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 -- 172.21.15.84:0/4012732460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6000c8d00 msgr2=0x7fb6000ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 --2- 172.21.15.84:0/4012732460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6000c8d00 0x7fb6000ce620 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00098a0 tx=0x7fb5f002b860 comp rx=0 tx=0).stop 2023-10-11T06:14:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 -- 172.21.15.84:0/4012732460 shutdown_connections 2023-10-11T06:14:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 --2- 172.21.15.84:0/4012732460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6000c8d00 0x7fb6000ce620 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 --2- 172.21.15.84:0/4012732460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb60015a0b0 0x7fb6000cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 -- 172.21.15.84:0/4012732460 >> 172.21.15.84:0/4012732460 conn(0x7fb600069c70 msgr2=0x7fb60006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 -- 172.21.15.84:0/4012732460 shutdown_connections 2023-10-11T06:14:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.209+0000 7fb60838e640 1 -- 172.21.15.84:0/4012732460 wait complete. 2023-10-11T06:14:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.211+0000 7fb60838e640 1 Processor -- start 2023-10-11T06:14:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.211+0000 7fb60838e640 1 -- start start 2023-10-11T06:14:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60838e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6000c8d00 0x7fb6000b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60838e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6000ceb60 0x7fb6000c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60838e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60015a0b0 0x7fb6001f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60838e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6001f21c0 con 0x7fb60015a0b0 2023-10-11T06:14:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60838e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6001f24a0 con 0x7fb6000c8d00 2023-10-11T06:14:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60838e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000b1780 con 0x7fb6000ceb60 2023-10-11T06:14:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60692b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60015a0b0 0x7fb6001f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60692b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60015a0b0 0x7fb6001f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53418/0 (socket says 172.21.15.84:53418) 2023-10-11T06:14:17.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.212+0000 7fb60692b640 1 -- 172.21.15.84:0/640695298 learned_addr learned my addr 172.21.15.84:0/640695298 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:17.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb60692b640 1 -- 172.21.15.84:0/640695298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6000ceb60 msgr2=0x7fb6000c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb60692b640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6000ceb60 0x7fb6000c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb60692b640 1 -- 172.21.15.84:0/640695298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6000c8d00 msgr2=0x7fb6000b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb60692b640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6000c8d00 0x7fb6000b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb60692b640 1 -- 172.21.15.84:0/640695298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00094d0 con 0x7fb60015a0b0 2023-10-11T06:14:17.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb60692b640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60015a0b0 0x7fb6001f1700 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c780 tx=0x7fb5f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:17.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb5f6ffd640 1 -- 172.21.15.84:0/640695298 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800ce90 con 0x7fb60015a0b0 2023-10-11T06:14:17.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb5f6ffd640 1 -- 172.21.15.84:0/640695298 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f800a640 con 0x7fb60015a0b0 2023-10-11T06:14:17.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb5f6ffd640 1 -- 172.21.15.84:0/640695298 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800a9f0 con 0x7fb60015a0b0 2023-10-11T06:14:17.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.213+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6000b1a60 con 0x7fb60015a0b0 2023-10-11T06:14:17.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.214+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6000c6e80 con 0x7fb60015a0b0 2023-10-11T06:14:17.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.215+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000b15c0 con 0x7fb60015a0b0 2023-10-11T06:14:17.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.216+0000 7fb5f6ffd640 1 -- 172.21.15.84:0/640695298 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb5f800ab50 con 0x7fb60015a0b0 2023-10-11T06:14:17.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.217+0000 7fb5f6ffd640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb5e406a830 0x7fb5e406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.217+0000 7fb5f6ffd640 1 -- 172.21.15.84:0/640695298 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb5f8088790 con 0x7fb60015a0b0 2023-10-11T06:14:17.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.217+0000 7fb60612a640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb5e406a830 0x7fb5e406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.218+0000 7fb60612a640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb5e406a830 0x7fb5e406cce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc006fd0 tx=0x7fb5fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:17.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.221+0000 7fb5f6ffd640 1 -- 172.21.15.84:0/640695298 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb5f8053a20 con 0x7fb60015a0b0 2023-10-11T06:14:17.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.408+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6000b1c50 con 0x7fb60015a0b0 2023-10-11T06:14:17.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.409+0000 7fb5f6ffd640 1 -- 172.21.15.84:0/640695298 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5f8017020 con 0x7fb60015a0b0 2023-10-11T06:14:17.410 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:14:17.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb5e406a830 msgr2=0x7fb5e406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb5e406a830 0x7fb5e406cce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc006fd0 tx=0x7fb5fc008040 comp rx=0 tx=0).stop 2023-10-11T06:14:17.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60015a0b0 msgr2=0x7fb6001f1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60015a0b0 0x7fb6001f1700 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800c780 tx=0x7fb5f800cc40 comp rx=0 tx=0).stop 2023-10-11T06:14:17.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 shutdown_connections 2023-10-11T06:14:17.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb5e406a830 0x7fb5e406cce0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60015a0b0 0x7fb6001f1700 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6000ceb60 0x7fb6000c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 --2- 172.21.15.84:0/640695298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6000c8d00 0x7fb6000b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.411+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 >> 172.21.15.84:0/640695298 conn(0x7fb600069c70 msgr2=0x7fb6001522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:17.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.412+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 shutdown_connections 2023-10-11T06:14:17.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.412+0000 7fb60838e640 1 -- 172.21.15.84:0/640695298 wait complete. 2023-10-11T06:14:17.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:14:17.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.627+0000 7f62794db640 1 Processor -- start 2023-10-11T06:14:17.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.627+0000 7f62794db640 1 -- start start 2023-10-11T06:14:17.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.628+0000 7f62794db640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f62740b4ef0 0x7f62740aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.628+0000 7f62794db640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6274071d80 0x7f6274069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.628+0000 7f62794db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f627406a1b0 0x7f627415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.628+0000 7f62794db640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627415af60 con 0x7f627406a1b0 2023-10-11T06:14:17.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.628+0000 7f62794db640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627406a650 con 0x7f62740b4ef0 2023-10-11T06:14:17.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.628+0000 7f62794db640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62740cb670 con 0x7f6274071d80 2023-10-11T06:14:17.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f627406a1b0 0x7f627415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f627406a1b0 0x7f627415dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53426/0 (socket says 172.21.15.84:53426) 2023-10-11T06:14:17.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 -- 172.21.15.84:0/1958643917 learned_addr learned my addr 172.21.15.84:0/1958643917 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:17.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62727fc640 1 --2- 172.21.15.84:0/1958643917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6274071d80 0x7f6274069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f6272ffd640 1 --2- 172.21.15.84:0/1958643917 >> v2:172.21.15.99:3300/0 conn(0x7f62740b4ef0 0x7f62740aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 -- 172.21.15.84:0/1958643917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6274071d80 msgr2=0x7f6274069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:17.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 --2- 172.21.15.84:0/1958643917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6274071d80 0x7f6274069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 -- 172.21.15.84:0/1958643917 >> v2:172.21.15.99:3300/0 conn(0x7f62740b4ef0 msgr2=0x7f62740aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:17.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 --2- 172.21.15.84:0/1958643917 >> v2:172.21.15.99:3300/0 conn(0x7f62740b4ef0 0x7f62740aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.629+0000 7f62737fe640 1 -- 172.21.15.84:0/1958643917 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62740b52c0 con 0x7f627406a1b0 2023-10-11T06:14:17.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.630+0000 7f62737fe640 1 --2- 172.21.15.84:0/1958643917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f627415dae0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f627406ab30 tx=0x7f626c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f775e1b75a97f583 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:17.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f6271ffb640 1 -- 172.21.15.84:0/1958643917 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c031860 con 0x7f627406a1b0 2023-10-11T06:14:17.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f6271ffb640 1 -- 172.21.15.84:0/1958643917 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c0319c0 con 0x7f627406a1b0 2023-10-11T06:14:17.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f6271ffb640 1 -- 172.21.15.84:0/1958643917 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c031d50 con 0x7f627406a1b0 2023-10-11T06:14:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 -- 172.21.15.84:0/1958643917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 msgr2=0x7f627415dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 --2- 172.21.15.84:0/1958643917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f627415dae0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f627406ab30 tx=0x7f626c02b900 comp rx=0 tx=0).stop 2023-10-11T06:14:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 -- 172.21.15.84:0/1958643917 shutdown_connections 2023-10-11T06:14:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 --2- 172.21.15.84:0/1958643917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f627415dae0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 --2- 172.21.15.84:0/1958643917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6274071d80 0x7f6274069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 --2- 172.21.15.84:0/1958643917 >> v2:172.21.15.99:3300/0 conn(0x7f62740b4ef0 0x7f62740aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 -- 172.21.15.84:0/1958643917 >> 172.21.15.84:0/1958643917 conn(0x7f627406b9b0 msgr2=0x7f627406d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.631+0000 7f62794db640 1 -- 172.21.15.84:0/1958643917 shutdown_connections 2023-10-11T06:14:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.632+0000 7f62794db640 1 -- 172.21.15.84:0/1958643917 wait complete. 2023-10-11T06:14:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.633+0000 7f62794db640 1 Processor -- start 2023-10-11T06:14:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.633+0000 7f62794db640 1 -- start start 2023-10-11T06:14:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.634+0000 7f62794db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f62741580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.634+0000 7f62794db640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6274071d80 0x7f62740ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.634+0000 7f62794db640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62740b4ef0 0x7f62740c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.634+0000 7f62794db640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6274158620 con 0x7f627406a1b0 2023-10-11T06:14:17.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.634+0000 7f62794db640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62740acb40 con 0x7f6274071d80 2023-10-11T06:14:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.634+0000 7f62794db640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62740ace20 con 0x7f62740b4ef0 2023-10-11T06:14:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f62741580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f62741580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53438/0 (socket says 172.21.15.84:53438) 2023-10-11T06:14:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 -- 172.21.15.84:0/889216493 learned_addr learned my addr 172.21.15.84:0/889216493 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:17.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f62727fc640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6274071d80 0x7f62740ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f62737fe640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62740b4ef0 0x7f62740c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 -- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62740b4ef0 msgr2=0x7f62740c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62740b4ef0 0x7f62740c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 -- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6274071d80 msgr2=0x7f62740ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6274071d80 0x7f62740ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 -- 172.21.15.84:0/889216493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c0094d0 con 0x7f627406a1b0 2023-10-11T06:14:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.635+0000 7f6272ffd640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f62741580e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6264007e40 tx=0x7f626400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.636+0000 7f626b7fe640 1 -- 172.21.15.84:0/889216493 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264014620 con 0x7f627406a1b0 2023-10-11T06:14:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.636+0000 7f626b7fe640 1 -- 172.21.15.84:0/889216493 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626400ee90 con 0x7f627406a1b0 2023-10-11T06:14:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.636+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62740ad100 con 0x7f627406a1b0 2023-10-11T06:14:17.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.636+0000 7f626b7fe640 1 -- 172.21.15.84:0/889216493 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640149c0 con 0x7f627406a1b0 2023-10-11T06:14:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.636+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62740c44b0 con 0x7f627406a1b0 2023-10-11T06:14:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.637+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62740c4310 con 0x7f627406a1b0 2023-10-11T06:14:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.638+0000 7f626b7fe640 1 -- 172.21.15.84:0/889216493 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6264014b20 con 0x7f627406a1b0 2023-10-11T06:14:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.639+0000 7f626b7fe640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f625406a830 0x7f625406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:17.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.639+0000 7f626b7fe640 1 -- 172.21.15.84:0/889216493 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62640899e0 con 0x7f627406a1b0 2023-10-11T06:14:17.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.639+0000 7f62727fc640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f625406a830 0x7f625406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:17.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.640+0000 7f62727fc640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f625406a830 0x7f625406cce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62740b4900 tx=0x7f625c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:17.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.643+0000 7f626b7fe640 1 -- 172.21.15.84:0/889216493 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f62740c4310 con 0x7f627406a1b0 2023-10-11T06:14:17.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.843+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f62740c4310 con 0x7f627406a1b0 2023-10-11T06:14:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.843+0000 7f626b7fe640 1 -- 172.21.15.84:0/889216493 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f62740c4310 con 0x7f627406a1b0 2023-10-11T06:14:17.845 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:14:17.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f625406a830 msgr2=0x7f625406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f625406a830 0x7f625406cce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62740b4900 tx=0x7f625c008040 comp rx=0 tx=0).stop 2023-10-11T06:14:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 msgr2=0x7f62741580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f62741580e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6264007e40 tx=0x7f626400e4b0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 shutdown_connections 2023-10-11T06:14:17.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f625406a830 0x7f625406cce0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62740b4ef0 0x7f62740c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6274071d80 0x7f62740ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 --2- 172.21.15.84:0/889216493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f627406a1b0 0x7f62741580e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 >> 172.21.15.84:0/889216493 conn(0x7f627406b9b0 msgr2=0x7f62741599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:17.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 shutdown_connections 2023-10-11T06:14:17.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:17.846+0000 7f62794db640 1 -- 172.21.15.84:0/889216493 wait complete. 2023-10-11T06:14:17.866 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:14:17.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:17.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:17.963 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[108837]: fuse finished with error 0 and tester_r 0 2023-10-11T06:14:17.967 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:14:24.069 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:14:24.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:24.070 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:14:24.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-11T06:14:24.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.338+0000 7ff262eb6640 1 Processor -- start 2023-10-11T06:14:24.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.338+0000 7ff262eb6640 1 -- start start 2023-10-11T06:14:24.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff262eb6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0cafb0 0x7ff25c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:24.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff262eb6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff25c0cc410 0x7ff25c0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:24.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff262eb6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c0cb970 con 0x7ff25c15c290 2023-10-11T06:14:24.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff262eb6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0cad70 con 0x7ff25c0cc410 2023-10-11T06:14:24.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff262eb6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c15fe40 con 0x7ff25c0cafb0 2023-10-11T06:14:24.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff261453640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff25c0cc410 0x7ff25c0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:24.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff261453640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff25c0cc410 0x7ff25c0cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59386/0 (socket says 172.21.15.84:59386) 2023-10-11T06:14:24.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff261453640 1 -- 172.21.15.84:0/1730215472 learned_addr learned my addr 172.21.15.84:0/1730215472 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:24.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.340+0000 7ff25bfff640 1 --2- 172.21.15.84:0/1730215472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0cafb0 0x7ff25c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:24.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.341+0000 7ff261453640 1 -- 172.21.15.84:0/1730215472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0cafb0 msgr2=0x7ff25c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:24.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.341+0000 7ff261453640 1 --2- 172.21.15.84:0/1730215472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0cafb0 0x7ff25c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.341+0000 7ff261453640 1 -- 172.21.15.84:0/1730215472 >> v1:172.21.15.99:6789/0 conn(0x7ff25c15c290 legacy=0x7ff25c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:24.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.341+0000 7ff261453640 1 -- 172.21.15.84:0/1730215472 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c06ff30 con 0x7ff25c0cc410 2023-10-11T06:14:24.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.342+0000 7ff261453640 1 --2- 172.21.15.84:0/1730215472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0cc870 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff254009e50 tx=0x7ff25402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acb7cef5fc108de server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:24.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.342+0000 7ff25b7fe640 1 -- 172.21.15.84:0/1730215472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25402ca80 con 0x7ff25c0cc410 2023-10-11T06:14:24.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.342+0000 7ff25b7fe640 1 -- 172.21.15.84:0/1730215472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25402cbe0 con 0x7ff25c0cc410 2023-10-11T06:14:24.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.342+0000 7ff25b7fe640 1 -- 172.21.15.84:0/1730215472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2540343f0 con 0x7ff25c0cc410 2023-10-11T06:14:24.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 -- 172.21.15.84:0/1730215472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 msgr2=0x7ff25c0cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:24.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 --2- 172.21.15.84:0/1730215472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0cc870 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff254009e50 tx=0x7ff25402bee0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 -- 172.21.15.84:0/1730215472 shutdown_connections 2023-10-11T06:14:24.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 --2- 172.21.15.84:0/1730215472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0cc870 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 --2- 172.21.15.84:0/1730215472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0cafb0 0x7ff25c0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 -- 172.21.15.84:0/1730215472 >> 172.21.15.84:0/1730215472 conn(0x7ff25c069c70 msgr2=0x7ff25c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:24.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 -- 172.21.15.84:0/1730215472 shutdown_connections 2023-10-11T06:14:24.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.343+0000 7ff262eb6640 1 -- 172.21.15.84:0/1730215472 wait complete. 2023-10-11T06:14:24.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.345+0000 7ff262eb6640 1 Processor -- start 2023-10-11T06:14:24.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.345+0000 7ff262eb6640 1 -- start start 2023-10-11T06:14:24.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff262eb6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cafb0 0x7ff25c149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:24.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff262eb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:24.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff262eb6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c15c290 0x7ff25c15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:24.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff262eb6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c14a000 con 0x7ff25c0cc410 2023-10-11T06:14:24.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff262eb6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c15e7e0 con 0x7ff25c0cafb0 2023-10-11T06:14:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff262eb6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c15e1d0 con 0x7ff25c15c290 2023-10-11T06:14:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff25bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59398/0 (socket says 172.21.15.84:59398) 2023-10-11T06:14:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff25bfff640 1 -- 172.21.15.84:0/3503272263 learned_addr learned my addr 172.21.15.84:0/3503272263 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff261453640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c15c290 0x7ff25c15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff260c52640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cafb0 0x7ff25c149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff25bfff640 1 -- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c15c290 msgr2=0x7ff25c15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff25bfff640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c15c290 0x7ff25c15ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff25bfff640 1 -- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cafb0 msgr2=0x7ff25c149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.346+0000 7ff25bfff640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cafb0 0x7ff25c149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.347+0000 7ff25bfff640 1 -- 172.21.15.84:0/3503272263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff254009b30 con 0x7ff25c0cc410 2023-10-11T06:14:24.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.347+0000 7ff25bfff640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0b1430 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d620 tx=0x7ff24800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:24.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.347+0000 7ff2597fa640 1 -- 172.21.15.84:0/3503272263 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24800b620 con 0x7ff25c0cc410 2023-10-11T06:14:24.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.347+0000 7ff2597fa640 1 -- 172.21.15.84:0/3503272263 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24800b780 con 0x7ff25c0cc410 2023-10-11T06:14:24.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.347+0000 7ff2597fa640 1 -- 172.21.15.84:0/3503272263 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24800bb30 con 0x7ff25c0cc410 2023-10-11T06:14:24.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.347+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c0c2150 con 0x7ff25c0cc410 2023-10-11T06:14:24.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.347+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c15e470 con 0x7ff25c0cc410 2023-10-11T06:14:24.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.348+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c0c9980 con 0x7ff25c0cc410 2023-10-11T06:14:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.349+0000 7ff2597fa640 1 -- 172.21.15.84:0/3503272263 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff24800bc90 con 0x7ff25c0cc410 2023-10-11T06:14:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.350+0000 7ff2597fa640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff24006a830 0x7ff24006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.350+0000 7ff2597fa640 1 -- 172.21.15.84:0/3503272263 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2480894e0 con 0x7ff25c0cc410 2023-10-11T06:14:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.350+0000 7ff260c52640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff24006a830 0x7ff24006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.351+0000 7ff260c52640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff24006a830 0x7ff24006cce0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff250003e00 tx=0x7ff250009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.355+0000 7ff2597fa640 1 -- 172.21.15.84:0/3503272263 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff248054750 con 0x7ff25c0cc410 2023-10-11T06:14:24.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.584+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff25c073ed0 con 0x7ff25c0cc410 2023-10-11T06:14:24.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.585+0000 7ff2597fa640 1 -- 172.21.15.84:0/3503272263 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff248057d70 con 0x7ff25c0cc410 2023-10-11T06:14:24.586 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-11T06:14:24.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff24006a830 msgr2=0x7ff24006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:24.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff24006a830 0x7ff24006cce0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff250003e00 tx=0x7ff250009040 comp rx=0 tx=0).stop 2023-10-11T06:14:24.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 msgr2=0x7ff25c0b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:24.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0b1430 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d620 tx=0x7ff24800dae0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 shutdown_connections 2023-10-11T06:14:24.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff24006a830 0x7ff24006cce0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c15c290 0x7ff25c15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c0cc410 0x7ff25c0b1430 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 --2- 172.21.15.84:0/3503272263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cafb0 0x7ff25c149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:24.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 >> 172.21.15.84:0/3503272263 conn(0x7ff25c069c70 msgr2=0x7ff25c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:24.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 shutdown_connections 2023-10-11T06:14:24.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:24.588+0000 7ff262eb6640 1 -- 172.21.15.84:0/3503272263 wait complete. 2023-10-11T06:14:24.606 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-11T06:14:24.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-11T06:14:24.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-10-11T06:14:24.607 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-11T06:14:24.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:24.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-11T06:14:24.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-11T06:14:24.609 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-11T06:14:24.609 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:14:24.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:24.610 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-10-11T06:14:24.665 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-11T06:14:24.665 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-11T06:14:24.665 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-10-11T06:14:24.665 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:24.665 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-10-11T06:14:24.666 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:24.666 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:14:24.666 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:24.666 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-10-11T06:14:24.666 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:14:24.667 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:24.667 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-10-11T06:14:24.667 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-11T06:14:24.667 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:24.667 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-11T06:14:24.667 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:24.668 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-11T06:14:24.668 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42212sec preferred_lft 42212sec 2023-10-11T06:14:24.668 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2023-10-11T06:14:24.668 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:24.668 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-11T06:14:24.668 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff 2023-10-11T06:14:24.669 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-11T06:14:24.669 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:24.669 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::1c33:d5ff:fea1:21f9/64 scope link 2023-10-11T06:14:24.669 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:24.669 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:14:24.669 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether d6:ff:bc:ac:c7:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:14:24.670 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::d4ff:bcff:feac:c798/64 scope link 2023-10-11T06:14:24.670 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:24.670 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-11T06:14:24.670 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 42:88:59:b7:53:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:14:24.670 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4088:59ff:feb7:538e/64 scope link 2023-10-11T06:14:24.671 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-10-11T06:14:24.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:24.671 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-11T06:14:24.762 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-10-11T06:14:24.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:24.767 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-10-11T06:14:24.823 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-10-11T06:14:24.823 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-11T06:14:24.824 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-11T06:14:24.824 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-11T06:14:24.824 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-11T06:14:24.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:24.825 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-10-11T06:14:24.825 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-10-11T06:14:24.825 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-11T06:14:24.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-11T06:14:24.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-11T06:14:24.826 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-10-11T06:14:25.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:25.013 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:14:25.063 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T06:14:25.063 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:14:25.064 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:14:25.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:25.064 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-10-11T06:14:25.161 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-11T06:14:25.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:14:25.162 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:25.214 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-11T06:14:25.214 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-11T06:14:25.215 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:25.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:14:25.355 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:14:25.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:14:25.448 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:14:25.452 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:14:25.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:25.453 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:14:25.508 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-11T06:14:25.508 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-11T06:14:25.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:14:25.591 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:14:25.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:14:25.662 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:14:25.664 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:14:25.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:25.665 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:14:25.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-10-11T06:14:25.702+0000 7f302a066480 -1 init, newargv = 0x55e9bebc6840 newargc=15 2023-10-11T06:14:25.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[109886]: starting ceph client 2023-10-11T06:14:25.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[109886]: starting fuse 2023-10-11T06:14:26.812 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-10-11T06:14:26.917 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-11T06:14:26.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-11T06:14:27.012 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-11T06:14:27.016 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-11T06:14:27.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:27.017 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-10-11T06:14:27.073 INFO:teuthology.orchestra.run.smithi084.stdout:40 2023-10-11T06:14:27.074 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-10-11T06:14:27.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:14:27.075 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-10-11T06:14:27.075 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-10-11T06:14:27.075 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:14:27.076 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-10-11T06:14:27.076 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:27.077 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-10-11T06:14:27.077 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-11T06:14:27.077 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-10-11T06:14:27.078 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-11T06:14:27.078 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:27.078 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-10-11T06:14:27.079 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-10-11T06:14:27.079 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-10-11T06:14:27.079 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-10-11T06:14:27.080 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:27.080 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-10-11T06:14:27.080 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-11T06:14:27.081 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-11T06:14:27.081 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-11T06:14:27.081 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-10-11T06:14:27.082 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-10-11T06:14:27.082 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-10-11T06:14:27.082 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-11T06:14:27.083 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:27.083 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-10-11T06:14:27.083 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:14:27.266 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.109886.asok 2023-10-11T06:14:27.411 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:14:27.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:14:27.444 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109886.asok 2023-10-11T06:14:27.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:27.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109886.asok status 2023-10-11T06:14:27.670 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-10-11T06:14:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "9a774796f0ecdd11a4b771de396e627aa1a187af", 2023-10-11T06:14:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.0-662-g9a774796 (9a774796f0ecdd11a4b771de396e627aa1a187af) reef (stable)", 2023-10-11T06:14:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-10-11T06:14:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-10-11T06:14:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-11T06:14:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "109886", 2023-10-11T06:14:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-10-11T06:14:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-10-11T06:14:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-10-11T06:14:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6405, 2023-10-11T06:14:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-10-11T06:14:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-10-11T06:14:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-10-11T06:14:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 6405 2023-10-11T06:14:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:14:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:14:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:14:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3033747593 2023-10-11T06:14:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-10-11T06:14:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-10-11T06:14:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-10-11T06:14:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-10-11T06:14:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3033747593 2023-10-11T06:14:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-10-11T06:14:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.6405 192.168.0.1:0/3033747593", 2023-10-11T06:14:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/3033747593", 2023-10-11T06:14:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-10-11T06:14:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 24, 2023-10-11T06:14:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 54, 2023-10-11T06:14:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-10-11T06:14:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-10-11T06:14:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-10-11T06:14:27.677 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:27.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:27.685 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-10-11T06:14:27.685 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:27.739 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-10-11T06:14:27.740 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:27.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:27.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:27.832 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-11T06:14:27.833 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-10-11T06:14:27.833 DEBUG:teuthology.orchestra.run.smithi084:> import errno 2023-10-11T06:14:27.833 DEBUG:teuthology.orchestra.run.smithi084:> 2023-10-11T06:14:27.833 DEBUG:teuthology.orchestra.run.smithi084:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-10-11T06:14:27.834 DEBUG:teuthology.orchestra.run.smithi084:> try: 2023-10-11T06:14:27.834 DEBUG:teuthology.orchestra.run.smithi084:> if True: 2023-10-11T06:14:27.834 DEBUG:teuthology.orchestra.run.smithi084:> ret = os.read(fd, 1024) 2023-10-11T06:14:27.835 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-10-11T06:14:27.835 DEBUG:teuthology.orchestra.run.smithi084:> os.write(fd, b'"'"'content'"'"') 2023-10-11T06:14:27.835 DEBUG:teuthology.orchestra.run.smithi084:> except OSError as e: 2023-10-11T06:14:27.836 DEBUG:teuthology.orchestra.run.smithi084:> if e.errno != errno.EPERM: 2023-10-11T06:14:27.836 DEBUG:teuthology.orchestra.run.smithi084:> raise 2023-10-11T06:14:27.836 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-10-11T06:14:27.836 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("client does not check permission of data pool") 2023-10-11T06:14:27.837 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-10-11T06:14:27.983 INFO:teuthology.orchestra.run.smithi084.stderr:Traceback (most recent call last): 2023-10-11T06:14:27.983 INFO:teuthology.orchestra.run.smithi084.stderr: File "", line 15, in 2023-10-11T06:14:27.983 INFO:teuthology.orchestra.run.smithi084.stderr:RuntimeError: client does not check permission of data pool 2023-10-11T06:14:28.150 INFO:teuthology.orchestra.run.smithi084.stderr:daemon-helper: command failed with exit status 1 2023-10-11T06:14:28.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:14:28.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-10-11T06:14:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.431+0000 7f3a81fc7640 1 Processor -- start 2023-10-11T06:14:28.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.431+0000 7f3a81fc7640 1 -- start start 2023-10-11T06:14:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a81fc7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3a7c15a050 0x7f3a7c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a81fc7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 0x7f3a7c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a81fc7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c0cc5c0 con 0x7f3a7c0cca50 2023-10-11T06:14:28.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a81fc7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7c0d05f0 con 0x7f3a7c15a050 2023-10-11T06:14:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a81fc7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0cb9b0 con 0x7f3a7c0cb4e0 2023-10-11T06:14:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a7b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3a7c15a050 0x7f3a7c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a7b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3a7c15a050 0x7f3a7c0ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59410/0 (socket says 172.21.15.84:59410) 2023-10-11T06:14:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.432+0000 7f3a7b7fe640 1 -- 172.21.15.84:0/3298405911 learned_addr learned my addr 172.21.15.84:0/3298405911 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.433+0000 7f3a7b7fe640 1 -- 172.21.15.84:0/3298405911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 msgr2=0x7f3a7c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:28.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.433+0000 7f3a7b7fe640 1 --2- 172.21.15.84:0/3298405911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 0x7f3a7c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.433+0000 7f3a7b7fe640 1 -- 172.21.15.84:0/3298405911 >> v1:172.21.15.99:6789/0 conn(0x7f3a7c0cca50 legacy=0x7f3a7c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:28.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.433+0000 7f3a7b7fe640 1 -- 172.21.15.84:0/3298405911 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0cbc90 con 0x7f3a7c15a050 2023-10-11T06:14:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.434+0000 7f3a7b7fe640 1 --2- 172.21.15.84:0/3298405911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ce350 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009860 tx=0x7f3a6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef273b55b4b7aea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.434+0000 7f3a7a7fc640 1 -- 172.21.15.84:0/3298405911 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c031640 con 0x7f3a7c15a050 2023-10-11T06:14:28.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.434+0000 7f3a7a7fc640 1 -- 172.21.15.84:0/3298405911 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c0317a0 con 0x7f3a7c15a050 2023-10-11T06:14:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.434+0000 7f3a7a7fc640 1 -- 172.21.15.84:0/3298405911 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c031b30 con 0x7f3a7c15a050 2023-10-11T06:14:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3298405911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 msgr2=0x7f3a7c0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3298405911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ce350 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009860 tx=0x7f3a6c02b8f0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3298405911 shutdown_connections 2023-10-11T06:14:28.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3298405911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 0x7f3a7c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3298405911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ce350 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3298405911 >> 172.21.15.84:0/3298405911 conn(0x7f3a7c06a590 msgr2=0x7f3a7c099ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:28.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3298405911 shutdown_connections 2023-10-11T06:14:28.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.435+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3298405911 wait complete. 2023-10-11T06:14:28.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.437+0000 7f3a81fc7640 1 Processor -- start 2023-10-11T06:14:28.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.437+0000 7f3a81fc7640 1 -- start start 2023-10-11T06:14:28.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a81fc7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 0x7f3a7c158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a81fc7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c0cca50 0x7f3a7c0ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a81fc7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a81fc7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0b59d0 con 0x7f3a7c15a050 2023-10-11T06:14:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a81fc7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c09e290 con 0x7f3a7c0cca50 2023-10-11T06:14:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a81fc7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c7980 con 0x7f3a7c0cb4e0 2023-10-11T06:14:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a7bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a7bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59416/0 (socket says 172.21.15.84:59416) 2023-10-11T06:14:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.438+0000 7f3a7bfff640 1 -- 172.21.15.84:0/3595386815 learned_addr learned my addr 172.21.15.84:0/3595386815 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7b7fe640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 0x7f3a7c158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7affd640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c0cca50 0x7f3a7c0ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7bfff640 1 -- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 msgr2=0x7f3a7c158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:28.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7bfff640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 0x7f3a7c158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7bfff640 1 -- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c0cca50 msgr2=0x7f3a7c0ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:28.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7bfff640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c0cca50 0x7f3a7c0ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7bfff640 1 -- 172.21.15.84:0/3595386815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c009470 con 0x7f3a7c15a050 2023-10-11T06:14:28.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a7bfff640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ae970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400b3f0 tx=0x7f3a7400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:28.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.439+0000 7f3a73fff640 1 -- 172.21.15.84:0/3595386815 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74011830 con 0x7f3a7c15a050 2023-10-11T06:14:28.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.440+0000 7f3a73fff640 1 -- 172.21.15.84:0/3595386815 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a74011990 con 0x7f3a7c15a050 2023-10-11T06:14:28.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.440+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c0c7c60 con 0x7f3a7c15a050 2023-10-11T06:14:28.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.440+0000 7f3a73fff640 1 -- 172.21.15.84:0/3595386815 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74011d40 con 0x7f3a7c15a050 2023-10-11T06:14:28.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.440+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c0c8220 con 0x7f3a7c15a050 2023-10-11T06:14:28.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.441+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c0c8440 con 0x7f3a7c15a050 2023-10-11T06:14:28.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.442+0000 7f3a73fff640 1 -- 172.21.15.84:0/3595386815 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f3a7401a400 con 0x7f3a7c15a050 2023-10-11T06:14:28.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.442+0000 7f3a73fff640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a5c06a830 0x7f3a5c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:28.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.443+0000 7f3a73fff640 1 -- 172.21.15.84:0/3595386815 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a74089110 con 0x7f3a7c15a050 2023-10-11T06:14:28.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.443+0000 7f3a7b7fe640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a5c06a830 0x7f3a5c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:28.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.443+0000 7f3a7b7fe640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a5c06a830 0x7f3a5c06cce0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c036000 tx=0x7f3a6c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:28.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.447+0000 7f3a73fff640 1 -- 172.21.15.84:0/3595386815 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f3a74054380 con 0x7f3a7c15a050 2023-10-11T06:14:28.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.704+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3a7c067090 con 0x7f3a7c15a050 2023-10-11T06:14:28.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.710+0000 7f3a73fff640 1 -- 172.21.15.84:0/3595386815 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3a740579a0 con 0x7f3a7c15a050 2023-10-11T06:14:28.711 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-10-11T06:14:28.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a5c06a830 msgr2=0x7f3a5c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:28.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a5c06a830 0x7f3a5c06cce0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c036000 tx=0x7f3a6c03c040 comp rx=0 tx=0).stop 2023-10-11T06:14:28.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 msgr2=0x7f3a7c0ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:28.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ae970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400b3f0 tx=0x7f3a7400b8b0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 shutdown_connections 2023-10-11T06:14:28.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f3a5c06a830 0x7f3a5c06cce0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a7c15a050 0x7f3a7c0ae970 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c0cca50 0x7f3a7c0ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 --2- 172.21.15.84:0/3595386815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a7c0cb4e0 0x7f3a7c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:28.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 >> 172.21.15.84:0/3595386815 conn(0x7f3a7c06a590 msgr2=0x7f3a7c099ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:28.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 shutdown_connections 2023-10-11T06:14:28.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:28.713+0000 7f3a81fc7640 1 -- 172.21.15.84:0/3595386815 wait complete. 2023-10-11T06:14:28.732 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-11T06:14:28.732 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:14:28.837 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-11T06:14:28.956 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-11T06:14:28.957 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:14:28.957 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:29.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.235+0000 7fc68d167640 1 Processor -- start 2023-10-11T06:14:29.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.235+0000 7fc68d167640 1 -- start start 2023-10-11T06:14:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.236+0000 7fc68d167640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880ce0d0 0x7fc6880c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc68d167640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6880cd300 0x7fc68815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc68d167640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6881593f0 con 0x7fc6880adb90 2023-10-11T06:14:29.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc68d167640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68815a340 con 0x7fc6880cd300 2023-10-11T06:14:29.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc68d167640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6880ce4a0 con 0x7fc6880ce0d0 2023-10-11T06:14:29.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc686d9d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880ce0d0 0x7fc6880c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc68759e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6880cd300 0x7fc68815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc686d9d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880ce0d0 0x7fc6880c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:46896/0 (socket says 172.21.15.84:46896) 2023-10-11T06:14:29.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.237+0000 7fc686d9d640 1 -- 172.21.15.84:0/4229768457 learned_addr learned my addr 172.21.15.84:0/4229768457 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:29.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.238+0000 7fc686d9d640 1 -- 172.21.15.84:0/4229768457 >> v2:172.21.15.99:3300/0 conn(0x7fc6880cd300 msgr2=0x7fc68815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:29.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.238+0000 7fc686d9d640 1 --2- 172.21.15.84:0/4229768457 >> v2:172.21.15.99:3300/0 conn(0x7fc6880cd300 0x7fc68815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.238+0000 7fc686d9d640 1 -- 172.21.15.84:0/4229768457 >> v1:172.21.15.99:6789/0 conn(0x7fc6880adb90 legacy=0x7fc6880caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.238+0000 7fc686d9d640 1 -- 172.21.15.84:0/4229768457 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6881596c0 con 0x7fc6880ce0d0 2023-10-11T06:14:29.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.239+0000 7fc686d9d640 1 --2- 172.21.15.84:0/4229768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880ce0d0 0x7fc6880c8cc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc67800baf0 tx=0x7fc67802e130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4325d80fc681ae74 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:29.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.240+0000 7fc685d9b640 1 -- 172.21.15.84:0/4229768457 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678004030 con 0x7fc6880ce0d0 2023-10-11T06:14:29.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.240+0000 7fc685d9b640 1 -- 172.21.15.84:0/4229768457 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678004190 con 0x7fc6880ce0d0 2023-10-11T06:14:29.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.240+0000 7fc685d9b640 1 -- 172.21.15.84:0/4229768457 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678003610 con 0x7fc6880ce0d0 2023-10-11T06:14:29.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 -- 172.21.15.84:0/4229768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880ce0d0 msgr2=0x7fc6880c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 --2- 172.21.15.84:0/4229768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880ce0d0 0x7fc6880c8cc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc67800baf0 tx=0x7fc67802e130 comp rx=0 tx=0).stop 2023-10-11T06:14:29.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 -- 172.21.15.84:0/4229768457 shutdown_connections 2023-10-11T06:14:29.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 --2- 172.21.15.84:0/4229768457 >> v2:172.21.15.99:3300/0 conn(0x7fc6880cd300 0x7fc68815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 --2- 172.21.15.84:0/4229768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880ce0d0 0x7fc6880c8cc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 -- 172.21.15.84:0/4229768457 >> 172.21.15.84:0/4229768457 conn(0x7fc688069c70 msgr2=0x7fc68806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:29.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 -- 172.21.15.84:0/4229768457 shutdown_connections 2023-10-11T06:14:29.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.241+0000 7fc68d167640 1 -- 172.21.15.84:0/4229768457 wait complete. 2023-10-11T06:14:29.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.243+0000 7fc68d167640 1 Processor -- start 2023-10-11T06:14:29.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.243+0000 7fc68d167640 1 -- start start 2023-10-11T06:14:29.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.244+0000 7fc68d167640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6880adb90 0x7fc6881470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.244+0000 7fc68d167640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880cd300 0x7fc6881454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.244+0000 7fc68d167640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6880ce0d0 0x7fc6880b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.244+0000 7fc68d167640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6880ae850 con 0x7fc6880ce0d0 2023-10-11T06:14:29.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.244+0000 7fc68d167640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6880aea60 con 0x7fc6880adb90 2023-10-11T06:14:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.244+0000 7fc68d167640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6880aed40 con 0x7fc6880cd300 2023-10-11T06:14:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6880ce0d0 0x7fc6880b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6880ce0d0 0x7fc6880b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57272/0 (socket says 172.21.15.84:57272) 2023-10-11T06:14:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 -- 172.21.15.84:0/3639146587 learned_addr learned my addr 172.21.15.84:0/3639146587 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:29.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc686d9d640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6880adb90 0x7fc6881470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68659c640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880cd300 0x7fc6881454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 -- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880cd300 msgr2=0x7fc6881454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880cd300 0x7fc6881454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 -- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6880adb90 msgr2=0x7fc6881470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6880adb90 0x7fc6881470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.245+0000 7fc68759e640 1 -- 172.21.15.84:0/3639146587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67800b700 con 0x7fc6880ce0d0 2023-10-11T06:14:29.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.246+0000 7fc68759e640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6880ce0d0 0x7fc6880b2300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc68000efc0 tx=0x7fc68000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:29.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.246+0000 7fc66f7fe640 1 -- 172.21.15.84:0/3639146587 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680014480 con 0x7fc6880ce0d0 2023-10-11T06:14:29.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.246+0000 7fc66f7fe640 1 -- 172.21.15.84:0/3639146587 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68000f040 con 0x7fc6880ce0d0 2023-10-11T06:14:29.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.246+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6880af020 con 0x7fc6880ce0d0 2023-10-11T06:14:29.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.246+0000 7fc66f7fe640 1 -- 172.21.15.84:0/3639146587 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680014820 con 0x7fc6880ce0d0 2023-10-11T06:14:29.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.246+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc688145a00 con 0x7fc6880ce0d0 2023-10-11T06:14:29.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.247+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6880af540 con 0x7fc6880ce0d0 2023-10-11T06:14:29.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.248+0000 7fc66f7fe640 1 -- 172.21.15.84:0/3639146587 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc680014980 con 0x7fc6880ce0d0 2023-10-11T06:14:29.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.250+0000 7fc66f7fe640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc67406a830 0x7fc67406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.250+0000 7fc66f7fe640 1 -- 172.21.15.84:0/3639146587 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc680089db0 con 0x7fc6880ce0d0 2023-10-11T06:14:29.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.250+0000 7fc686d9d640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc67406a830 0x7fc67406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.251+0000 7fc686d9d640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc67406a830 0x7fc67406cce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc678037000 tx=0x7fc678032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:29.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.254+0000 7fc66f7fe640 1 -- 172.21.15.84:0/3639146587 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc6880af540 con 0x7fc6880ce0d0 2023-10-11T06:14:29.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.443+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6880af540 con 0x7fc6880ce0d0 2023-10-11T06:14:29.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.446+0000 7fc66f7fe640 1 -- 172.21.15.84:0/3639146587 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7fc6880af540 con 0x7fc6880ce0d0 2023-10-11T06:14:29.448 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:29.448 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:29.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.448+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc67406a830 msgr2=0x7fc67406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.448+0000 7fc68d167640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc67406a830 0x7fc67406cce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc678037000 tx=0x7fc678032000 comp rx=0 tx=0).stop 2023-10-11T06:14:29.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.448+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6880ce0d0 msgr2=0x7fc6880b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.448+0000 7fc68d167640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6880ce0d0 0x7fc6880b2300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc68000efc0 tx=0x7fc68000c490 comp rx=0 tx=0).stop 2023-10-11T06:14:29.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 shutdown_connections 2023-10-11T06:14:29.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc67406a830 0x7fc67406cce0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6880ce0d0 0x7fc6880b2300 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6880cd300 0x7fc6881454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 --2- 172.21.15.84:0/3639146587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6880adb90 0x7fc6881470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 >> 172.21.15.84:0/3639146587 conn(0x7fc688069c70 msgr2=0x7fc688157a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:29.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 shutdown_connections 2023-10-11T06:14:29.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.449+0000 7fc68d167640 1 -- 172.21.15.84:0/3639146587 wait complete. 2023-10-11T06:14:29.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:13:44.841435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:13:45.401366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:14:29.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:14:29.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.670+0000 7fb4ab72e640 1 Processor -- start 2023-10-11T06:14:29.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.670+0000 7fb4ab72e640 1 -- start start 2023-10-11T06:14:29.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.672+0000 7fb4ab72e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4a414ff30 0x7fb4a40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.672+0000 7fb4ab72e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a4150300 0x7fb4a40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.672+0000 7fb4ab72e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a40c9b90 con 0x7fb4a40cc7c0 2023-10-11T06:14:29.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.672+0000 7fb4ab72e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a415da10 con 0x7fb4a414ff30 2023-10-11T06:14:29.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.672+0000 7fb4ab72e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a415ce00 con 0x7fb4a4150300 2023-10-11T06:14:29.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a8cc9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a4150300 0x7fb4a40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a94ca640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4a414ff30 0x7fb4a40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a8cc9640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a4150300 0x7fb4a40cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:46914/0 (socket says 172.21.15.84:46914) 2023-10-11T06:14:29.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a8cc9640 1 -- 172.21.15.84:0/936279791 learned_addr learned my addr 172.21.15.84:0/936279791 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:29.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a8cc9640 1 -- 172.21.15.84:0/936279791 >> v2:172.21.15.99:3300/0 conn(0x7fb4a414ff30 msgr2=0x7fb4a40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:29.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a8cc9640 1 --2- 172.21.15.84:0/936279791 >> v2:172.21.15.99:3300/0 conn(0x7fb4a414ff30 0x7fb4a40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a8cc9640 1 -- 172.21.15.84:0/936279791 >> v1:172.21.15.99:6789/0 conn(0x7fb4a40cc7c0 legacy=0x7fb4a40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.673+0000 7fb4a8cc9640 1 -- 172.21.15.84:0/936279791 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a415d0e0 con 0x7fb4a4150300 2023-10-11T06:14:29.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.675+0000 7fb4a8cc9640 1 --2- 172.21.15.84:0/936279791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a4150300 0x7fb4a40cc280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4940098a0 tx=0x7fb49402db00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ec24e54d248364 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:29.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.675+0000 7fb49bfff640 1 -- 172.21.15.84:0/936279791 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49402f8c0 con 0x7fb4a4150300 2023-10-11T06:14:29.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.675+0000 7fb49bfff640 1 -- 172.21.15.84:0/936279791 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49402fa20 con 0x7fb4a4150300 2023-10-11T06:14:29.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.675+0000 7fb49bfff640 1 -- 172.21.15.84:0/936279791 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49402fdb0 con 0x7fb4a4150300 2023-10-11T06:14:29.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 -- 172.21.15.84:0/936279791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a4150300 msgr2=0x7fb4a40cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/936279791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a4150300 0x7fb4a40cc280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4940098a0 tx=0x7fb49402db00 comp rx=0 tx=0).stop 2023-10-11T06:14:29.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 -- 172.21.15.84:0/936279791 shutdown_connections 2023-10-11T06:14:29.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/936279791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a4150300 0x7fb4a40cc280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/936279791 >> v2:172.21.15.99:3300/0 conn(0x7fb4a414ff30 0x7fb4a40c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 -- 172.21.15.84:0/936279791 >> 172.21.15.84:0/936279791 conn(0x7fb4a4069c70 msgr2=0x7fb4a406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:29.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 -- 172.21.15.84:0/936279791 shutdown_connections 2023-10-11T06:14:29.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.676+0000 7fb4ab72e640 1 -- 172.21.15.84:0/936279791 wait complete. 2023-10-11T06:14:29.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.678+0000 7fb4ab72e640 1 Processor -- start 2023-10-11T06:14:29.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.678+0000 7fb4ab72e640 1 -- start start 2023-10-11T06:14:29.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4ab72e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a414ff30 0x7fb4a4159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4ab72e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a4150300 0x7fb4a414c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4ab72e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a40b1860 0x7fb4a406cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4ab72e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40b22b0 con 0x7fb4a414ff30 2023-10-11T06:14:29.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4ab72e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a4155300 con 0x7fb4a4150300 2023-10-11T06:14:29.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4ab72e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41555e0 con 0x7fb4a40b1860 2023-10-11T06:14:29.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a414ff30 0x7fb4a4159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a414ff30 0x7fb4a4159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57278/0 (socket says 172.21.15.84:57278) 2023-10-11T06:14:29.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 -- 172.21.15.84:0/1882134996 learned_addr learned my addr 172.21.15.84:0/1882134996 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:29.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a8cc9640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a4150300 0x7fb4a414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a9ccb640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a40b1860 0x7fb4a406cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 -- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a40b1860 msgr2=0x7fb4a406cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a40b1860 0x7fb4a406cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 -- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a4150300 msgr2=0x7fb4a414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a4150300 0x7fb4a414c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.679+0000 7fb4a94ca640 1 -- 172.21.15.84:0/1882134996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940094d0 con 0x7fb4a414ff30 2023-10-11T06:14:29.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.680+0000 7fb4a94ca640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a414ff30 0x7fb4a4159890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000e9b0 tx=0x7fb4a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:29.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.680+0000 7fb499ffb640 1 -- 172.21.15.84:0/1882134996 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0009670 con 0x7fb4a414ff30 2023-10-11T06:14:29.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.680+0000 7fb499ffb640 1 -- 172.21.15.84:0/1882134996 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a000c9a0 con 0x7fb4a414ff30 2023-10-11T06:14:29.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.680+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a41558c0 con 0x7fb4a414ff30 2023-10-11T06:14:29.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.680+0000 7fb499ffb640 1 -- 172.21.15.84:0/1882134996 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000cd50 con 0x7fb4a414ff30 2023-10-11T06:14:29.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.680+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a40ce900 con 0x7fb4a414ff30 2023-10-11T06:14:29.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.681+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a406ab90 con 0x7fb4a414ff30 2023-10-11T06:14:29.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.682+0000 7fb499ffb640 1 -- 172.21.15.84:0/1882134996 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb4a0013470 con 0x7fb4a414ff30 2023-10-11T06:14:29.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.683+0000 7fb499ffb640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb48806a830 0x7fb48806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:29.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.683+0000 7fb499ffb640 1 -- 172.21.15.84:0/1882134996 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4a0089250 con 0x7fb4a414ff30 2023-10-11T06:14:29.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.683+0000 7fb4a8cc9640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb48806a830 0x7fb48806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:29.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.684+0000 7fb4a8cc9640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb48806a830 0x7fb48806cce0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb494038000 tx=0x7fb494008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:29.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.688+0000 7fb499ffb640 1 -- 172.21.15.84:0/1882134996 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb4a00544c0 con 0x7fb4a414ff30 2023-10-11T06:14:29.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.877+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4a4073ed0 con 0x7fb4a414ff30 2023-10-11T06:14:29.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.878+0000 7fb499ffb640 1 -- 172.21.15.84:0/1882134996 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4a0057ae0 con 0x7fb4a414ff30 2023-10-11T06:14:29.879 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:14:29.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb48806a830 msgr2=0x7fb48806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb48806a830 0x7fb48806cce0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb494038000 tx=0x7fb494008030 comp rx=0 tx=0).stop 2023-10-11T06:14:29.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a414ff30 msgr2=0x7fb4a4159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:29.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a414ff30 0x7fb4a4159890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000e9b0 tx=0x7fb4a000ee70 comp rx=0 tx=0).stop 2023-10-11T06:14:29.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 shutdown_connections 2023-10-11T06:14:29.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb48806a830 0x7fb48806cce0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a40b1860 0x7fb4a406cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a4150300 0x7fb4a414c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 --2- 172.21.15.84:0/1882134996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4a414ff30 0x7fb4a4159890 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:29.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 >> 172.21.15.84:0/1882134996 conn(0x7fb4a4069c70 msgr2=0x7fb4a40cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:29.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.880+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 shutdown_connections 2023-10-11T06:14:29.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:29.881+0000 7fb4ab72e640 1 -- 172.21.15.84:0/1882134996 wait complete. 2023-10-11T06:14:29.900 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:14:30.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.108+0000 7f2b58d34640 1 Processor -- start 2023-10-11T06:14:30.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.109+0000 7f2b58d34640 1 -- start start 2023-10-11T06:14:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.110+0000 7f2b58d34640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2b5414ff30 0x7f2b540c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.111+0000 7f2b58d34640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b540c9b90 con 0x7f2b540ce510 2023-10-11T06:14:30.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.111+0000 7f2b58d34640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5415da10 con 0x7f2b54150300 2023-10-11T06:14:30.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.111+0000 7f2b58d34640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b540cc990 con 0x7f2b5414ff30 2023-10-11T06:14:30.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.111+0000 7f2b52d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2b540ce510 0x7f2b540ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43522/0 (socket says 172.21.15.84:43522) 2023-10-11T06:14:30.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.111+0000 7f2b52d9d640 1 -- 172.21.15.84:0/1691910344 learned_addr learned my addr 172.21.15.84:0/1691910344 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:30.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.111+0000 7f2b5259c640 1 --2- 172.21.15.84:0/1691910344 >> v2:172.21.15.99:3300/0 conn(0x7f2b5414ff30 0x7f2b540c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.111+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498970229 0 0) 0x7f2b540c9b90 con 0x7f2b540ce510 2023-10-11T06:14:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b3c003520 con 0x7f2b540ce510 2023-10-11T06:14:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1194078642 0 0) 0x7f2b3c003520 con 0x7f2b540ce510 2023-10-11T06:14:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 >> v2:172.21.15.99:3300/0 conn(0x7f2b5414ff30 msgr2=0x7f2b540c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 --2- 172.21.15.84:0/1691910344 >> v2:172.21.15.99:3300/0 conn(0x7f2b5414ff30 0x7f2b540c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 >> v1:172.21.15.99:6789/0 conn(0x7f2b54150300 legacy=0x7f2b540cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5415ce00 con 0x7f2b540ce510 2023-10-11T06:14:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f2b4c0019f0 con 0x7f2b540ce510 2023-10-11T06:14:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 >> v1:172.21.15.84:6789/0 conn(0x7f2b540ce510 legacy=0x7f2b540ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b3c0039c0 0x7f2b3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b3c0047c0 0x7f2b3c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0051c0 0x7f2b3c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540cc990 con 0x7f2b3c0039c0 2023-10-11T06:14:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c00c0d0 con 0x7f2b3c0047c0 2023-10-11T06:14:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c00c290 con 0x7f2b3c0051c0 2023-10-11T06:14:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.112+0000 7f2b5259c640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b3c0039c0 0x7f2b3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b5259c640 1 -- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0051c0 msgr2=0x7f2b3c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b5259c640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0051c0 0x7f2b3c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b5259c640 1 -- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b3c0047c0 msgr2=0x7f2b3c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b51d9b640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b3c0047c0 0x7f2b3c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b5259c640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b3c0047c0 0x7f2b3c004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b5259c640 1 -- 172.21.15.84:0/1691910344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c003520 con 0x7f2b3c0039c0 2023-10-11T06:14:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b52d9d640 1 -- 172.21.15.84:0/1691910344 reap_dead start 2023-10-11T06:14:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.113+0000 7f2b51d9b640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b3c0047c0 0x7f2b3c004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.114+0000 7f2b5259c640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b3c0039c0 0x7f2b3c003dd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009ac0 tx=0x7f2b3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7034bdf76c764b62 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.114+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4402c6b0 con 0x7f2b3c0039c0 2023-10-11T06:14:30.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.114+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4402c810 con 0x7f2b3c0039c0 2023-10-11T06:14:30.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.114+0000 7f2b5159a640 1 -- 172.21.15.84:0/1691910344 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4402cba0 con 0x7f2b3c0039c0 2023-10-11T06:14:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.115+0000 7f2b58d34640 1 -- 172.21.15.84:0/1691910344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b3c0039c0 msgr2=0x7f2b3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.115+0000 7f2b58d34640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b3c0039c0 0x7f2b3c003dd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009ac0 tx=0x7f2b3c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.115+0000 7f2b58d34640 1 -- 172.21.15.84:0/1691910344 shutdown_connections 2023-10-11T06:14:30.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.115+0000 7f2b58d34640 1 --2- 172.21.15.84:0/1691910344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b3c0039c0 0x7f2b3c003dd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.115+0000 7f2b58d34640 1 -- 172.21.15.84:0/1691910344 >> 172.21.15.84:0/1691910344 conn(0x7f2b54069c70 msgr2=0x7f2b5406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:30.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.115+0000 7f2b58d34640 1 -- 172.21.15.84:0/1691910344 shutdown_connections 2023-10-11T06:14:30.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.115+0000 7f2b58d34640 1 -- 172.21.15.84:0/1691910344 wait complete. 2023-10-11T06:14:30.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.117+0000 7f2b58d34640 1 Processor -- start 2023-10-11T06:14:30.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.117+0000 7f2b58d34640 1 -- start start 2023-10-11T06:14:30.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b58d34640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0039c0 0x7f2b541f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b58d34640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b5406a040 0x7f2b5406a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b58d34640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b54158210 0x7f2b54158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b58d34640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5406a9d0 con 0x7f2b5406a040 2023-10-11T06:14:30.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b58d34640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b541f4f70 con 0x7f2b54158210 2023-10-11T06:14:30.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b58d34640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b541f1800 con 0x7f2b3c0039c0 2023-10-11T06:14:30.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b5406a040 0x7f2b5406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b5406a040 0x7f2b5406a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57308/0 (socket says 172.21.15.84:57308) 2023-10-11T06:14:30.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 -- 172.21.15.84:0/2725478168 learned_addr learned my addr 172.21.15.84:0/2725478168 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b5259c640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0039c0 0x7f2b541f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 -- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0039c0 msgr2=0x7f2b541f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0039c0 0x7f2b541f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b52d9d640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b54158210 0x7f2b54158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 -- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b54158210 msgr2=0x7f2b54158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b54158210 0x7f2b54158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b51d9b640 1 -- 172.21.15.84:0/2725478168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540b28c0 con 0x7f2b5406a040 2023-10-11T06:14:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b5259c640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0039c0 0x7f2b541f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.118+0000 7f2b52d9d640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b54158210 0x7f2b54158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:30.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.119+0000 7f2b51d9b640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b5406a040 0x7f2b5406a490 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800d430 tx=0x7f2b4800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.119+0000 7f2b42ffd640 1 -- 172.21.15.84:0/2725478168 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800b070 con 0x7f2b5406a040 2023-10-11T06:14:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.119+0000 7f2b42ffd640 1 -- 172.21.15.84:0/2725478168 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b480052d0 con 0x7f2b5406a040 2023-10-11T06:14:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.119+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b541f51e0 con 0x7f2b5406a040 2023-10-11T06:14:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.119+0000 7f2b42ffd640 1 -- 172.21.15.84:0/2725478168 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48003610 con 0x7f2b5406a040 2023-10-11T06:14:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.119+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b541f57a0 con 0x7f2b5406a040 2023-10-11T06:14:30.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.121+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b541f5930 con 0x7f2b5406a040 2023-10-11T06:14:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.121+0000 7f2b42ffd640 1 -- 172.21.15.84:0/2725478168 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2b48003770 con 0x7f2b5406a040 2023-10-11T06:14:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.122+0000 7f2b42ffd640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b3406a860 0x7f2b3406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.122+0000 7f2b42ffd640 1 -- 172.21.15.84:0/2725478168 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b480883f0 con 0x7f2b5406a040 2023-10-11T06:14:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.122+0000 7f2b5259c640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b3406a860 0x7f2b3406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.123+0000 7f2b5259c640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b3406a860 0x7f2b3406cd10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b44036000 tx=0x7f2b44030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:30.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.126+0000 7f2b42ffd640 1 -- 172.21.15.84:0/2725478168 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2b48053720 con 0x7f2b5406a040 2023-10-11T06:14:30.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.317+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2b5406ab90 con 0x7f2b5406a040 2023-10-11T06:14:30.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.318+0000 7f2b42ffd640 1 -- 172.21.15.84:0/2725478168 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2b48056d40 con 0x7f2b5406a040 2023-10-11T06:14:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:14:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.320+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b3406a860 msgr2=0x7f2b3406cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.320+0000 7f2b58d34640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b3406a860 0x7f2b3406cd10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b44036000 tx=0x7f2b44030000 comp rx=0 tx=0).stop 2023-10-11T06:14:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.320+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b5406a040 msgr2=0x7f2b5406a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.320+0000 7f2b58d34640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b5406a040 0x7f2b5406a490 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800d430 tx=0x7f2b4800d520 comp rx=0 tx=0).stop 2023-10-11T06:14:30.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 shutdown_connections 2023-10-11T06:14:30.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b3406a860 0x7f2b3406cd10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b54158210 0x7f2b54158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b5406a040 0x7f2b5406a490 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 --2- 172.21.15.84:0/2725478168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b3c0039c0 0x7f2b541f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 >> 172.21.15.84:0/2725478168 conn(0x7f2b54069c70 msgr2=0x7f2b540cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:30.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 shutdown_connections 2023-10-11T06:14:30.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.321+0000 7f2b58d34640 1 -- 172.21.15.84:0/2725478168 wait complete. 2023-10-11T06:14:30.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:14:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.552+0000 7f77b6949640 1 Processor -- start 2023-10-11T06:14:30.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.553+0000 7f77b6949640 1 -- start start 2023-10-11T06:14:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.554+0000 7f77b6949640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b015f7e0 0x7f77b00cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.554+0000 7f77b6949640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f77b00cd6c0 0x7f77b00d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.554+0000 7f77b6949640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b00d3460 con 0x7f77b00d3000 2023-10-11T06:14:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.554+0000 7f77b6949640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b00cf220 con 0x7f77b00cd6c0 2023-10-11T06:14:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.554+0000 7f77b6949640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b00cf500 con 0x7f77b015f7e0 2023-10-11T06:14:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77af7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f77b00cd6c0 0x7f77b00d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77af7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f77b00cd6c0 0x7f77b00d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57318/0 (socket says 172.21.15.84:57318) 2023-10-11T06:14:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77af7fe640 1 -- 172.21.15.84:0/935068070 learned_addr learned my addr 172.21.15.84:0/935068070 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77affff640 1 --2- 172.21.15.84:0/935068070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b015f7e0 0x7f77b00cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77af7fe640 1 -- 172.21.15.84:0/935068070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b015f7e0 msgr2=0x7f77b00cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77af7fe640 1 --2- 172.21.15.84:0/935068070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b015f7e0 0x7f77b00cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77af7fe640 1 -- 172.21.15.84:0/935068070 >> v1:172.21.15.84:6789/0 conn(0x7f77b00d3000 legacy=0x7f77b015fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77af7fe640 1 -- 172.21.15.84:0/935068070 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b00cf7e0 con 0x7f77b00cd6c0 2023-10-11T06:14:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.555+0000 7f77affff640 1 --2- 172.21.15.84:0/935068070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b015f7e0 0x7f77b00cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:30.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.557+0000 7f77af7fe640 1 --2- 172.21.15.84:0/935068070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00d2ac0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7798009870 tx=0x7f779802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8a5090d33794eb server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.557+0000 7f77aeffd640 1 -- 172.21.15.84:0/935068070 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798030840 con 0x7f77b00cd6c0 2023-10-11T06:14:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.557+0000 7f77aeffd640 1 -- 172.21.15.84:0/935068070 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77980309a0 con 0x7f77b00cd6c0 2023-10-11T06:14:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.557+0000 7f77aeffd640 1 -- 172.21.15.84:0/935068070 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798030d30 con 0x7f77b00cd6c0 2023-10-11T06:14:30.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 -- 172.21.15.84:0/935068070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 msgr2=0x7f77b00d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 --2- 172.21.15.84:0/935068070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00d2ac0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7798009870 tx=0x7f779802b980 comp rx=0 tx=0).stop 2023-10-11T06:14:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 -- 172.21.15.84:0/935068070 shutdown_connections 2023-10-11T06:14:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 --2- 172.21.15.84:0/935068070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00d2ac0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 --2- 172.21.15.84:0/935068070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b015f7e0 0x7f77b00cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 -- 172.21.15.84:0/935068070 >> 172.21.15.84:0/935068070 conn(0x7f77b00713a0 msgr2=0x7f77b00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 -- 172.21.15.84:0/935068070 shutdown_connections 2023-10-11T06:14:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.558+0000 7f77b6949640 1 -- 172.21.15.84:0/935068070 wait complete. 2023-10-11T06:14:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.560+0000 7f77b6949640 1 Processor -- start 2023-10-11T06:14:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.560+0000 7f77b6949640 1 -- start start 2023-10-11T06:14:30.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77b6949640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77b6949640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b00d3000 0x7f77b00c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77b6949640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b015f7e0 0x7f77b0156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77b6949640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b00c9a30 con 0x7f77b00cd6c0 2023-10-11T06:14:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77b6949640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b00c7ee0 con 0x7f77b015f7e0 2023-10-11T06:14:30.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77b6949640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b00b5960 con 0x7f77b00d3000 2023-10-11T06:14:30.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57320/0 (socket says 172.21.15.84:57320) 2023-10-11T06:14:30.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 -- 172.21.15.84:0/1183333429 learned_addr learned my addr 172.21.15.84:0/1183333429 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:30.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 -- 172.21.15.84:0/1183333429 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b00d3000 msgr2=0x7f77b00c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b00d3000 0x7f77b00c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 -- 172.21.15.84:0/1183333429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b015f7e0 msgr2=0x7f77b0156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b015f7e0 0x7f77b0156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.561+0000 7f77affff640 1 -- 172.21.15.84:0/1183333429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980094d0 con 0x7f77b00cd6c0 2023-10-11T06:14:30.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.562+0000 7f77affff640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00c7220 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77a000ce00 tx=0x7f77a000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:30.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.562+0000 7f77acff9640 1 -- 172.21.15.84:0/1183333429 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000f890 con 0x7f77b00cd6c0 2023-10-11T06:14:30.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.562+0000 7f77acff9640 1 -- 172.21.15.84:0/1183333429 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a000f9f0 con 0x7f77b00cd6c0 2023-10-11T06:14:30.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.562+0000 7f77acff9640 1 -- 172.21.15.84:0/1183333429 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000fdc0 con 0x7f77b00cd6c0 2023-10-11T06:14:30.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.562+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b00c7920 con 0x7f77b00cd6c0 2023-10-11T06:14:30.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.562+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b00b5b70 con 0x7f77b00cd6c0 2023-10-11T06:14:30.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.563+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b0073ed0 con 0x7f77b00cd6c0 2023-10-11T06:14:30.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.564+0000 7f77acff9640 1 -- 172.21.15.84:0/1183333429 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f77a000b040 con 0x7f77b00cd6c0 2023-10-11T06:14:30.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.565+0000 7f77acff9640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f779006a830 0x7f779006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:30.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.565+0000 7f77acff9640 1 -- 172.21.15.84:0/1183333429 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77a00899d0 con 0x7f77b00cd6c0 2023-10-11T06:14:30.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.565+0000 7f77af7fe640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f779006a830 0x7f779006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:30.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.566+0000 7f77af7fe640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f779006a830 0x7f779006cce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7798036000 tx=0x7f779803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:30.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.570+0000 7f77acff9640 1 -- 172.21.15.84:0/1183333429 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f77a0054c40 con 0x7f77b00cd6c0 2023-10-11T06:14:30.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.759+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77b00b5030 con 0x7f77b00cd6c0 2023-10-11T06:14:30.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.760+0000 7f77acff9640 1 -- 172.21.15.84:0/1183333429 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77a001d020 con 0x7f77b00cd6c0 2023-10-11T06:14:30.761 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:14:30.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f779006a830 msgr2=0x7f779006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f779006a830 0x7f779006cce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7798036000 tx=0x7f779803b040 comp rx=0 tx=0).stop 2023-10-11T06:14:30.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 msgr2=0x7f77b00c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:30.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00c7220 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f77a000ce00 tx=0x7f77a000a650 comp rx=0 tx=0).stop 2023-10-11T06:14:30.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 shutdown_connections 2023-10-11T06:14:30.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f779006a830 0x7f779006cce0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b015f7e0 0x7f77b0156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77b00d3000 0x7f77b00c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 --2- 172.21.15.84:0/1183333429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77b00cd6c0 0x7f77b00c7220 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:30.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.762+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 >> 172.21.15.84:0/1183333429 conn(0x7f77b00713a0 msgr2=0x7f77b015fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:30.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.763+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 shutdown_connections 2023-10-11T06:14:30.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:30.763+0000 7f77b6949640 1 -- 172.21.15.84:0/1183333429 wait complete. 2023-10-11T06:14:30.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:14:31.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.004+0000 7f4722051640 1 Processor -- start 2023-10-11T06:14:31.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.004+0000 7f4722051640 1 -- start start 2023-10-11T06:14:31.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.005+0000 7f4722051640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f471c14ff30 0x7f471c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.005+0000 7f4722051640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c0c9b90 con 0x7f471c150300 2023-10-11T06:14:31.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.005+0000 7f4722051640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c15da10 con 0x7f471c0ce510 2023-10-11T06:14:31.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.005+0000 7f4722051640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471c0cc990 con 0x7f471c14ff30 2023-10-11T06:14:31.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f471c150300 0x7f471c0cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43534/0 (socket says 172.21.15.84:43534) 2023-10-11T06:14:31.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471affd640 1 -- 172.21.15.84:0/1136935271 learned_addr learned my addr 172.21.15.84:0/1136935271 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471b7fe640 1 --2- 172.21.15.84:0/1136935271 >> v2:172.21.15.99:3300/0 conn(0x7f471c14ff30 0x7f471c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493307511 0 0) 0x7f471c0c9b90 con 0x7f471c150300 2023-10-11T06:14:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4704003520 con 0x7f471c150300 2023-10-11T06:14:31.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2205178625 0 0) 0x7f4704003520 con 0x7f471c150300 2023-10-11T06:14:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 >> v2:172.21.15.99:3300/0 conn(0x7f471c14ff30 msgr2=0x7f471c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 --2- 172.21.15.84:0/1136935271 >> v2:172.21.15.99:3300/0 conn(0x7f471c14ff30 0x7f471c0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 >> v1:172.21.15.99:6789/0 conn(0x7f471c0ce510 legacy=0x7f471c0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c15ce00 con 0x7f471c150300 2023-10-11T06:14:31.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.006+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f47100019f0 con 0x7f471c150300 2023-10-11T06:14:31.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 >> v1:172.21.15.84:6789/0 conn(0x7f471c150300 legacy=0x7f471c0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471a7fc640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4704003970 0x7f4704003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471a7fc640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f4704004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471a7fc640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704005170 0x7f470400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0cc990 con 0x7f47040047c0 2023-10-11T06:14:31.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470400c080 con 0x7f4704005170 2023-10-11T06:14:31.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470400c240 con 0x7f4704003970 2023-10-11T06:14:31.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471affd640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f4704004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471b7fe640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4704003970 0x7f4704003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471affd640 1 -- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4704003970 msgr2=0x7f4704003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:31.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471bfff640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704005170 0x7f470400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471affd640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4704003970 0x7f4704003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471affd640 1 -- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704005170 msgr2=0x7f470400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:31.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471affd640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704005170 0x7f470400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471affd640 1 -- 172.21.15.84:0/1136935271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704003520 con 0x7f47040047c0 2023-10-11T06:14:31.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.007+0000 7f471bfff640 1 -- 172.21.15.84:0/1136935271 reap_dead start 2023-10-11T06:14:31.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.008+0000 7f471affd640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f4704004c30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f471000a820 tx=0x7f47040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3836566c0e571a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:31.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100072d0 con 0x7f47040047c0 2023-10-11T06:14:31.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471002f070 con 0x7f47040047c0 2023-10-11T06:14:31.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f471a7fc640 1 -- 172.21.15.84:0/1136935271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100076f0 con 0x7f47040047c0 2023-10-11T06:14:31.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f4722051640 1 -- 172.21.15.84:0/1136935271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 msgr2=0x7f4704004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f4722051640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f4704004c30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f471000a820 tx=0x7f47040057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f4722051640 1 -- 172.21.15.84:0/1136935271 shutdown_connections 2023-10-11T06:14:31.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f4722051640 1 --2- 172.21.15.84:0/1136935271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f4704004c30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f4722051640 1 -- 172.21.15.84:0/1136935271 >> 172.21.15.84:0/1136935271 conn(0x7f471c069c70 msgr2=0x7f471c06a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:31.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f4722051640 1 -- 172.21.15.84:0/1136935271 shutdown_connections 2023-10-11T06:14:31.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.009+0000 7f4722051640 1 -- 172.21.15.84:0/1136935271 wait complete. 2023-10-11T06:14:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.011+0000 7f4722051640 1 Processor -- start 2023-10-11T06:14:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.011+0000 7f4722051640 1 -- start start 2023-10-11T06:14:31.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.012+0000 7f4722051640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f471c152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.012+0000 7f4722051640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f471c06c7c0 0x7f471c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.012+0000 7f4722051640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f471c142bc0 0x7f471c143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.012+0000 7f4722051640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c135bb0 con 0x7f47040047c0 2023-10-11T06:14:31.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.012+0000 7f4722051640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c06d170 con 0x7f471c06c7c0 2023-10-11T06:14:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.012+0000 7f4722051640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c142730 con 0x7f471c142bc0 2023-10-11T06:14:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.012+0000 7f471b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f471c152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f471c152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57350/0 (socket says 172.21.15.84:57350) 2023-10-11T06:14:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 -- 172.21.15.84:0/483657320 learned_addr learned my addr 172.21.15.84:0/483657320 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 -- 172.21.15.84:0/483657320 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f471c142bc0 msgr2=0x7f471c143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f471c142bc0 0x7f471c143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 -- 172.21.15.84:0/483657320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f471c06c7c0 msgr2=0x7f471c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f471c06c7c0 0x7f471c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 -- 172.21.15.84:0/483657320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c153520 con 0x7f47040047c0 2023-10-11T06:14:31.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f471b7fe640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f471c152380 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f470c010940 tx=0x7f470c010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:31.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.013+0000 7f4703fff640 1 -- 172.21.15.84:0/483657320 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c009690 con 0x7f47040047c0 2023-10-11T06:14:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.014+0000 7f4703fff640 1 -- 172.21.15.84:0/483657320 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00ec20 con 0x7f47040047c0 2023-10-11T06:14:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.014+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c152c80 con 0x7f47040047c0 2023-10-11T06:14:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.014+0000 7f4703fff640 1 -- 172.21.15.84:0/483657320 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c01c400 con 0x7f47040047c0 2023-10-11T06:14:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.014+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c153240 con 0x7f47040047c0 2023-10-11T06:14:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.015+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c06ab90 con 0x7f47040047c0 2023-10-11T06:14:31.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.016+0000 7f4703fff640 1 -- 172.21.15.84:0/483657320 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f470c01c560 con 0x7f47040047c0 2023-10-11T06:14:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.017+0000 7f4703fff640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f470806a860 0x7f470806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.017+0000 7f4703fff640 1 -- 172.21.15.84:0/483657320 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f470c089660 con 0x7f47040047c0 2023-10-11T06:14:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.017+0000 7f471affd640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f470806a860 0x7f470806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.018+0000 7f471affd640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f470806a860 0x7f470806cd10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4710034000 tx=0x7f471002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:31.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.022+0000 7f4703fff640 1 -- 172.21.15.84:0/483657320 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f470c0548d0 con 0x7f47040047c0 2023-10-11T06:14:31.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.210+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f471c073ed0 con 0x7f47040047c0 2023-10-11T06:14:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.211+0000 7f4703fff640 1 -- 172.21.15.84:0/483657320 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f470c057ef0 con 0x7f47040047c0 2023-10-11T06:14:31.212 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:14:31.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f470806a860 msgr2=0x7f470806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f470806a860 0x7f470806cd10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4710034000 tx=0x7f471002f000 comp rx=0 tx=0).stop 2023-10-11T06:14:31.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 msgr2=0x7f471c152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f471c152380 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f470c010940 tx=0x7f470c010e00 comp rx=0 tx=0).stop 2023-10-11T06:14:31.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 shutdown_connections 2023-10-11T06:14:31.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f470806a860 0x7f470806cd10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f471c142bc0 0x7f471c143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f471c06c7c0 0x7f471c06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 --2- 172.21.15.84:0/483657320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47040047c0 0x7f471c152380 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.213+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 >> 172.21.15.84:0/483657320 conn(0x7f471c069c70 msgr2=0x7f471c0c45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:31.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.214+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 shutdown_connections 2023-10-11T06:14:31.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.214+0000 7f4722051640 1 -- 172.21.15.84:0/483657320 wait complete. 2023-10-11T06:14:31.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-11T06:14:31.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.447+0000 7f27941fd640 1 Processor -- start 2023-10-11T06:14:31.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.447+0000 7f27941fd640 1 -- start start 2023-10-11T06:14:31.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.448+0000 7f27941fd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f278c0cafb0 0x7f278c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.448+0000 7f27941fd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f278c0cc410 0x7f278c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.448+0000 7f27941fd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c0cb970 con 0x7f278c15c290 2023-10-11T06:14:31.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.448+0000 7f27941fd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0cad70 con 0x7f278c0cafb0 2023-10-11T06:14:31.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.448+0000 7f27941fd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c15fe40 con 0x7f278c0cc410 2023-10-11T06:14:31.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791798640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f278c0cafb0 0x7f278c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791798640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f278c0cafb0 0x7f278c0cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57356/0 (socket says 172.21.15.84:57356) 2023-10-11T06:14:31.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791798640 1 -- 172.21.15.84:0/664630284 learned_addr learned my addr 172.21.15.84:0/664630284 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791f99640 1 --1- 172.21.15.84:0/664630284 >> v1:172.21.15.84:6789/0 conn(0x7f278c15c290 0x7f278c0c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43542/0 (socket says 172.21.15.84:43542) 2023-10-11T06:14:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f279279a640 1 --2- 172.21.15.84:0/664630284 >> v2:172.21.15.99:3300/0 conn(0x7f278c0cc410 0x7f278c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791798640 1 -- 172.21.15.84:0/664630284 >> v2:172.21.15.99:3300/0 conn(0x7f278c0cc410 msgr2=0x7f278c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791798640 1 --2- 172.21.15.84:0/664630284 >> v2:172.21.15.99:3300/0 conn(0x7f278c0cc410 0x7f278c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791798640 1 -- 172.21.15.84:0/664630284 >> v1:172.21.15.84:6789/0 conn(0x7f278c15c290 legacy=0x7f278c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f2791798640 1 -- 172.21.15.84:0/664630284 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c06ff30 con 0x7f278c0cafb0 2023-10-11T06:14:31.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.449+0000 7f279279a640 1 --2- 172.21.15.84:0/664630284 >> v2:172.21.15.99:3300/0 conn(0x7f278c0cc410 0x7f278c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:31.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.450+0000 7f2791798640 1 --2- 172.21.15.84:0/664630284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c0cb430 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f278c06fd30 tx=0x7f277c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f06d0df74e6692 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:31.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f2790f97640 1 -- 172.21.15.84:0/664630284 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c030870 con 0x7f278c0cafb0 2023-10-11T06:14:31.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f2790f97640 1 -- 172.21.15.84:0/664630284 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c0309d0 con 0x7f278c0cafb0 2023-10-11T06:14:31.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f2790f97640 1 -- 172.21.15.84:0/664630284 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c030d60 con 0x7f278c0cafb0 2023-10-11T06:14:31.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f27941fd640 1 -- 172.21.15.84:0/664630284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 msgr2=0x7f278c0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f27941fd640 1 --2- 172.21.15.84:0/664630284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c0cb430 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f278c06fd30 tx=0x7f277c02b9b0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f27941fd640 1 -- 172.21.15.84:0/664630284 shutdown_connections 2023-10-11T06:14:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f27941fd640 1 --2- 172.21.15.84:0/664630284 >> v2:172.21.15.99:3300/0 conn(0x7f278c0cc410 0x7f278c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f27941fd640 1 --2- 172.21.15.84:0/664630284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c0cb430 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f27941fd640 1 -- 172.21.15.84:0/664630284 >> 172.21.15.84:0/664630284 conn(0x7f278c069c70 msgr2=0x7f278c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:31.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.451+0000 7f27941fd640 1 -- 172.21.15.84:0/664630284 shutdown_connections 2023-10-11T06:14:31.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.452+0000 7f27941fd640 1 -- 172.21.15.84:0/664630284 wait complete. 2023-10-11T06:14:31.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.453+0000 7f27941fd640 1 Processor -- start 2023-10-11T06:14:31.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.453+0000 7f27941fd640 1 -- start start 2023-10-11T06:14:31.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.454+0000 7f27941fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c1f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.454+0000 7f27941fd640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f278c0cc410 0x7f278c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.454+0000 7f27941fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f278c15c290 0x7f278c0ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.454+0000 7f27941fd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1f2dc0 con 0x7f278c0cafb0 2023-10-11T06:14:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.454+0000 7f27941fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1f2430 con 0x7f278c15c290 2023-10-11T06:14:31.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.454+0000 7f27941fd640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1f2710 con 0x7f278c0cc410 2023-10-11T06:14:31.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.454+0000 7f2791f99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c1f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c1f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57364/0 (socket says 172.21.15.84:57364) 2023-10-11T06:14:31.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 -- 172.21.15.84:0/2944599070 learned_addr learned my addr 172.21.15.84:0/2944599070 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:31.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f279279a640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f278c15c290 0x7f278c0ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 -- 172.21.15.84:0/2944599070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f278c0cc410 msgr2=0x7f278c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f278c0cc410 0x7f278c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 -- 172.21.15.84:0/2944599070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f278c15c290 msgr2=0x7f278c0ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f278c15c290 0x7f278c0ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 -- 172.21.15.84:0/2944599070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c0094d0 con 0x7f278c0cafb0 2023-10-11T06:14:31.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f2791f99640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c1f1bd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f278800c500 tx=0x7f278800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:31.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f27827fc640 1 -- 172.21.15.84:0/2944599070 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788007500 con 0x7f278c0cafb0 2023-10-11T06:14:31.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f27827fc640 1 -- 172.21.15.84:0/2944599070 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27880057a0 con 0x7f278c0cafb0 2023-10-11T06:14:31.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f27827fc640 1 -- 172.21.15.84:0/2944599070 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788007660 con 0x7f278c0cafb0 2023-10-11T06:14:31.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.455+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c1f29f0 con 0x7f278c0cafb0 2023-10-11T06:14:31.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.456+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c1f2110 con 0x7f278c0cafb0 2023-10-11T06:14:31.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.457+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c1f3210 con 0x7f278c0cafb0 2023-10-11T06:14:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.458+0000 7f27827fc640 1 -- 172.21.15.84:0/2944599070 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2788005bd0 con 0x7f278c0cafb0 2023-10-11T06:14:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.458+0000 7f27827fc640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f277006a860 0x7f277006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.459+0000 7f27827fc640 1 -- 172.21.15.84:0/2944599070 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27880885a0 con 0x7f278c0cafb0 2023-10-11T06:14:31.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.459+0000 7f2791798640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f277006a860 0x7f277006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:31.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.460+0000 7f2791798640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f277006a860 0x7f277006cd10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f277c036000 tx=0x7f277c03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:31.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.463+0000 7f27827fc640 1 -- 172.21.15.84:0/2944599070 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f278c1f3210 con 0x7f278c0cafb0 2023-10-11T06:14:31.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.661+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f278c1f3210 con 0x7f278c0cafb0 2023-10-11T06:14:31.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.662+0000 7f27827fc640 1 -- 172.21.15.84:0/2944599070 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2788017080 con 0x7f278c0cafb0 2023-10-11T06:14:31.664 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-10-11T06:14:31.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f277006a860 msgr2=0x7f277006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f277006a860 0x7f277006cd10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f277c036000 tx=0x7f277c03b040 comp rx=0 tx=0).stop 2023-10-11T06:14:31.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 msgr2=0x7f278c1f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:31.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c1f1bd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f278800c500 tx=0x7f278800c9c0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 shutdown_connections 2023-10-11T06:14:31.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f277006a860 0x7f277006cd10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f278c15c290 0x7f278c0ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f278c0cc410 0x7f278c0b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 --2- 172.21.15.84:0/2944599070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f278c0cafb0 0x7f278c1f1bd0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:31.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.664+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 >> 172.21.15.84:0/2944599070 conn(0x7f278c069c70 msgr2=0x7f278c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:31.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.665+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 shutdown_connections 2023-10-11T06:14:31.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:31.665+0000 7f27941fd640 1 -- 172.21.15.84:0/2944599070 wait complete. 2023-10-11T06:14:31.684 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-10-11T06:14:31.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:31.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-11T06:14:31.766 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[109886]: fuse finished with error 0 and tester_r 0 2023-10-11T06:14:31.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:14:31.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:31.770 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-11T06:14:31.902 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:14:31.903 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:31.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:14:31.964 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:14:31.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:31.965 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-11T06:14:32.017 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-11T06:14:32.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:14:32.018 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:14:32.019 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:32.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053598,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T06:14:32.078 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:14:32.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:32.079 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-11T06:14:32.132 INFO:teuthology.orchestra.run.smithi084.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-11T06:14:32.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:14:32.133 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:14:32.134 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:32.193 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:14:32.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:14:32.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:32.194 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-11T06:14:32.245 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-11T06:14:32.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:14:32.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:14:32.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.470+0000 7fe57fe2d640 1 Processor -- start 2023-10-11T06:14:32.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.470+0000 7fe57fe2d640 1 -- start start 2023-10-11T06:14:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.471+0000 7fe57fe2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5780ce0d0 0x7fe5780c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.471+0000 7fe57fe2d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe5780adb90 0x7fe5780caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.471+0000 7fe57fe2d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5781593f0 con 0x7fe5780cd300 2023-10-11T06:14:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.471+0000 7fe57fe2d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57815a340 con 0x7fe5780ce0d0 2023-10-11T06:14:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.471+0000 7fe57fe2d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5780ce4a0 con 0x7fe5780adb90 2023-10-11T06:14:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57dbc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5780ce0d0 0x7fe5780c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57e3ca640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe5780cd300 0x7fe5780b0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43546/0 (socket says 172.21.15.84:43546) 2023-10-11T06:14:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57dbc9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5780ce0d0 0x7fe5780c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57370/0 (socket says 172.21.15.84:57370) 2023-10-11T06:14:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57e3ca640 1 -- 172.21.15.84:0/3495404823 learned_addr learned my addr 172.21.15.84:0/3495404823 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57d3c8640 1 --2- 172.21.15.84:0/3495404823 >> v2:172.21.15.99:3300/0 conn(0x7fe5780adb90 0x7fe5780caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57dbc9640 1 -- 172.21.15.84:0/3495404823 >> v2:172.21.15.99:3300/0 conn(0x7fe5780adb90 msgr2=0x7fe5780caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:32.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57dbc9640 1 --2- 172.21.15.84:0/3495404823 >> v2:172.21.15.99:3300/0 conn(0x7fe5780adb90 0x7fe5780caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57dbc9640 1 -- 172.21.15.84:0/3495404823 >> v1:172.21.15.84:6789/0 conn(0x7fe5780cd300 legacy=0x7fe5780b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57dbc9640 1 -- 172.21.15.84:0/3495404823 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5781596c0 con 0x7fe5780ce0d0 2023-10-11T06:14:32.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.472+0000 7fe57cbc7640 1 -- 172.21.15.84:0/3495404823 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941338606 0 0) 0x7fe5781593f0 con 0x7fe5780cd300 2023-10-11T06:14:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.473+0000 7fe57dbc9640 1 --2- 172.21.15.84:0/3495404823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780ce0d0 0x7fe5780c8cc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe57806e440 tx=0x7fe57402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87aa413756c2225 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57cbc7640 1 -- 172.21.15.84:0/3495404823 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57402d040 con 0x7fe5780ce0d0 2023-10-11T06:14:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57cbc7640 1 -- 172.21.15.84:0/3495404823 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574031810 con 0x7fe5780ce0d0 2023-10-11T06:14:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57cbc7640 1 -- 172.21.15.84:0/3495404823 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574031ba0 con 0x7fe5780ce0d0 2023-10-11T06:14:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 -- 172.21.15.84:0/3495404823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780ce0d0 msgr2=0x7fe5780c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/3495404823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780ce0d0 0x7fe5780c8cc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe57806e440 tx=0x7fe57402b8b0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 -- 172.21.15.84:0/3495404823 shutdown_connections 2023-10-11T06:14:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/3495404823 >> v2:172.21.15.99:3300/0 conn(0x7fe5780adb90 0x7fe5780caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/3495404823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780ce0d0 0x7fe5780c8cc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 -- 172.21.15.84:0/3495404823 >> 172.21.15.84:0/3495404823 conn(0x7fe578069c70 msgr2=0x7fe57806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 -- 172.21.15.84:0/3495404823 shutdown_connections 2023-10-11T06:14:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.474+0000 7fe57fe2d640 1 -- 172.21.15.84:0/3495404823 wait complete. 2023-10-11T06:14:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.476+0000 7fe57fe2d640 1 Processor -- start 2023-10-11T06:14:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.476+0000 7fe57fe2d640 1 -- start start 2023-10-11T06:14:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57fe2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780adb90 0x7fe578148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57fe2d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5780cd300 0x7fe578156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57fe2d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5780ce0d0 0x7fe578157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57fe2d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780c7620 con 0x7fe5780adb90 2023-10-11T06:14:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57fe2d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe578148f80 con 0x7fe5780cd300 2023-10-11T06:14:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57fe2d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780c7340 con 0x7fe5780ce0d0 2023-10-11T06:14:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57dbc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780adb90 0x7fe578148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57dbc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780adb90 0x7fe578148a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57372/0 (socket says 172.21.15.84:57372) 2023-10-11T06:14:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.477+0000 7fe57dbc9640 1 -- 172.21.15.84:0/993354649 learned_addr learned my addr 172.21.15.84:0/993354649 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe57dbc9640 1 -- 172.21.15.84:0/993354649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5780ce0d0 msgr2=0x7fe578157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe57dbc9640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5780ce0d0 0x7fe578157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe57dbc9640 1 -- 172.21.15.84:0/993354649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5780cd300 msgr2=0x7fe578156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe57dbc9640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5780cd300 0x7fe578156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe57dbc9640 1 -- 172.21.15.84:0/993354649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5740093a0 con 0x7fe5780adb90 2023-10-11T06:14:32.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe57dbc9640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780adb90 0x7fe578148a40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe5740098b0 tx=0x7fe5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:32.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe56e7fc640 1 -- 172.21.15.84:0/993354649 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57403ad60 con 0x7fe5780adb90 2023-10-11T06:14:32.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe56e7fc640 1 -- 172.21.15.84:0/993354649 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574005990 con 0x7fe5780adb90 2023-10-11T06:14:32.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe56e7fc640 1 -- 172.21.15.84:0/993354649 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574005d40 con 0x7fe5780adb90 2023-10-11T06:14:32.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.478+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe578154900 con 0x7fe5780adb90 2023-10-11T06:14:32.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.479+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe578154ec0 con 0x7fe5780adb90 2023-10-11T06:14:32.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.480+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe578154760 con 0x7fe5780adb90 2023-10-11T06:14:32.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.480+0000 7fe56e7fc640 1 -- 172.21.15.84:0/993354649 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe574005ea0 con 0x7fe5780adb90 2023-10-11T06:14:32.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.481+0000 7fe56e7fc640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe55c06a830 0x7fe55c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.481+0000 7fe56e7fc640 1 -- 172.21.15.84:0/993354649 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5740ab480 con 0x7fe5780adb90 2023-10-11T06:14:32.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.482+0000 7fe57d3c8640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe55c06a830 0x7fe55c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.482+0000 7fe57d3c8640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe55c06a830 0x7fe55c06cce0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe568006fd0 tx=0x7fe568008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:32.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.485+0000 7fe56e7fc640 1 -- 172.21.15.84:0/993354649 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe578154760 con 0x7fe5780adb90 2023-10-11T06:14:32.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.711+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe578154760 con 0x7fe5780adb90 2023-10-11T06:14:32.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.713+0000 7fe56e7fc640 1 -- 172.21.15.84:0/993354649 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4648 (secure 0 0 0) 0x7fe578154760 con 0x7fe5780adb90 2023-10-11T06:14:32.714 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:32.715 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1835716973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1835716973},{"type":"v1","addr":"172.21.15.99:6833","nonce":1835716973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1415879342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1415879342},{"type":"v1","addr":"172.21.15.99:6835","nonce":1415879342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2122307729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2122307729},{"type":"v1","addr":"172.21.15.84:6836","nonce":2122307729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:13:46.389013+0000","modified":"2023-10-11T06:13:49.406181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6027":{"gid":6027,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2048038445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2048038445},{"type":"v1","addr":"172.21.15.84:6837","nonce":2048038445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T06:14:32.715 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-10-11T06:14:32.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.715+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe55c06a830 msgr2=0x7fe55c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe55c06a830 0x7fe55c06cce0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe568006fd0 tx=0x7fe568008040 comp rx=0 tx=0).stop 2023-10-11T06:14:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780adb90 msgr2=0x7fe578148a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780adb90 0x7fe578148a40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe5740098b0 tx=0x7fe5740046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 shutdown_connections 2023-10-11T06:14:32.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe55c06a830 0x7fe55c06cce0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5780ce0d0 0x7fe578157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5780cd300 0x7fe578156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 --2- 172.21.15.84:0/993354649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5780adb90 0x7fe578148a40 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 >> 172.21.15.84:0/993354649 conn(0x7fe578069c70 msgr2=0x7fe5780ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 shutdown_connections 2023-10-11T06:14:32.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.716+0000 7fe57fe2d640 1 -- 172.21.15.84:0/993354649 wait complete. 2023-10-11T06:14:32.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.931+0000 7f66c3f27640 1 Processor -- start 2023-10-11T06:14:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.931+0000 7f66c3f27640 1 -- start start 2023-10-11T06:14:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.932+0000 7f66c3f27640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f66bc15d4f0 0x7f66bc0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c3f27640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 0x7f66bc0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c3f27640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66bc071a90 con 0x7f66bc0ca300 2023-10-11T06:14:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c3f27640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc072310 con 0x7f66bc15d4f0 2023-10-11T06:14:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c3f27640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc070280 con 0x7f66bc0c9360 2023-10-11T06:14:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c1cc3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f66bc15d4f0 0x7f66bc0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c1cc3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f66bc15d4f0 0x7f66bc0cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57380/0 (socket says 172.21.15.84:57380) 2023-10-11T06:14:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c1cc3640 1 -- 172.21.15.84:0/3249433728 learned_addr learned my addr 172.21.15.84:0/3249433728 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c1cc3640 1 -- 172.21.15.84:0/3249433728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 msgr2=0x7f66bc0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c1cc3640 1 --2- 172.21.15.84:0/3249433728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 0x7f66bc0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c1cc3640 1 -- 172.21.15.84:0/3249433728 >> v1:172.21.15.99:6789/0 conn(0x7f66bc0ca300 legacy=0x7f66bc0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.933+0000 7f66c1cc3640 1 -- 172.21.15.84:0/3249433728 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc071cc0 con 0x7f66bc15d4f0 2023-10-11T06:14:32.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.934+0000 7f66c1cc3640 1 --2- 172.21.15.84:0/3249433728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc0cb540 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cf6a0 tx=0x7f66ac02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e4e82ce2fc8741 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:32.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c0cc1640 1 -- 172.21.15.84:0/3249433728 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac02ca40 con 0x7f66bc15d4f0 2023-10-11T06:14:32.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c0cc1640 1 -- 172.21.15.84:0/3249433728 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac02cba0 con 0x7f66bc15d4f0 2023-10-11T06:14:32.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c0cc1640 1 -- 172.21.15.84:0/3249433728 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0343f0 con 0x7f66bc15d4f0 2023-10-11T06:14:32.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c3f27640 1 -- 172.21.15.84:0/3249433728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 msgr2=0x7f66bc0cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c3f27640 1 --2- 172.21.15.84:0/3249433728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc0cb540 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cf6a0 tx=0x7f66ac02bea0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c3f27640 1 -- 172.21.15.84:0/3249433728 shutdown_connections 2023-10-11T06:14:32.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c3f27640 1 --2- 172.21.15.84:0/3249433728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 0x7f66bc0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c3f27640 1 --2- 172.21.15.84:0/3249433728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc0cb540 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c3f27640 1 -- 172.21.15.84:0/3249433728 >> 172.21.15.84:0/3249433728 conn(0x7f66bc069c70 msgr2=0x7f66bc06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:32.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.935+0000 7f66c3f27640 1 -- 172.21.15.84:0/3249433728 shutdown_connections 2023-10-11T06:14:32.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.936+0000 7f66c3f27640 1 -- 172.21.15.84:0/3249433728 wait complete. 2023-10-11T06:14:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.937+0000 7f66c3f27640 1 Processor -- start 2023-10-11T06:14:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.937+0000 7f66c3f27640 1 -- start start 2023-10-11T06:14:32.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.938+0000 7f66c3f27640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 0x7f66bc0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.938+0000 7f66c3f27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66bc0ca300 0x7f66bc157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.938+0000 7f66c3f27640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc1f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.938+0000 7f66c3f27640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc149790 con 0x7f66bc15d4f0 2023-10-11T06:14:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.938+0000 7f66c3f27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc15bcb0 con 0x7f66bc0ca300 2023-10-11T06:14:32.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.938+0000 7f66c3f27640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0b2c20 con 0x7f66bc0c9360 2023-10-11T06:14:32.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc1f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc1f1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57386/0 (socket says 172.21.15.84:57386) 2023-10-11T06:14:32.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 -- 172.21.15.84:0/1423283725 learned_addr learned my addr 172.21.15.84:0/1423283725 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:32.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 -- 172.21.15.84:0/1423283725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 msgr2=0x7f66bc0b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 0x7f66bc0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c14c2640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66bc0ca300 0x7f66bc157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 -- 172.21.15.84:0/1423283725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66bc0ca300 msgr2=0x7f66bc157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:32.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66bc0ca300 0x7f66bc157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:32.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 -- 172.21.15.84:0/1423283725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009ad0 con 0x7f66bc15d4f0 2023-10-11T06:14:32.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.939+0000 7f66c24c4640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc1f1ea0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f66b400b440 tx=0x7f66b400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.940+0000 7f66ba7fc640 1 -- 172.21.15.84:0/1423283725 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4011830 con 0x7f66bc15d4f0 2023-10-11T06:14:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.940+0000 7f66ba7fc640 1 -- 172.21.15.84:0/1423283725 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b4011990 con 0x7f66bc15d4f0 2023-10-11T06:14:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.940+0000 7f66ba7fc640 1 -- 172.21.15.84:0/1423283725 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4011d40 con 0x7f66bc15d4f0 2023-10-11T06:14:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.940+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc14c8e0 con 0x7f66bc15d4f0 2023-10-11T06:14:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.940+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc15b280 con 0x7f66bc15d4f0 2023-10-11T06:14:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.941+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc073ed0 con 0x7f66bc15d4f0 2023-10-11T06:14:32.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.942+0000 7f66ba7fc640 1 -- 172.21.15.84:0/1423283725 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f66b401a400 con 0x7f66bc15d4f0 2023-10-11T06:14:32.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.943+0000 7f66ba7fc640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f66a806a830 0x7f66a806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:32.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.943+0000 7f66ba7fc640 1 -- 172.21.15.84:0/1423283725 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66b4089170 con 0x7f66bc15d4f0 2023-10-11T06:14:32.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.943+0000 7f66c1cc3640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f66a806a830 0x7f66a806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:32.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.944+0000 7f66c1cc3640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f66a806a830 0x7f66a806cce0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f66ac035000 tx=0x7f66ac02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:32.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:32.947+0000 7f66ba7fc640 1 -- 172.21.15.84:0/1423283725 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f66b4054350 con 0x7f66bc15d4f0 2023-10-11T06:14:33.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.138+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66bc0b09b0 con 0x7f66bc15d4f0 2023-10-11T06:14:33.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.140+0000 7f66ba7fc640 1 -- 172.21.15.84:0/1423283725 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7f66b4057970 con 0x7f66bc15d4f0 2023-10-11T06:14:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:33.142 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:33.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f66a806a830 msgr2=0x7f66a806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f66a806a830 0x7f66a806cce0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f66ac035000 tx=0x7f66ac02e000 comp rx=0 tx=0).stop 2023-10-11T06:14:33.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 msgr2=0x7f66bc1f1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc1f1ea0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f66b400b440 tx=0x7f66b400b900 comp rx=0 tx=0).stop 2023-10-11T06:14:33.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 shutdown_connections 2023-10-11T06:14:33.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f66a806a830 0x7f66a806cce0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66bc15d4f0 0x7f66bc1f1ea0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66bc0ca300 0x7f66bc157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 --2- 172.21.15.84:0/1423283725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f66bc0c9360 0x7f66bc0b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.143+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 >> 172.21.15.84:0/1423283725 conn(0x7f66bc069c70 msgr2=0x7f66bc0ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:33.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.144+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 shutdown_connections 2023-10-11T06:14:33.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.144+0000 7f66c3f27640 1 -- 172.21.15.84:0/1423283725 wait complete. 2023-10-11T06:14:33.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-11T06:11:00.844920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-11T06:11:06.509213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-11T06:13:44.841435+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-11T06:13:45.401366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-11T06:14:33.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-11T06:14:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.380+0000 7f6e9f07d640 1 Processor -- start 2023-10-11T06:14:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.380+0000 7f6e9f07d640 1 -- start start 2023-10-11T06:14:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.381+0000 7f6e9f07d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e980d2ac0 0x7f6e980d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e9f07d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e980cf220 con 0x7f6e980d0940 2023-10-11T06:14:33.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e9f07d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e980cf500 con 0x7f6e9815f7e0 2023-10-11T06:14:33.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e9f07d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e980d3480 con 0x7f6e980d2ac0 2023-10-11T06:14:33.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e97fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e980d2ac0 0x7f6e980d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e97fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e980d2ac0 0x7f6e980d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57392/0 (socket says 172.21.15.84:57392) 2023-10-11T06:14:33.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e97fff640 1 -- 172.21.15.84:0/1327005083 learned_addr learned my addr 172.21.15.84:0/1327005083 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:33.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e9d61a640 1 --1- 172.21.15.84:0/1327005083 >> v1:172.21.15.84:6789/0 conn(0x7f6e980d0940 0x7f6e980d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43562/0 (socket says 172.21.15.84:43562) 2023-10-11T06:14:33.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e97fff640 1 -- 172.21.15.84:0/1327005083 >> v1:172.21.15.99:6789/0 conn(0x7f6e9815f7e0 legacy=0x7f6e980cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:33.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e97fff640 1 -- 172.21.15.84:0/1327005083 >> v1:172.21.15.84:6789/0 conn(0x7f6e980d0940 legacy=0x7f6e980d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.382+0000 7f6e97fff640 1 -- 172.21.15.84:0/1327005083 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980cf7e0 con 0x7f6e980d2ac0 2023-10-11T06:14:33.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.383+0000 7f6e97fff640 1 --2- 172.21.15.84:0/1327005083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e980d2ac0 0x7f6e980d2f40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009870 tx=0x7f6e8c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a4e1ac37269352 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:33.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.384+0000 7f6e977fe640 1 -- 172.21.15.84:0/1327005083 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c030860 con 0x7f6e980d2ac0 2023-10-11T06:14:33.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.384+0000 7f6e977fe640 1 -- 172.21.15.84:0/1327005083 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c0309c0 con 0x7f6e980d2ac0 2023-10-11T06:14:33.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.384+0000 7f6e977fe640 1 -- 172.21.15.84:0/1327005083 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c030d50 con 0x7f6e980d2ac0 2023-10-11T06:14:33.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.384+0000 7f6e9f07d640 1 -- 172.21.15.84:0/1327005083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e980d2ac0 msgr2=0x7f6e980d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.384+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/1327005083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e980d2ac0 0x7f6e980d2f40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009870 tx=0x7f6e8c02b9a0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.385+0000 7f6e9f07d640 1 -- 172.21.15.84:0/1327005083 shutdown_connections 2023-10-11T06:14:33.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.385+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/1327005083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e980d2ac0 0x7f6e980d2f40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.385+0000 7f6e9f07d640 1 -- 172.21.15.84:0/1327005083 >> 172.21.15.84:0/1327005083 conn(0x7f6e980713a0 msgr2=0x7f6e980717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:33.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.385+0000 7f6e9f07d640 1 -- 172.21.15.84:0/1327005083 shutdown_connections 2023-10-11T06:14:33.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.385+0000 7f6e9f07d640 1 -- 172.21.15.84:0/1327005083 wait complete. 2023-10-11T06:14:33.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.386+0000 7f6e9f07d640 1 Processor -- start 2023-10-11T06:14:33.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.386+0000 7f6e9f07d640 1 -- start start 2023-10-11T06:14:33.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.387+0000 7f6e9f07d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e980d0940 0x7f6e98150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.387+0000 7f6e9f07d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d2ac0 0x7f6e980b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9f07d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815f7e0 0x7f6e9815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9f07d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980b60c0 con 0x7f6e9815f7e0 2023-10-11T06:14:33.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9f07d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980c6990 con 0x7f6e980d2ac0 2023-10-11T06:14:33.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9f07d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980c6c70 con 0x7f6e980d0940 2023-10-11T06:14:33.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815f7e0 0x7f6e9815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815f7e0 0x7f6e9815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57402/0 (socket says 172.21.15.84:57402) 2023-10-11T06:14:33.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 -- 172.21.15.84:0/3829932224 learned_addr learned my addr 172.21.15.84:0/3829932224 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:33.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9ce19640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e980d0940 0x7f6e98150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e97fff640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d2ac0 0x7f6e980b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 -- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e980d0940 msgr2=0x7f6e98150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e980d0940 0x7f6e98150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 -- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d2ac0 msgr2=0x7f6e980b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d2ac0 0x7f6e980b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e9d61a640 1 -- 172.21.15.84:0/3829932224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c0094d0 con 0x7f6e9815f7e0 2023-10-11T06:14:33.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.388+0000 7f6e97fff640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d2ac0 0x7f6e980b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:33.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.389+0000 7f6e9d61a640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815f7e0 0x7f6e9815e450 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c400 tx=0x7f6e9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.389+0000 7f6e957fa640 1 -- 172.21.15.84:0/3829932224 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900074e0 con 0x7f6e9815f7e0 2023-10-11T06:14:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.389+0000 7f6e957fa640 1 -- 172.21.15.84:0/3829932224 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e90007640 con 0x7f6e9815f7e0 2023-10-11T06:14:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.389+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e980c6f50 con 0x7f6e9815f7e0 2023-10-11T06:14:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.389+0000 7f6e957fa640 1 -- 172.21.15.84:0/3829932224 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900049e0 con 0x7f6e9815f7e0 2023-10-11T06:14:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.389+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e980c7510 con 0x7f6e9815f7e0 2023-10-11T06:14:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.390+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e980d1d70 con 0x7f6e9815f7e0 2023-10-11T06:14:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.391+0000 7f6e957fa640 1 -- 172.21.15.84:0/3829932224 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f6e90004b40 con 0x7f6e9815f7e0 2023-10-11T06:14:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.392+0000 7f6e957fa640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6e8406a830 0x7f6e8406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.392+0000 7f6e957fa640 1 -- 172.21.15.84:0/3829932224 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e90088a30 con 0x7f6e9815f7e0 2023-10-11T06:14:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.392+0000 7f6e9ce19640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6e8406a830 0x7f6e8406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.393+0000 7f6e9ce19640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6e8406a830 0x7f6e8406cce0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009f60 tx=0x7f6e880094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.396+0000 7f6e957fa640 1 -- 172.21.15.84:0/3829932224 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f6e90053ca0 con 0x7f6e9815f7e0 2023-10-11T06:14:33.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.583+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e98073ed0 con 0x7f6e9815f7e0 2023-10-11T06:14:33.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.584+0000 7f6e957fa640 1 -- 172.21.15.84:0/3829932224 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e900572c0 con 0x7f6e9815f7e0 2023-10-11T06:14:33.585 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-10-11T06:14:33.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.586+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6e8406a830 msgr2=0x7f6e8406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.586+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6e8406a830 0x7f6e8406cce0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009f60 tx=0x7f6e880094f0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.586+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815f7e0 msgr2=0x7f6e9815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.586+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815f7e0 0x7f6e9815e450 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c400 tx=0x7f6e9000c8c0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.586+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 shutdown_connections 2023-10-11T06:14:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.586+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f6e8406a830 0x7f6e8406cce0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.587+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e9815f7e0 0x7f6e9815e450 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.587+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d2ac0 0x7f6e980b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.587+0000 7f6e9f07d640 1 --2- 172.21.15.84:0/3829932224 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e980d0940 0x7f6e98150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.587+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 >> 172.21.15.84:0/3829932224 conn(0x7f6e980713a0 msgr2=0x7f6e9815fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.587+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 shutdown_connections 2023-10-11T06:14:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.587+0000 7f6e9f07d640 1 -- 172.21.15.84:0/3829932224 wait complete. 2023-10-11T06:14:33.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-11T06:14:33.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.818+0000 7f79df350640 1 Processor -- start 2023-10-11T06:14:33.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.818+0000 7f79df350640 1 -- start start 2023-10-11T06:14:33.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.819+0000 7f79df350640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79d80b2a00 0x7f79d8165090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.819+0000 7f79df350640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80ccec0 con 0x7f79d80cc820 2023-10-11T06:14:33.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79df350640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80ce5e0 con 0x7f79d815a1f0 2023-10-11T06:14:33.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79df350640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80cb280 con 0x7f79d80b2a00 2023-10-11T06:14:33.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79dc8eb640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f79d80cc820 0x7f79d80b22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43576/0 (socket says 172.21.15.84:43576) 2023-10-11T06:14:33.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79dc8eb640 1 -- 172.21.15.84:0/2264558408 learned_addr learned my addr 172.21.15.84:0/2264558408 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:33.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79dd8ed640 1 --2- 172.21.15.84:0/2264558408 >> v2:172.21.15.84:3300/0 conn(0x7f79d80b2a00 0x7f79d8165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79d7fff640 1 -- 172.21.15.84:0/2264558408 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490564996 0 0) 0x7f79d80ccec0 con 0x7f79d80cc820 2023-10-11T06:14:33.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79dd8ed640 1 -- 172.21.15.84:0/2264558408 >> v1:172.21.15.99:6789/0 conn(0x7f79d815a1f0 legacy=0x7f79d815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:33.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79dd8ed640 1 -- 172.21.15.84:0/2264558408 >> v1:172.21.15.84:6789/0 conn(0x7f79d80cc820 legacy=0x7f79d80b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.820+0000 7f79dd8ed640 1 -- 172.21.15.84:0/2264558408 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80ce8c0 con 0x7f79d80b2a00 2023-10-11T06:14:33.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.822+0000 7f79dd8ed640 1 --2- 172.21.15.84:0/2264558408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d80b2a00 0x7f79d8165090 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f79d00097a0 tx=0x7f79d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2233aa7b532cdeef server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:33.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.822+0000 7f79d7fff640 1 -- 172.21.15.84:0/2264558408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0031830 con 0x7f79d80b2a00 2023-10-11T06:14:33.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.822+0000 7f79d7fff640 1 -- 172.21.15.84:0/2264558408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0031990 con 0x7f79d80b2a00 2023-10-11T06:14:33.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.822+0000 7f79d7fff640 1 -- 172.21.15.84:0/2264558408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0031d20 con 0x7f79d80b2a00 2023-10-11T06:14:33.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.822+0000 7f79df350640 1 -- 172.21.15.84:0/2264558408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d80b2a00 msgr2=0x7f79d8165090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.822+0000 7f79df350640 1 --2- 172.21.15.84:0/2264558408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d80b2a00 0x7f79d8165090 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f79d00097a0 tx=0x7f79d002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.823+0000 7f79df350640 1 -- 172.21.15.84:0/2264558408 shutdown_connections 2023-10-11T06:14:33.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.823+0000 7f79df350640 1 --2- 172.21.15.84:0/2264558408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d80b2a00 0x7f79d8165090 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.823+0000 7f79df350640 1 -- 172.21.15.84:0/2264558408 >> 172.21.15.84:0/2264558408 conn(0x7f79d806ef20 msgr2=0x7f79d80cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:33.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.823+0000 7f79df350640 1 -- 172.21.15.84:0/2264558408 shutdown_connections 2023-10-11T06:14:33.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.823+0000 7f79df350640 1 -- 172.21.15.84:0/2264558408 wait complete. 2023-10-11T06:14:33.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.825+0000 7f79df350640 1 Processor -- start 2023-10-11T06:14:33.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.825+0000 7f79df350640 1 -- start start 2023-10-11T06:14:33.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79df350640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79d80b2a00 0x7f79d80c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79df350640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79d80cc820 0x7f79d80c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79df350640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d815a1f0 0x7f79d8155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79df350640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80b65a0 con 0x7f79d815a1f0 2023-10-11T06:14:33.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79df350640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80ca860 con 0x7f79d80cc820 2023-10-11T06:14:33.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79df350640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80b37d0 con 0x7f79d80b2a00 2023-10-11T06:14:33.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd8ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d815a1f0 0x7f79d8155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd8ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d815a1f0 0x7f79d8155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57420/0 (socket says 172.21.15.84:57420) 2023-10-11T06:14:33.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd8ed640 1 -- 172.21.15.84:0/369864712 learned_addr learned my addr 172.21.15.84:0/369864712 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:33.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd0ec640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79d80b2a00 0x7f79d80c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dc8eb640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79d80cc820 0x7f79d80c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd8ed640 1 -- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79d80b2a00 msgr2=0x7f79d80c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd8ed640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79d80b2a00 0x7f79d80c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd8ed640 1 -- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79d80cc820 msgr2=0x7f79d80c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:33.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.826+0000 7f79dd8ed640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79d80cc820 0x7f79d80c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:33.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.827+0000 7f79dd8ed640 1 -- 172.21.15.84:0/369864712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0009400 con 0x7f79d815a1f0 2023-10-11T06:14:33.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.827+0000 7f79dd8ed640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d815a1f0 0x7f79d8155880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f79d00098c0 tx=0x7f79d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:33.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.827+0000 7f79d5ffb640 1 -- 172.21.15.84:0/369864712 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d003b9f0 con 0x7f79d815a1f0 2023-10-11T06:14:33.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.827+0000 7f79d5ffb640 1 -- 172.21.15.84:0/369864712 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d003bb50 con 0x7f79d815a1f0 2023-10-11T06:14:33.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.827+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d80ca580 con 0x7f79d815a1f0 2023-10-11T06:14:33.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.827+0000 7f79d5ffb640 1 -- 172.21.15.84:0/369864712 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00453f0 con 0x7f79d815a1f0 2023-10-11T06:14:33.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.827+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d80b3d50 con 0x7f79d815a1f0 2023-10-11T06:14:33.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.828+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d80b3f40 con 0x7f79d815a1f0 2023-10-11T06:14:33.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.829+0000 7f79d5ffb640 1 -- 172.21.15.84:0/369864712 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f79d0045550 con 0x7f79d815a1f0 2023-10-11T06:14:33.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.830+0000 7f79d5ffb640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f79bc06a830 0x7f79bc06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:33.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.830+0000 7f79d5ffb640 1 -- 172.21.15.84:0/369864712 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79d00ab3b0 con 0x7f79d815a1f0 2023-10-11T06:14:33.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.831+0000 7f79dd0ec640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f79bc06a830 0x7f79bc06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:33.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.831+0000 7f79dd0ec640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f79bc06a830 0x7f79bc06cce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009f20 tx=0x7f79cc0094d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:33.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:33.835+0000 7f79d5ffb640 1 -- 172.21.15.84:0/369864712 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f79d0076740 con 0x7f79d815a1f0 2023-10-11T06:14:34.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.027+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79d8067090 con 0x7f79d815a1f0 2023-10-11T06:14:34.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.027+0000 7f79d5ffb640 1 -- 172.21.15.84:0/369864712 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f79d0079d60 con 0x7f79d815a1f0 2023-10-11T06:14:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-10-11T06:14:34.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f79bc06a830 msgr2=0x7f79bc06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f79bc06a830 0x7f79bc06cce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009f20 tx=0x7f79cc0094d0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d815a1f0 msgr2=0x7f79d8155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d815a1f0 0x7f79d8155880 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f79d00098c0 tx=0x7f79d00046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 shutdown_connections 2023-10-11T06:14:34.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f79bc06a830 0x7f79bc06cce0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d815a1f0 0x7f79d8155880 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79d80cc820 0x7f79d80c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 --2- 172.21.15.84:0/369864712 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79d80b2a00 0x7f79d80c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 >> 172.21.15.84:0/369864712 conn(0x7f79d806ef20 msgr2=0x7f79d80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:34.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 shutdown_connections 2023-10-11T06:14:34.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.030+0000 7f79df350640 1 -- 172.21.15.84:0/369864712 wait complete. 2023-10-11T06:14:34.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-11T06:14:34.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.267+0000 7f978dc23640 1 Processor -- start 2023-10-11T06:14:34.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.267+0000 7f978dc23640 1 -- start start 2023-10-11T06:14:34.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.268+0000 7f978dc23640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 0x7f97880d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.268+0000 7f978dc23640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97880cf220 con 0x7f978815f7e0 2023-10-11T06:14:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.268+0000 7f978dc23640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97880cf500 con 0x7f97880d2ac0 2023-10-11T06:14:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.268+0000 7f978dc23640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cf7e0 con 0x7f97880d2f40 2023-10-11T06:14:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97877fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f978815f7e0 0x7f97880cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43588/0 (socket says 172.21.15.84:43588) 2023-10-11T06:14:34.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97877fe640 1 -- 172.21.15.84:0/1403770666 learned_addr learned my addr 172.21.15.84:0/1403770666 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:34.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093164395 0 0) 0x7f97880cf220 con 0x7f978815f7e0 2023-10-11T06:14:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f9787fff640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 0x7f97880d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9770003520 con 0x7f978815f7e0 2023-10-11T06:14:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519454667 0 0) 0x7f9770003520 con 0x7f978815f7e0 2023-10-11T06:14:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 msgr2=0x7f97880d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 0x7f97880d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 >> v1:172.21.15.99:6789/0 conn(0x7f97880d2ac0 legacy=0x7f97880d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97880cfac0 con 0x7f978815f7e0 2023-10-11T06:14:34.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f9778001670 con 0x7f978815f7e0 2023-10-11T06:14:34.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 >> v1:172.21.15.84:6789/0 conn(0x7f978815f7e0 legacy=0x7f97880cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 0x7f97700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.269+0000 7f97867fc640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9770003fd0 0x7f97700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f97867fc640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9770004d00 0x7f9770005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cf7e0 con 0x7f9770003fd0 2023-10-11T06:14:34.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977000bb10 con 0x7f9770004d00 2023-10-11T06:14:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977000bcd0 con 0x7f97880d2f40 2023-10-11T06:14:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f9786ffd640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9770003fd0 0x7f97700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f97877fe640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 0x7f97700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f9787fff640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9770004d00 0x7f9770005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f9786ffd640 1 -- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 msgr2=0x7f97700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f9786ffd640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 0x7f97700039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f9786ffd640 1 -- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9770004d00 msgr2=0x7f9770005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f9786ffd640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9770004d00 0x7f9770005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f9786ffd640 1 -- 172.21.15.84:0/1403770666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770003520 con 0x7f9770003fd0 2023-10-11T06:14:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.270+0000 7f97877fe640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97880d2f40 0x7f97700039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.271+0000 7f9786ffd640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9770003fd0 0x7f97700047c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f977c001750 tx=0x7f97700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9b33d26e6517ca server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.271+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0052d0 con 0x7f9770003fd0 2023-10-11T06:14:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c02e050 con 0x7f9770003fd0 2023-10-11T06:14:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f97867fc640 1 -- 172.21.15.84:0/1403770666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c003830 con 0x7f9770003fd0 2023-10-11T06:14:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f978dc23640 1 -- 172.21.15.84:0/1403770666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9770003fd0 msgr2=0x7f97700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f978dc23640 1 --2- 172.21.15.84:0/1403770666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9770003fd0 0x7f97700047c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f977c001750 tx=0x7f97700057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f9787fff640 1 -- 172.21.15.84:0/1403770666 reap_dead start 2023-10-11T06:14:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f978dc23640 1 -- 172.21.15.84:0/1403770666 shutdown_connections 2023-10-11T06:14:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f978dc23640 1 -- 172.21.15.84:0/1403770666 >> 172.21.15.84:0/1403770666 conn(0x7f97880713a0 msgr2=0x7f97880717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f978dc23640 1 -- 172.21.15.84:0/1403770666 shutdown_connections 2023-10-11T06:14:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.272+0000 7f978dc23640 1 -- 172.21.15.84:0/1403770666 wait complete. 2023-10-11T06:14:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.274+0000 7f978dc23640 1 Processor -- start 2023-10-11T06:14:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.274+0000 7f978dc23640 1 -- start start 2023-10-11T06:14:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f978dc23640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9770003fd0 0x7f97880b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f978dc23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f978814df10 0x7f97880cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f978dc23640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97880cca30 0x7f97880d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f978dc23640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cb200 con 0x7f97880cca30 2023-10-11T06:14:34.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f978dc23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cb450 con 0x7f978814df10 2023-10-11T06:14:34.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f978dc23640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cb730 con 0x7f9770003fd0 2023-10-11T06:14:34.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f9787fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97880cca30 0x7f97880d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f9787fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97880cca30 0x7f97880d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57430/0 (socket says 172.21.15.84:57430) 2023-10-11T06:14:34.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.275+0000 7f9787fff640 1 -- 172.21.15.84:0/1489392328 learned_addr learned my addr 172.21.15.84:0/1489392328 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:34.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f97877fe640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9770003fd0 0x7f97880b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f9787fff640 1 -- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9770003fd0 msgr2=0x7f97880b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f9787fff640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9770003fd0 0x7f97880b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f9787fff640 1 -- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f978814df10 msgr2=0x7f97880cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f9786ffd640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f978814df10 0x7f97880cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f9787fff640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f978814df10 0x7f97880cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f9787fff640 1 -- 172.21.15.84:0/1489392328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97880cba10 con 0x7f97880cca30 2023-10-11T06:14:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f97877fe640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9770003fd0 0x7f97880b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f9787fff640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97880cca30 0x7f97880d0f00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f97880b5720 tx=0x7f9780009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.276+0000 7f976ffff640 1 -- 172.21.15.84:0/1489392328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780014600 con 0x7f97880cca30 2023-10-11T06:14:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.277+0000 7f976ffff640 1 -- 172.21.15.84:0/1489392328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9780014780 con 0x7f97880cca30 2023-10-11T06:14:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.277+0000 7f976ffff640 1 -- 172.21.15.84:0/1489392328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780014b30 con 0x7f97880cca30 2023-10-11T06:14:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.277+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97880cbcf0 con 0x7f97880cca30 2023-10-11T06:14:34.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.277+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97880b5ea0 con 0x7f97880cca30 2023-10-11T06:14:34.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.278+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97880cc210 con 0x7f97880cca30 2023-10-11T06:14:34.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.279+0000 7f976ffff640 1 -- 172.21.15.84:0/1489392328 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f978001d3b0 con 0x7f97880cca30 2023-10-11T06:14:34.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.280+0000 7f976ffff640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f977406a830 0x7f977406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.280+0000 7f976ffff640 1 -- 172.21.15.84:0/1489392328 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9780088980 con 0x7f97880cca30 2023-10-11T06:14:34.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.280+0000 7f97877fe640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f977406a830 0x7f977406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.281+0000 7f97877fe640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f977406a830 0x7f977406cce0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f97780079d0 tx=0x7f9778009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:34.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.284+0000 7f976ffff640 1 -- 172.21.15.84:0/1489392328 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f97880cc210 con 0x7f97880cca30 2023-10-11T06:14:34.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.479+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97880cc210 con 0x7f97880cca30 2023-10-11T06:14:34.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.480+0000 7f976ffff640 1 -- 172.21.15.84:0/1489392328 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97880cc210 con 0x7f97880cca30 2023-10-11T06:14:34.482 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-10-11T06:14:34.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f977406a830 msgr2=0x7f977406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f977406a830 0x7f977406cce0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f97780079d0 tx=0x7f9778009040 comp rx=0 tx=0).stop 2023-10-11T06:14:34.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97880cca30 msgr2=0x7f97880d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97880cca30 0x7f97880d0f00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f97880b5720 tx=0x7f9780009980 comp rx=0 tx=0).stop 2023-10-11T06:14:34.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 shutdown_connections 2023-10-11T06:14:34.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f977406a830 0x7f977406cce0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97880cca30 0x7f97880d0f00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f978814df10 0x7f97880cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 --2- 172.21.15.84:0/1489392328 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9770003fd0 0x7f97880b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 >> 172.21.15.84:0/1489392328 conn(0x7f97880713a0 msgr2=0x7f978815fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 shutdown_connections 2023-10-11T06:14:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.484+0000 7f978dc23640 1 -- 172.21.15.84:0/1489392328 wait complete. 2023-10-11T06:14:34.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-11T06:14:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.720+0000 7f7914d3c640 1 Processor -- start 2023-10-11T06:14:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.720+0000 7f7914d3c640 1 -- start start 2023-10-11T06:14:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7914d3c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79100d2ac0 0x7f79100d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7914d3c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 0x7f79100d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7914d3c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79100cf220 con 0x7f791015f7e0 2023-10-11T06:14:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7914d3c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79100d3480 con 0x7f79100d2ac0 2023-10-11T06:14:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7914d3c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79100cf500 con 0x7f79100d0940 2023-10-11T06:14:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7907fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79100d2ac0 0x7f79100d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7907fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79100d2ac0 0x7f79100d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57444/0 (socket says 172.21.15.84:57444) 2023-10-11T06:14:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.722+0000 7f7907fff640 1 -- 172.21.15.84:0/2783728761 learned_addr learned my addr 172.21.15.84:0/2783728761 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.723+0000 7f7907fff640 1 -- 172.21.15.84:0/2783728761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 msgr2=0x7f79100d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.723+0000 7f7907fff640 1 --2- 172.21.15.84:0/2783728761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 0x7f79100d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.723+0000 7f7907fff640 1 -- 172.21.15.84:0/2783728761 >> v1:172.21.15.99:6789/0 conn(0x7f791015f7e0 legacy=0x7f79100cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:34.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.723+0000 7f7907fff640 1 -- 172.21.15.84:0/2783728761 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79100cf7e0 con 0x7f79100d2ac0 2023-10-11T06:14:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.724+0000 7f7907fff640 1 --2- 172.21.15.84:0/2783728761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79100d2ac0 0x7f79100d2f40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009870 tx=0x7f78f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2144b35bc76f99 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.724+0000 7f790dd9b640 1 -- 172.21.15.84:0/2783728761 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8031640 con 0x7f79100d2ac0 2023-10-11T06:14:34.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.724+0000 7f790dd9b640 1 -- 172.21.15.84:0/2783728761 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f80317a0 con 0x7f79100d2ac0 2023-10-11T06:14:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.724+0000 7f790dd9b640 1 -- 172.21.15.84:0/2783728761 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8031b30 con 0x7f79100d2ac0 2023-10-11T06:14:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 -- 172.21.15.84:0/2783728761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79100d2ac0 msgr2=0x7f79100d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 --2- 172.21.15.84:0/2783728761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79100d2ac0 0x7f79100d2f40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009870 tx=0x7f78f802b8f0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 -- 172.21.15.84:0/2783728761 shutdown_connections 2023-10-11T06:14:34.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 --2- 172.21.15.84:0/2783728761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 0x7f79100d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 --2- 172.21.15.84:0/2783728761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79100d2ac0 0x7f79100d2f40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 -- 172.21.15.84:0/2783728761 >> 172.21.15.84:0/2783728761 conn(0x7f79100713a0 msgr2=0x7f79100717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 -- 172.21.15.84:0/2783728761 shutdown_connections 2023-10-11T06:14:34.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.725+0000 7f7914d3c640 1 -- 172.21.15.84:0/2783728761 wait complete. 2023-10-11T06:14:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.727+0000 7f7914d3c640 1 Processor -- start 2023-10-11T06:14:34.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.727+0000 7f7914d3c640 1 -- start start 2023-10-11T06:14:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f7914d3c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 0x7f791014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f7914d3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79100d2ac0 0x7f79100b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f7914d3c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f791015f7e0 0x7f791015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f7914d3c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79100b7fe0 con 0x7f791015f7e0 2023-10-11T06:14:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f7914d3c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791015b220 con 0x7f79100d2ac0 2023-10-11T06:14:34.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f7914d3c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79100b8860 con 0x7f79100d0940 2023-10-11T06:14:34.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f791015f7e0 0x7f791015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f791015f7e0 0x7f791015ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57450/0 (socket says 172.21.15.84:57450) 2023-10-11T06:14:34.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 -- 172.21.15.84:0/674617115 learned_addr learned my addr 172.21.15.84:0/674617115 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:34.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f7907fff640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79100d2ac0 0x7f79100b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790e59c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 0x7f791014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 -- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 msgr2=0x7f791014df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 0x7f791014df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 -- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79100d2ac0 msgr2=0x7f79100b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79100d2ac0 0x7f79100b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.728+0000 7f790ed9d640 1 -- 172.21.15.84:0/674617115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80094d0 con 0x7f791015f7e0 2023-10-11T06:14:34.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.729+0000 7f7907fff640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79100d2ac0 0x7f79100b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.729+0000 7f790ed9d640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f791015f7e0 0x7f791015ce60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f790800b970 tx=0x7f790800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.729+0000 7f7906ffd640 1 -- 172.21.15.84:0/674617115 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800cb40 con 0x7f791015f7e0 2023-10-11T06:14:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.729+0000 7f7906ffd640 1 -- 172.21.15.84:0/674617115 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790800ccc0 con 0x7f791015f7e0 2023-10-11T06:14:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.729+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791015af40 con 0x7f791015f7e0 2023-10-11T06:14:34.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.729+0000 7f7906ffd640 1 -- 172.21.15.84:0/674617115 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908014680 con 0x7f791015f7e0 2023-10-11T06:14:34.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.729+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79100b84f0 con 0x7f791015f7e0 2023-10-11T06:14:34.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.730+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7910073ed0 con 0x7f791015f7e0 2023-10-11T06:14:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.731+0000 7f7906ffd640 1 -- 172.21.15.84:0/674617115 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f790801d3b0 con 0x7f791015f7e0 2023-10-11T06:14:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.732+0000 7f7906ffd640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78f006a830 0x7f78f006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.732+0000 7f7906ffd640 1 -- 172.21.15.84:0/674617115 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7908088f30 con 0x7f791015f7e0 2023-10-11T06:14:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.732+0000 7f790e59c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78f006a830 0x7f78f006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:34.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.733+0000 7f790e59c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78f006a830 0x7f78f006cce0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f79101609d0 tx=0x7f7900008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:34.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.736+0000 7f7906ffd640 1 -- 172.21.15.84:0/674617115 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f79080541a0 con 0x7f791015f7e0 2023-10-11T06:14:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.927+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79100b5030 con 0x7f791015f7e0 2023-10-11T06:14:34.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.928+0000 7f7906ffd640 1 -- 172.21.15.84:0/674617115 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7908019020 con 0x7f791015f7e0 2023-10-11T06:14:34.930 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-10-11T06:14:34.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78f006a830 msgr2=0x7f78f006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78f006a830 0x7f78f006cce0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f79101609d0 tx=0x7f7900008040 comp rx=0 tx=0).stop 2023-10-11T06:14:34.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f791015f7e0 msgr2=0x7f791015ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:34.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f791015f7e0 0x7f791015ce60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f790800b970 tx=0x7f790800be30 comp rx=0 tx=0).stop 2023-10-11T06:14:34.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 shutdown_connections 2023-10-11T06:14:34.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f78f006a830 0x7f78f006cce0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f791015f7e0 0x7f791015ce60 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79100d2ac0 0x7f79100b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 --2- 172.21.15.84:0/674617115 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79100d0940 0x7f791014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:34.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 >> 172.21.15.84:0/674617115 conn(0x7f79100713a0 msgr2=0x7f79100d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:34.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 shutdown_connections 2023-10-11T06:14:34.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:34.931+0000 7f7914d3c640 1 -- 172.21.15.84:0/674617115 wait complete. 2023-10-11T06:14:34.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:14:35.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.158+0000 7ff09b149640 1 Processor -- start 2023-10-11T06:14:35.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.159+0000 7ff09b149640 1 -- start start 2023-10-11T06:14:35.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.160+0000 7ff09b149640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff09415c330 0x7ff0940b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.160+0000 7ff09b149640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09415c700 con 0x7ff0940d0b00 2023-10-11T06:14:35.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.160+0000 7ff09b149640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0940b02a0 con 0x7ff0940afe10 2023-10-11T06:14:35.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.160+0000 7ff09b149640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0940b0f20 con 0x7ff09415c330 2023-10-11T06:14:35.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff09a147640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff09415c330 0x7ff0940b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff09a147640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff09415c330 0x7ff0940b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57460/0 (socket says 172.21.15.84:57460) 2023-10-11T06:14:35.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff09a147640 1 -- 172.21.15.84:0/14457856 learned_addr learned my addr 172.21.15.84:0/14457856 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:35.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff09a948640 1 --1- 172.21.15.84:0/14457856 >> v1:172.21.15.84:6789/0 conn(0x7ff0940d0b00 0x7ff0940cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43600/0 (socket says 172.21.15.84:43600) 2023-10-11T06:14:35.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff09a147640 1 -- 172.21.15.84:0/14457856 >> v1:172.21.15.99:6789/0 conn(0x7ff0940afe10 legacy=0x7ff0940d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:35.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff09a147640 1 -- 172.21.15.84:0/14457856 >> v1:172.21.15.84:6789/0 conn(0x7ff0940d0b00 legacy=0x7ff0940cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff099145640 1 -- 172.21.15.84:0/14457856 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65098551 0 0) 0x7ff08c001490 con 0x7ff0940d0b00 2023-10-11T06:14:35.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.161+0000 7ff09a147640 1 -- 172.21.15.84:0/14457856 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940b0580 con 0x7ff09415c330 2023-10-11T06:14:35.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.162+0000 7ff09a147640 1 --2- 172.21.15.84:0/14457856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff0940b1340 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff090009850 tx=0x7ff09002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd5279a9ee535ca server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:35.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff099145640 1 -- 172.21.15.84:0/14457856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090004230 con 0x7ff09415c330 2023-10-11T06:14:35.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff099145640 1 -- 172.21.15.84:0/14457856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090004390 con 0x7ff09415c330 2023-10-11T06:14:35.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff099145640 1 -- 172.21.15.84:0/14457856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090003810 con 0x7ff09415c330 2023-10-11T06:14:35.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff09b149640 1 -- 172.21.15.84:0/14457856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 msgr2=0x7ff0940b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff09b149640 1 --2- 172.21.15.84:0/14457856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff0940b1340 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff090009850 tx=0x7ff09002b860 comp rx=0 tx=0).stop 2023-10-11T06:14:35.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff09b149640 1 -- 172.21.15.84:0/14457856 shutdown_connections 2023-10-11T06:14:35.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff09b149640 1 --2- 172.21.15.84:0/14457856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff0940b1340 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff09b149640 1 -- 172.21.15.84:0/14457856 >> 172.21.15.84:0/14457856 conn(0x7ff09409bb10 msgr2=0x7ff0940d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:35.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff09b149640 1 -- 172.21.15.84:0/14457856 shutdown_connections 2023-10-11T06:14:35.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.163+0000 7ff09b149640 1 -- 172.21.15.84:0/14457856 wait complete. 2023-10-11T06:14:35.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.165+0000 7ff09b149640 1 Processor -- start 2023-10-11T06:14:35.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.165+0000 7ff09b149640 1 -- start start 2023-10-11T06:14:35.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.166+0000 7ff09b149640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0940afe10 0x7ff0941e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.166+0000 7ff09b149640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0940d0b00 0x7ff09406ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.166+0000 7ff09b149640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff09406b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.166+0000 7ff09b149640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff094152a20 con 0x7ff09415c330 2023-10-11T06:14:35.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.166+0000 7ff09b149640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff094154430 con 0x7ff0940afe10 2023-10-11T06:14:35.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.166+0000 7ff09b149640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0941e3ad0 con 0x7ff0940d0b00 2023-10-11T06:14:35.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff09406b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff09406b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57476/0 (socket says 172.21.15.84:57476) 2023-10-11T06:14:35.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 -- 172.21.15.84:0/3597516781 learned_addr learned my addr 172.21.15.84:0/3597516781 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:35.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a147640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0940afe10 0x7ff0941e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff099946640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0940d0b00 0x7ff09406ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 -- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0940d0b00 msgr2=0x7ff09406ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0940d0b00 0x7ff09406ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 -- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0940afe10 msgr2=0x7ff0941e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0940afe10 0x7ff0941e4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff09a948640 1 -- 172.21.15.84:0/3597516781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff090009460 con 0x7ff09415c330 2023-10-11T06:14:35.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.167+0000 7ff099946640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0940d0b00 0x7ff09406ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:35.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.168+0000 7ff09a948640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff09406b2a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c730 tx=0x7ff08c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:35.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.168+0000 7ff08affd640 1 -- 172.21.15.84:0/3597516781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00ce40 con 0x7ff09415c330 2023-10-11T06:14:35.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.168+0000 7ff08affd640 1 -- 172.21.15.84:0/3597516781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c00a640 con 0x7ff09415c330 2023-10-11T06:14:35.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.168+0000 7ff08affd640 1 -- 172.21.15.84:0/3597516781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00aa70 con 0x7ff09415c330 2023-10-11T06:14:35.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.168+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0941e3db0 con 0x7ff09415c330 2023-10-11T06:14:35.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.168+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0941e4370 con 0x7ff09415c330 2023-10-11T06:14:35.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.169+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09404e890 con 0x7ff09415c330 2023-10-11T06:14:35.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.170+0000 7ff08affd640 1 -- 172.21.15.84:0/3597516781 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff08c00abd0 con 0x7ff09415c330 2023-10-11T06:14:35.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.171+0000 7ff08affd640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff07806a830 0x7ff07806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.171+0000 7ff08affd640 1 -- 172.21.15.84:0/3597516781 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff08c088370 con 0x7ff09415c330 2023-10-11T06:14:35.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.171+0000 7ff09a147640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff07806a830 0x7ff07806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.172+0000 7ff09a147640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff07806a830 0x7ff07806cce0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff090034000 tx=0x7ff090008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:35.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.176+0000 7ff08affd640 1 -- 172.21.15.84:0/3597516781 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff08c0535e0 con 0x7ff09415c330 2023-10-11T06:14:35.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.404+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff09415c700 con 0x7ff09415c330 2023-10-11T06:14:35.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.405+0000 7ff08affd640 1 -- 172.21.15.84:0/3597516781 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4648 (secure 0 0 0) 0x7ff09415c700 con 0x7ff09415c330 2023-10-11T06:14:35.406 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:35.407 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1835716973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1835716973},{"type":"v1","addr":"172.21.15.99:6833","nonce":1835716973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1415879342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1415879342},{"type":"v1","addr":"172.21.15.99:6835","nonce":1415879342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2122307729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2122307729},{"type":"v1","addr":"172.21.15.84:6836","nonce":2122307729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:13:46.389013+0000","modified":"2023-10-11T06:13:49.406181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6027":{"gid":6027,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2048038445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2048038445},{"type":"v1","addr":"172.21.15.84:6837","nonce":2048038445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T06:14:35.407 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-10-11T06:14:35.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff07806a830 msgr2=0x7ff07806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff07806a830 0x7ff07806cce0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff090034000 tx=0x7ff090008030 comp rx=0 tx=0).stop 2023-10-11T06:14:35.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 msgr2=0x7ff09406b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff09406b2a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c730 tx=0x7ff08c00cbf0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 shutdown_connections 2023-10-11T06:14:35.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff07806a830 0x7ff07806cce0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff09415c330 0x7ff09406b2a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0940d0b00 0x7ff09406ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 --2- 172.21.15.84:0/3597516781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0940afe10 0x7ff0941e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 >> 172.21.15.84:0/3597516781 conn(0x7ff09409bb10 msgr2=0x7ff0940ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:35.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 shutdown_connections 2023-10-11T06:14:35.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.408+0000 7ff09b149640 1 -- 172.21.15.84:0/3597516781 wait complete. 2023-10-11T06:14:35.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:35.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.638+0000 7fa4fca84640 1 Processor -- start 2023-10-11T06:14:35.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.638+0000 7fa4fca84640 1 -- start start 2023-10-11T06:14:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.639+0000 7fa4fca84640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4f815c630 0x7fa4f80cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.639+0000 7fa4fca84640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4f806ad80 0x7fa4f806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.639+0000 7fa4fca84640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f80b13b0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.639+0000 7fa4fca84640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f806b740 con 0x7fa4f806ad80 2023-10-11T06:14:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.639+0000 7fa4fca84640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f80b0030 con 0x7fa4f815c630 2023-10-11T06:14:35.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.640+0000 7fa4f659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4f815c630 0x7fa4f80cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.640+0000 7fa4f5d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4f806ad80 0x7fa4f806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.640+0000 7fa4f659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4f815c630 0x7fa4f80cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:33858/0 (socket says 172.21.15.84:33858) 2023-10-11T06:14:35.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.640+0000 7fa4f659c640 1 -- 172.21.15.84:0/1108667300 learned_addr learned my addr 172.21.15.84:0/1108667300 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:35.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.641+0000 7fa4f659c640 1 -- 172.21.15.84:0/1108667300 >> v2:172.21.15.84:3300/0 conn(0x7fa4f806ad80 msgr2=0x7fa4f806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:35.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.641+0000 7fa4f559a640 1 -- 172.21.15.84:0/1108667300 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885207264 0 0) 0x7fa4f80b13b0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.641+0000 7fa4f659c640 1 --2- 172.21.15.84:0/1108667300 >> v2:172.21.15.84:3300/0 conn(0x7fa4f806ad80 0x7fa4f806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.641+0000 7fa4f659c640 1 -- 172.21.15.84:0/1108667300 >> v1:172.21.15.99:6789/0 conn(0x7fa4f80b21d0 legacy=0x7fa4f80b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.641+0000 7fa4f659c640 1 -- 172.21.15.84:0/1108667300 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f80cde90 con 0x7fa4f815c630 2023-10-11T06:14:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.642+0000 7fa4f659c640 1 --2- 172.21.15.84:0/1108667300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f815c630 0x7fa4f80cd950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa4f806db70 tx=0x7fa4e802c050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8cfa1186b778977 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:35.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4f559a640 1 -- 172.21.15.84:0/1108667300 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e802cbf0 con 0x7fa4f815c630 2023-10-11T06:14:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4f559a640 1 -- 172.21.15.84:0/1108667300 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e802cd50 con 0x7fa4f815c630 2023-10-11T06:14:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4f559a640 1 -- 172.21.15.84:0/1108667300 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8034610 con 0x7fa4f815c630 2023-10-11T06:14:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4fca84640 1 -- 172.21.15.84:0/1108667300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f815c630 msgr2=0x7fa4f80cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1108667300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f815c630 0x7fa4f80cd950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa4f806db70 tx=0x7fa4e802c050 comp rx=0 tx=0).stop 2023-10-11T06:14:35.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4fca84640 1 -- 172.21.15.84:0/1108667300 shutdown_connections 2023-10-11T06:14:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1108667300 >> v2:172.21.15.84:3300/0 conn(0x7fa4f806ad80 0x7fa4f806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1108667300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f815c630 0x7fa4f80cd950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.643+0000 7fa4fca84640 1 -- 172.21.15.84:0/1108667300 >> 172.21.15.84:0/1108667300 conn(0x7fa4f806a590 msgr2=0x7fa4f8099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.644+0000 7fa4fca84640 1 -- 172.21.15.84:0/1108667300 shutdown_connections 2023-10-11T06:14:35.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.644+0000 7fa4fca84640 1 -- 172.21.15.84:0/1108667300 wait complete. 2023-10-11T06:14:35.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.645+0000 7fa4fca84640 1 Processor -- start 2023-10-11T06:14:35.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.646+0000 7fa4fca84640 1 -- start start 2023-10-11T06:14:35.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.646+0000 7fa4fca84640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f806ad80 0x7fa4f8152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4fca84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4f80b21d0 0x7fa4f81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4fca84640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4f815c630 0x7fa4f815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4fca84640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80b4a50 con 0x7fa4f80b21d0 2023-10-11T06:14:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4fca84640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f8152620 con 0x7fa4f806ad80 2023-10-11T06:14:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4fca84640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80b4120 con 0x7fa4f815c630 2023-10-11T06:14:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4f80b21d0 0x7fa4f81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4f80b21d0 0x7fa4f81520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57496/0 (socket says 172.21.15.84:57496) 2023-10-11T06:14:35.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 -- 172.21.15.84:0/1386353070 learned_addr learned my addr 172.21.15.84:0/1386353070 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 -- 172.21.15.84:0/1386353070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4f815c630 msgr2=0x7fa4f815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f659c640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f806ad80 0x7fa4f8152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4f815c630 0x7fa4f815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 -- 172.21.15.84:0/1386353070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f806ad80 msgr2=0x7fa4f8152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f806ad80 0x7fa4f8152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.647+0000 7fa4f5d9b640 1 -- 172.21.15.84:0/1386353070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8009b30 con 0x7fa4f80b21d0 2023-10-11T06:14:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.648+0000 7fa4f5d9b640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4f80b21d0 0x7fa4f81520e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e870 tx=0x7fa4e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.648+0000 7fa4eeffd640 1 -- 172.21.15.84:0/1386353070 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00094f0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.648+0000 7fa4eeffd640 1 -- 172.21.15.84:0/1386353070 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e000c940 con 0x7fa4f80b21d0 2023-10-11T06:14:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.648+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f80b4400 con 0x7fa4f80b21d0 2023-10-11T06:14:35.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.648+0000 7fa4eeffd640 1 -- 172.21.15.84:0/1386353070 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000ccf0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.648+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f8152ea0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.649+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f80b4bc0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.650+0000 7fa4eeffd640 1 -- 172.21.15.84:0/1386353070 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fa4e0013470 con 0x7fa4f80b21d0 2023-10-11T06:14:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.651+0000 7fa4eeffd640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4d806a830 0x7fa4d806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.651+0000 7fa4eeffd640 1 -- 172.21.15.84:0/1386353070 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa4e00891c0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.651+0000 7fa4f659c640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4d806a830 0x7fa4d806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:35.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.652+0000 7fa4f659c640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4d806a830 0x7fa4d806cce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8035000 tx=0x7fa4e8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:35.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.656+0000 7fa4eeffd640 1 -- 172.21.15.84:0/1386353070 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fa4e00543a0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.844+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4f8067090 con 0x7fa4f80b21d0 2023-10-11T06:14:35.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.846+0000 7fa4eeffd640 1 -- 172.21.15.84:0/1386353070 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7fa4e00579c0 con 0x7fa4f80b21d0 2023-10-11T06:14:35.848 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:35.848 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:35.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.848+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4d806a830 msgr2=0x7fa4d806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.848+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4d806a830 0x7fa4d806cce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8035000 tx=0x7fa4e8030000 comp rx=0 tx=0).stop 2023-10-11T06:14:35.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.848+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4f80b21d0 msgr2=0x7fa4f81520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:35.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.848+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4f80b21d0 0x7fa4f81520e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e870 tx=0x7fa4e000ed30 comp rx=0 tx=0).stop 2023-10-11T06:14:35.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 shutdown_connections 2023-10-11T06:14:35.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fa4d806a830 0x7fa4d806cce0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4f815c630 0x7fa4f815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4f80b21d0 0x7fa4f81520e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 --2- 172.21.15.84:0/1386353070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f806ad80 0x7fa4f8152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:35.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 >> 172.21.15.84:0/1386353070 conn(0x7fa4f806a590 msgr2=0x7fa4f80cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:35.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 shutdown_connections 2023-10-11T06:14:35.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:35.849+0000 7fa4fca84640 1 -- 172.21.15.84:0/1386353070 wait complete. 2023-10-11T06:14:35.869 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T06:14:35.869 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:14:36.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.080+0000 7f2b6cc2d640 1 Processor -- start 2023-10-11T06:14:36.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.080+0000 7f2b6cc2d640 1 -- start start 2023-10-11T06:14:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6cc2d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b680ce0d0 0x7f2b680c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6cc2d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2b680adb90 0x7f2b680caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6cc2d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680cd300 0x7f2b680cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6cc2d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6815a340 con 0x7f2b680ce0d0 2023-10-11T06:14:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6cc2d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680ce4a0 con 0x7f2b680adb90 2023-10-11T06:14:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6cc2d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680cb960 con 0x7f2b680cd300 2023-10-11T06:14:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b680ce0d0 0x7f2b680c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b680ce0d0 0x7f2b680c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57498/0 (socket says 172.21.15.84:57498) 2023-10-11T06:14:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b6659c640 1 -- 172.21.15.84:0/332208136 learned_addr learned my addr 172.21.15.84:0/332208136 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.082+0000 7f2b65d9b640 1 --2- 172.21.15.84:0/332208136 >> v2:172.21.15.99:3300/0 conn(0x7f2b680adb90 0x7f2b680caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b66d9d640 1 --2- 172.21.15.84:0/332208136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680cd300 0x7f2b680cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b6659c640 1 -- 172.21.15.84:0/332208136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680cd300 msgr2=0x7f2b680cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b6659c640 1 --2- 172.21.15.84:0/332208136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680cd300 0x7f2b680cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b6659c640 1 -- 172.21.15.84:0/332208136 >> v2:172.21.15.99:3300/0 conn(0x7f2b680adb90 msgr2=0x7f2b680caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b6659c640 1 --2- 172.21.15.84:0/332208136 >> v2:172.21.15.99:3300/0 conn(0x7f2b680adb90 0x7f2b680caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b6659c640 1 -- 172.21.15.84:0/332208136 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b681593f0 con 0x7f2b680ce0d0 2023-10-11T06:14:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b65d9b640 1 --2- 172.21.15.84:0/332208136 >> v2:172.21.15.99:3300/0 conn(0x7f2b680adb90 0x7f2b680caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.083+0000 7f2b66d9d640 1 --2- 172.21.15.84:0/332208136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680cd300 0x7f2b680cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:36.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.084+0000 7f2b6659c640 1 --2- 172.21.15.84:0/332208136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680ce0d0 0x7f2b680c8cc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806e440 tx=0x7f2b5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24880094279e198 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:36.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.084+0000 7f2b6559a640 1 -- 172.21.15.84:0/332208136 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58004230 con 0x7f2b680ce0d0 2023-10-11T06:14:36.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6559a640 1 -- 172.21.15.84:0/332208136 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58004390 con 0x7f2b680ce0d0 2023-10-11T06:14:36.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6559a640 1 -- 172.21.15.84:0/332208136 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58003810 con 0x7f2b680ce0d0 2023-10-11T06:14:36.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/332208136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680ce0d0 msgr2=0x7f2b680c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/332208136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680ce0d0 0x7f2b680c8cc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806e440 tx=0x7f2b5802b8f0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/332208136 shutdown_connections 2023-10-11T06:14:36.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/332208136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680cd300 0x7f2b680cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/332208136 >> v2:172.21.15.99:3300/0 conn(0x7f2b680adb90 0x7f2b680caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/332208136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680ce0d0 0x7f2b680c8cc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/332208136 >> 172.21.15.84:0/332208136 conn(0x7f2b68069c70 msgr2=0x7f2b6806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:36.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/332208136 shutdown_connections 2023-10-11T06:14:36.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.085+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/332208136 wait complete. 2023-10-11T06:14:36.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.087+0000 7f2b6cc2d640 1 Processor -- start 2023-10-11T06:14:36.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.087+0000 7f2b6cc2d640 1 -- start start 2023-10-11T06:14:36.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6cc2d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680adb90 0x7f2b68152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6cc2d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b680cd300 0x7f2b681482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6cc2d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680ce0d0 0x7f2b68150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6cc2d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b681487e0 con 0x7f2b680adb90 2023-10-11T06:14:36.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6cc2d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68148d80 con 0x7f2b680cd300 2023-10-11T06:14:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6cc2d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68149060 con 0x7f2b680ce0d0 2023-10-11T06:14:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680adb90 0x7f2b68152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680adb90 0x7f2b68152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57506/0 (socket says 172.21.15.84:57506) 2023-10-11T06:14:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.088+0000 7f2b6659c640 1 -- 172.21.15.84:0/176175691 learned_addr learned my addr 172.21.15.84:0/176175691 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:36.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b6659c640 1 -- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680ce0d0 msgr2=0x7f2b68150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b6659c640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680ce0d0 0x7f2b68150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b6659c640 1 -- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b680cd300 msgr2=0x7f2b681482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b65d9b640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b680cd300 0x7f2b681482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b6659c640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b680cd300 0x7f2b681482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b6659c640 1 -- 172.21.15.84:0/176175691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009460 con 0x7f2b680adb90 2023-10-11T06:14:36.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b65d9b640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b680cd300 0x7f2b681482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:36.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b6659c640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680adb90 0x7f2b68152f40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2b58000c00 tx=0x7f2b580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:36.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b56ffd640 1 -- 172.21.15.84:0/176175691 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58003c40 con 0x7f2b680adb90 2023-10-11T06:14:36.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b56ffd640 1 -- 172.21.15.84:0/176175691 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5803f070 con 0x7f2b680adb90 2023-10-11T06:14:36.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b68149340 con 0x7f2b680adb90 2023-10-11T06:14:36.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.089+0000 7f2b56ffd640 1 -- 172.21.15.84:0/176175691 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58004500 con 0x7f2b680adb90 2023-10-11T06:14:36.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.090+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b68149900 con 0x7f2b680adb90 2023-10-11T06:14:36.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.091+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b68149af0 con 0x7f2b680adb90 2023-10-11T06:14:36.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.092+0000 7f2b56ffd640 1 -- 172.21.15.84:0/176175691 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2b58041030 con 0x7f2b680adb90 2023-10-11T06:14:36.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.092+0000 7f2b56ffd640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b4806a830 0x7f2b4806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.093+0000 7f2b56ffd640 1 -- 172.21.15.84:0/176175691 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b580ab2c0 con 0x7f2b680adb90 2023-10-11T06:14:36.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.093+0000 7f2b65d9b640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b4806a830 0x7f2b4806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.093+0000 7f2b65d9b640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b4806a830 0x7f2b4806cce0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2b6815c5f0 tx=0x7f2b5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:36.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.097+0000 7f2b56ffd640 1 -- 172.21.15.84:0/176175691 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2b580765d0 con 0x7f2b680adb90 2023-10-11T06:14:36.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.323+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b68073ed0 con 0x7f2b680adb90 2023-10-11T06:14:36.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.324+0000 7f2b56ffd640 1 -- 172.21.15.84:0/176175691 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f2b58079bf0 con 0x7f2b680adb90 2023-10-11T06:14:36.326 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:36.326 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1835716973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1835716973},{"type":"v1","addr":"172.21.15.99:6833","nonce":1835716973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1415879342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1415879342},{"type":"v1","addr":"172.21.15.99:6835","nonce":1415879342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2122307729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2122307729},{"type":"v1","addr":"172.21.15.84:6836","nonce":2122307729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:13:46.389013+0000","modified":"2023-10-11T06:13:49.406181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6027":{"gid":6027,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2048038445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2048038445},{"type":"v1","addr":"172.21.15.84:6837","nonce":2048038445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T06:14:36.327 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-10-11T06:14:36.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b4806a830 msgr2=0x7f2b4806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b4806a830 0x7f2b4806cce0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2b6815c5f0 tx=0x7f2b5c008040 comp rx=0 tx=0).stop 2023-10-11T06:14:36.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680adb90 msgr2=0x7f2b68152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680adb90 0x7f2b68152f40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2b58000c00 tx=0x7f2b580046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 shutdown_connections 2023-10-11T06:14:36.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2b4806a830 0x7f2b4806cce0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b680ce0d0 0x7f2b68150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b680cd300 0x7f2b681482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 --2- 172.21.15.84:0/176175691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b680adb90 0x7f2b68152f40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 >> 172.21.15.84:0/176175691 conn(0x7f2b68069c70 msgr2=0x7f2b6815af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:36.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 shutdown_connections 2023-10-11T06:14:36.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.327+0000 7f2b6cc2d640 1 -- 172.21.15.84:0/176175691 wait complete. 2023-10-11T06:14:36.346 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:14:36.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.556+0000 7f4418ef7640 1 Processor -- start 2023-10-11T06:14:36.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.556+0000 7f4418ef7640 1 -- start start 2023-10-11T06:14:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4418ef7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f441415d4f0 0x7f44140cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4418ef7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f44140c9360 0x7f44140ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4418ef7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4414071a90 con 0x7f44140ca300 2023-10-11T06:14:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4418ef7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4414072310 con 0x7f44140c9360 2023-10-11T06:14:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4418ef7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4414070280 con 0x7f441415d4f0 2023-10-11T06:14:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4411d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f44140ca300 0x7f44140c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43608/0 (socket says 172.21.15.84:43608) 2023-10-11T06:14:36.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4411d9b640 1 -- 172.21.15.84:0/241730329 learned_addr learned my addr 172.21.15.84:0/241730329 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:36.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f4412d9d640 1 --2- 172.21.15.84:0/241730329 >> v2:172.21.15.84:3300/0 conn(0x7f44140c9360 0x7f44140ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.558+0000 7f441259c640 1 --2- 172.21.15.84:0/241730329 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f441415d4f0 0x7f44140cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.559+0000 7f441159a640 1 -- 172.21.15.84:0/241730329 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160403426 0 0) 0x7f4414071a90 con 0x7f44140ca300 2023-10-11T06:14:36.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.559+0000 7f441159a640 1 -- 172.21.15.84:0/241730329 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43fc003520 con 0x7f44140ca300 2023-10-11T06:14:36.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.559+0000 7f4412d9d640 1 -- 172.21.15.84:0/241730329 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f441415d4f0 msgr2=0x7f44140cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:36.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.559+0000 7f4412d9d640 1 --2- 172.21.15.84:0/241730329 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f441415d4f0 0x7f44140cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.559+0000 7f4412d9d640 1 -- 172.21.15.84:0/241730329 >> v1:172.21.15.84:6789/0 conn(0x7f44140ca300 legacy=0x7f44140c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.559+0000 7f4412d9d640 1 -- 172.21.15.84:0/241730329 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414071cc0 con 0x7f44140c9360 2023-10-11T06:14:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.560+0000 7f4412d9d640 1 --2- 172.21.15.84:0/241730329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140c9360 0x7f44140ce550 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f440c009d90 tx=0x7f440c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c08224ac40a0ca server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.560+0000 7f441159a640 1 -- 172.21.15.84:0/241730329 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c02cc60 con 0x7f44140c9360 2023-10-11T06:14:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.560+0000 7f441159a640 1 -- 172.21.15.84:0/241730329 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c02cdc0 con 0x7f44140c9360 2023-10-11T06:14:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.560+0000 7f441159a640 1 -- 172.21.15.84:0/241730329 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c034640 con 0x7f44140c9360 2023-10-11T06:14:36.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 -- 172.21.15.84:0/241730329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140c9360 msgr2=0x7f44140ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 --2- 172.21.15.84:0/241730329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140c9360 0x7f44140ce550 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f440c009d90 tx=0x7f440c02beb0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 -- 172.21.15.84:0/241730329 shutdown_connections 2023-10-11T06:14:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 --2- 172.21.15.84:0/241730329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140c9360 0x7f44140ce550 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 --2- 172.21.15.84:0/241730329 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f441415d4f0 0x7f44140cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 -- 172.21.15.84:0/241730329 >> 172.21.15.84:0/241730329 conn(0x7f4414069c70 msgr2=0x7f441406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:36.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 -- 172.21.15.84:0/241730329 shutdown_connections 2023-10-11T06:14:36.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.561+0000 7f4418ef7640 1 -- 172.21.15.84:0/241730329 wait complete. 2023-10-11T06:14:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.563+0000 7f4418ef7640 1 Processor -- start 2023-10-11T06:14:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.563+0000 7f4418ef7640 1 -- start start 2023-10-11T06:14:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4418ef7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44140c9360 0x7f44140b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4418ef7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140ca300 0x7f44141598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4418ef7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f441415d4f0 0x7f441414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4418ef7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441415cd20 con 0x7f44140ca300 2023-10-11T06:14:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4418ef7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441415c760 con 0x7f441415d4f0 2023-10-11T06:14:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4418ef7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441415ca40 con 0x7f44140c9360 2023-10-11T06:14:36.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4411d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140ca300 0x7f44141598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4411d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140ca300 0x7f44141598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57516/0 (socket says 172.21.15.84:57516) 2023-10-11T06:14:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4411d9b640 1 -- 172.21.15.84:0/1325310394 learned_addr learned my addr 172.21.15.84:0/1325310394 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4411d9b640 1 -- 172.21.15.84:0/1325310394 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44140c9360 msgr2=0x7f44140b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4411d9b640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44140c9360 0x7f44140b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4411d9b640 1 -- 172.21.15.84:0/1325310394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f441415d4f0 msgr2=0x7f441414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.564+0000 7f4411d9b640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f441415d4f0 0x7f441414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.565+0000 7f4411d9b640 1 -- 172.21.15.84:0/1325310394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c009a70 con 0x7f44140ca300 2023-10-11T06:14:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.565+0000 7f4411d9b640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140ca300 0x7f44141598b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f440800c390 tx=0x7f440800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.565+0000 7f4402ffd640 1 -- 172.21.15.84:0/1325310394 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408005ce0 con 0x7f44140ca300 2023-10-11T06:14:36.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.565+0000 7f4402ffd640 1 -- 172.21.15.84:0/1325310394 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408005e40 con 0x7f44140ca300 2023-10-11T06:14:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.565+0000 7f4402ffd640 1 -- 172.21.15.84:0/1325310394 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44080072f0 con 0x7f44140ca300 2023-10-11T06:14:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.565+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44140b2fc0 con 0x7f44140ca300 2023-10-11T06:14:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.565+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44140b3d00 con 0x7f44140ca300 2023-10-11T06:14:36.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.567+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44140b34e0 con 0x7f44140ca300 2023-10-11T06:14:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.567+0000 7f4402ffd640 1 -- 172.21.15.84:0/1325310394 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4408007450 con 0x7f44140ca300 2023-10-11T06:14:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.568+0000 7f4402ffd640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43f406a830 0x7f43f406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.568+0000 7f4402ffd640 1 -- 172.21.15.84:0/1325310394 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4408088640 con 0x7f44140ca300 2023-10-11T06:14:36.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.569+0000 7f441259c640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43f406a830 0x7f43f406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:36.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.569+0000 7f441259c640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43f406a830 0x7f43f406cce0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4404006a30 tx=0x7f44040069c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:36.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.573+0000 7f4402ffd640 1 -- 172.21.15.84:0/1325310394 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f44140b34e0 con 0x7f44140ca300 2023-10-11T06:14:36.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.799+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44140b34e0 con 0x7f44140ca300 2023-10-11T06:14:36.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.800+0000 7f4402ffd640 1 -- 172.21.15.84:0/1325310394 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4648 (secure 0 0 0) 0x7f44140b34e0 con 0x7f44140ca300 2023-10-11T06:14:36.801 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:36.802 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1835716973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1835716973},{"type":"v1","addr":"172.21.15.99:6833","nonce":1835716973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1415879342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1415879342},{"type":"v1","addr":"172.21.15.99:6835","nonce":1415879342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2122307729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2122307729},{"type":"v1","addr":"172.21.15.84:6836","nonce":2122307729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-11T06:13:46.389013+0000","modified":"2023-10-11T06:13:49.406181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6027},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6027":{"gid":6027,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6837/2048038445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":2048038445},{"type":"v1","addr":"172.21.15.84:6837","nonce":2048038445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-11T06:14:36.802 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-10-11T06:14:36.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43f406a830 msgr2=0x7f43f406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43f406a830 0x7f43f406cce0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4404006a30 tx=0x7f44040069c0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140ca300 msgr2=0x7f44141598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:36.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140ca300 0x7f44141598b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f440800c390 tx=0x7f440800c850 comp rx=0 tx=0).stop 2023-10-11T06:14:36.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 shutdown_connections 2023-10-11T06:14:36.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f43f406a830 0x7f43f406cce0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f441415d4f0 0x7f441414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44140ca300 0x7f44141598b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 --2- 172.21.15.84:0/1325310394 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44140c9360 0x7f44140b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 >> 172.21.15.84:0/1325310394 conn(0x7f4414069c70 msgr2=0x7f44140ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 shutdown_connections 2023-10-11T06:14:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:36.803+0000 7f4418ef7640 1 -- 172.21.15.84:0/1325310394 wait complete. 2023-10-11T06:14:36.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:37.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.041+0000 7fee24df5640 1 Processor -- start 2023-10-11T06:14:37.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.041+0000 7fee24df5640 1 -- start start 2023-10-11T06:14:37.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.042+0000 7fee24df5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee200ae330 0x7fee2015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.042+0000 7fee24df5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee200ce6b0 con 0x7fee20070f70 2023-10-11T06:14:37.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.042+0000 7fee24df5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2015c700 con 0x7fee200ca9c0 2023-10-11T06:14:37.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.042+0000 7fee24df5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200cca00 con 0x7fee200ae330 2023-10-11T06:14:37.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.042+0000 7fee1dd9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fee20070f70 0x7fee2015bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43624/0 (socket says 172.21.15.84:43624) 2023-10-11T06:14:37.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.042+0000 7fee1dd9b640 1 -- 172.21.15.84:0/4134379750 learned_addr learned my addr 172.21.15.84:0/4134379750 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:37.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1ed9d640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee200ae330 0x7fee2015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745940977 0 0) 0x7fee200ce6b0 con 0x7fee20070f70 2023-10-11T06:14:37.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee0c003520 con 0x7fee20070f70 2023-10-11T06:14:37.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1534157547 0 0) 0x7fee0c003520 con 0x7fee20070f70 2023-10-11T06:14:37.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee200ae330 msgr2=0x7fee2015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:37.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee200ae330 0x7fee2015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 >> v1:172.21.15.99:6789/0 conn(0x7fee200ca9c0 legacy=0x7fee20070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200ceb40 con 0x7fee20070f70 2023-10-11T06:14:37.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fee08001670 con 0x7fee20070f70 2023-10-11T06:14:37.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 >> v1:172.21.15.84:6789/0 conn(0x7fee20070f70 legacy=0x7fee2015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0c0039a0 0x7fee0c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c005170 0x7fee0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee200cca00 con 0x7fee0c0047c0 2023-10-11T06:14:37.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c00bff0 con 0x7fee0c0039a0 2023-10-11T06:14:37.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c00c1b0 con 0x7fee0c005170 2023-10-11T06:14:37.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.043+0000 7fee1ed9d640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee200ae330 0x7fee2015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:14:37.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.044+0000 7fee1dd9b640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.044+0000 7fee1dd9b640 1 -- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c005170 msgr2=0x7fee0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:37.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.044+0000 7fee1ed9d640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c005170 0x7fee0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.044+0000 7fee1dd9b640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c005170 0x7fee0c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.044+0000 7fee1dd9b640 1 -- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0c0039a0 msgr2=0x7fee0c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:37.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.044+0000 7fee1dd9b640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0c0039a0 0x7fee0c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.044+0000 7fee1dd9b640 1 -- 172.21.15.84:0/4134379750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c003520 con 0x7fee0c0047c0 2023-10-11T06:14:37.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.045+0000 7fee1dd9b640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c210 tx=0x7fee0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7b7ae735997118 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.045+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08005ad0 con 0x7fee0c0047c0 2023-10-11T06:14:37.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.045+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee08005c30 con 0x7fee0c0047c0 2023-10-11T06:14:37.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.045+0000 7fee1d59a640 1 -- 172.21.15.84:0/4134379750 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0802c890 con 0x7fee0c0047c0 2023-10-11T06:14:37.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.046+0000 7fee24df5640 1 -- 172.21.15.84:0/4134379750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c0047c0 msgr2=0x7fee0c004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.046+0000 7fee24df5640 1 --2- 172.21.15.84:0/4134379750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee0c0047c0 0x7fee0c004c30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c210 tx=0x7fee0c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.046+0000 7fee1ed9d640 1 -- 172.21.15.84:0/4134379750 reap_dead start 2023-10-11T06:14:37.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.046+0000 7fee24df5640 1 -- 172.21.15.84:0/4134379750 shutdown_connections 2023-10-11T06:14:37.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.046+0000 7fee24df5640 1 -- 172.21.15.84:0/4134379750 >> 172.21.15.84:0/4134379750 conn(0x7fee200b1630 msgr2=0x7fee200cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:37.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.046+0000 7fee24df5640 1 -- 172.21.15.84:0/4134379750 shutdown_connections 2023-10-11T06:14:37.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.046+0000 7fee24df5640 1 -- 172.21.15.84:0/4134379750 wait complete. 2023-10-11T06:14:37.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.048+0000 7fee24df5640 1 Processor -- start 2023-10-11T06:14:37.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.048+0000 7fee24df5640 1 -- start start 2023-10-11T06:14:37.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee24df5640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c0047c0 0x7fee200c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee24df5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee2015c8b0 0x7fee201f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee24df5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2015d7d0 0x7fee201f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee24df5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2015e910 con 0x7fee2015c8b0 2023-10-11T06:14:37.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee24df5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee201f1840 con 0x7fee2015d7d0 2023-10-11T06:14:37.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee24df5640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2015dd90 con 0x7fee0c0047c0 2023-10-11T06:14:37.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee1dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee2015c8b0 0x7fee201f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee1dd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee2015c8b0 0x7fee201f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57532/0 (socket says 172.21.15.84:57532) 2023-10-11T06:14:37.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.049+0000 7fee1dd9b640 1 -- 172.21.15.84:0/1211524980 learned_addr learned my addr 172.21.15.84:0/1211524980 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:37.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1e59c640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c0047c0 0x7fee200c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1ed9d640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2015d7d0 0x7fee201f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1dd9b640 1 -- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c0047c0 msgr2=0x7fee200c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1dd9b640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c0047c0 0x7fee200c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1dd9b640 1 -- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2015d7d0 msgr2=0x7fee201f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1dd9b640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2015d7d0 0x7fee201f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1dd9b640 1 -- 172.21.15.84:0/1211524980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2015e070 con 0x7fee2015c8b0 2023-10-11T06:14:37.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1e59c640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c0047c0 0x7fee200c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee1dd9b640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee2015c8b0 0x7fee201f1c90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fee0802c430 tx=0x7fee08001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee16ffd640 1 -- 172.21.15.84:0/1211524980 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08039070 con 0x7fee2015c8b0 2023-10-11T06:14:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee16ffd640 1 -- 172.21.15.84:0/1211524980 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0802cca0 con 0x7fee2015c8b0 2023-10-11T06:14:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee16ffd640 1 -- 172.21.15.84:0/1211524980 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0803c040 con 0x7fee2015c8b0 2023-10-11T06:14:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2015e350 con 0x7fee2015c8b0 2023-10-11T06:14:37.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.050+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2015f610 con 0x7fee2015c8b0 2023-10-11T06:14:37.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.052+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee20067090 con 0x7fee2015c8b0 2023-10-11T06:14:37.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.052+0000 7fee16ffd640 1 -- 172.21.15.84:0/1211524980 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fee08032070 con 0x7fee2015c8b0 2023-10-11T06:14:37.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.053+0000 7fee16ffd640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee0006a860 0x7fee0006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.053+0000 7fee16ffd640 1 -- 172.21.15.84:0/1211524980 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee080aa4a0 con 0x7fee2015c8b0 2023-10-11T06:14:37.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.054+0000 7fee1e59c640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee0006a860 0x7fee0006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.054+0000 7fee1e59c640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee0006a860 0x7fee0006cd10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fee10007fd0 tx=0x7fee10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.058+0000 7fee16ffd640 1 -- 172.21.15.84:0/1211524980 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fee08075680 con 0x7fee2015c8b0 2023-10-11T06:14:37.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.251+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee20071660 con 0x7fee2015c8b0 2023-10-11T06:14:37.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.254+0000 7fee16ffd640 1 -- 172.21.15.84:0/1211524980 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17831 (secure 0 0 0) 0x7fee08078ca0 con 0x7fee2015c8b0 2023-10-11T06:14:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:37.256 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":54,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:13:47.392579+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-11T06:13:44.841435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-11T06:13:45.401366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:37.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee0006a860 msgr2=0x7fee0006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee0006a860 0x7fee0006cd10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fee10007fd0 tx=0x7fee10009040 comp rx=0 tx=0).stop 2023-10-11T06:14:37.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee2015c8b0 msgr2=0x7fee201f1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee2015c8b0 0x7fee201f1c90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fee0802c430 tx=0x7fee08001a00 comp rx=0 tx=0).stop 2023-10-11T06:14:37.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 shutdown_connections 2023-10-11T06:14:37.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee0006a860 0x7fee0006cd10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2015d7d0 0x7fee201f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee2015c8b0 0x7fee201f1c90 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 --2- 172.21.15.84:0/1211524980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee0c0047c0 0x7fee200c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 >> 172.21.15.84:0/1211524980 conn(0x7fee200b1630 msgr2=0x7fee2015bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:37.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.257+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 shutdown_connections 2023-10-11T06:14:37.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.258+0000 7fee24df5640 1 -- 172.21.15.84:0/1211524980 wait complete. 2023-10-11T06:14:37.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-11T06:14:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.485+0000 7f57ead5a640 1 Processor -- start 2023-10-11T06:14:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.486+0000 7f57ead5a640 1 -- start start 2023-10-11T06:14:37.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.487+0000 7f57ead5a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f57e415c290 0x7f57e40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.487+0000 7f57ead5a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f57e40d25e0 0x7f57e40d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.487+0000 7f57ead5a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e40d2a60 con 0x7f57e40d0a50 2023-10-11T06:14:37.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.487+0000 7f57ead5a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e40d0e90 con 0x7f57e40d25e0 2023-10-11T06:14:37.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.487+0000 7f57ead5a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e406cd10 con 0x7f57e415c290 2023-10-11T06:14:37.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f57e40d25e0 0x7f57e40d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f57e40d25e0 0x7f57e40d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57544/0 (socket says 172.21.15.84:57544) 2023-10-11T06:14:37.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e3fff640 1 -- 172.21.15.84:0/2930265502 learned_addr learned my addr 172.21.15.84:0/2930265502 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:37.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e8af6640 1 --2- 172.21.15.84:0/2930265502 >> v2:172.21.15.99:3300/0 conn(0x7f57e415c290 0x7f57e40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e3fff640 1 -- 172.21.15.84:0/2930265502 >> v2:172.21.15.99:3300/0 conn(0x7f57e415c290 msgr2=0x7f57e40af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:37.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e3fff640 1 --2- 172.21.15.84:0/2930265502 >> v2:172.21.15.99:3300/0 conn(0x7f57e415c290 0x7f57e40af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e3fff640 1 -- 172.21.15.84:0/2930265502 >> v1:172.21.15.99:6789/0 conn(0x7f57e40d0a50 legacy=0x7f57e406b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:37.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.488+0000 7f57e3fff640 1 -- 172.21.15.84:0/2930265502 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e40afb60 con 0x7f57e40d25e0 2023-10-11T06:14:37.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.489+0000 7f57e3fff640 1 --2- 172.21.15.84:0/2930265502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d25e0 0x7f57e40d0510 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f57e4072380 tx=0x7f57cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda01222199069b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.490+0000 7f57e37fe640 1 -- 172.21.15.84:0/2930265502 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0317e0 con 0x7f57e40d25e0 2023-10-11T06:14:37.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.490+0000 7f57e37fe640 1 -- 172.21.15.84:0/2930265502 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc031940 con 0x7f57e40d25e0 2023-10-11T06:14:37.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.490+0000 7f57e37fe640 1 -- 172.21.15.84:0/2930265502 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc031cd0 con 0x7f57e40d25e0 2023-10-11T06:14:37.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 -- 172.21.15.84:0/2930265502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d25e0 msgr2=0x7f57e40d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 --2- 172.21.15.84:0/2930265502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d25e0 0x7f57e40d0510 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f57e4072380 tx=0x7f57cc02b880 comp rx=0 tx=0).stop 2023-10-11T06:14:37.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 -- 172.21.15.84:0/2930265502 shutdown_connections 2023-10-11T06:14:37.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 --2- 172.21.15.84:0/2930265502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d25e0 0x7f57e40d0510 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 --2- 172.21.15.84:0/2930265502 >> v2:172.21.15.99:3300/0 conn(0x7f57e415c290 0x7f57e40af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 -- 172.21.15.84:0/2930265502 >> 172.21.15.84:0/2930265502 conn(0x7f57e4071cc0 msgr2=0x7f57e409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:37.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 -- 172.21.15.84:0/2930265502 shutdown_connections 2023-10-11T06:14:37.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.491+0000 7f57ead5a640 1 -- 172.21.15.84:0/2930265502 wait complete. 2023-10-11T06:14:37.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.492+0000 7f57ead5a640 1 Processor -- start 2023-10-11T06:14:37.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.493+0000 7f57ead5a640 1 -- start start 2023-10-11T06:14:37.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57ead5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d0a50 0x7f57e41e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57ead5a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57e40d25e0 0x7f57e40c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57ead5a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57e415c290 0x7f57e41e3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57ead5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41e1870 con 0x7f57e40d0a50 2023-10-11T06:14:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57ead5a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41e1b50 con 0x7f57e415c290 2023-10-11T06:14:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57ead5a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41e1e30 con 0x7f57e40d25e0 2023-10-11T06:14:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d0a50 0x7f57e41e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d0a50 0x7f57e41e78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57556/0 (socket says 172.21.15.84:57556) 2023-10-11T06:14:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 -- 172.21.15.84:0/3577430377 learned_addr learned my addr 172.21.15.84:0/3577430377 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 -- 172.21.15.84:0/3577430377 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57e40d25e0 msgr2=0x7f57e40c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57e40d25e0 0x7f57e40c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 -- 172.21.15.84:0/3577430377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57e415c290 msgr2=0x7f57e41e3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57e415c290 0x7f57e41e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 -- 172.21.15.84:0/3577430377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc0094d0 con 0x7f57e40d0a50 2023-10-11T06:14:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.494+0000 7f57e8af6640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d0a50 0x7f57e41e78d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d5f0 tx=0x7f57d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.495+0000 7f57e17fa640 1 -- 172.21.15.84:0/3577430377 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40125f0 con 0x7f57e40d0a50 2023-10-11T06:14:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.495+0000 7f57e17fa640 1 -- 172.21.15.84:0/3577430377 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4012750 con 0x7f57e40d0a50 2023-10-11T06:14:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.495+0000 7f57e17fa640 1 -- 172.21.15.84:0/3577430377 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4012b00 con 0x7f57e40d0a50 2023-10-11T06:14:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.495+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e41e2110 con 0x7f57e40d0a50 2023-10-11T06:14:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.495+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e41e7e10 con 0x7f57e40d0a50 2023-10-11T06:14:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.496+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e40b2ab0 con 0x7f57e40d0a50 2023-10-11T06:14:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.497+0000 7f57e17fa640 1 -- 172.21.15.84:0/3577430377 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f57d4012c60 con 0x7f57e40d0a50 2023-10-11T06:14:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.498+0000 7f57e17fa640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f57c406a830 0x7f57c406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.498+0000 7f57e17fa640 1 -- 172.21.15.84:0/3577430377 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57d4089460 con 0x7f57e40d0a50 2023-10-11T06:14:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.499+0000 7f57e3fff640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f57c406a830 0x7f57c406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.499+0000 7f57e3fff640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f57c406a830 0x7f57c406cce0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f57cc036000 tx=0x7f57cc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.503+0000 7f57e17fa640 1 -- 172.21.15.84:0/3577430377 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f57d40546e0 con 0x7f57e40d0a50 2023-10-11T06:14:37.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.726+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f57e4073ed0 con 0x7f57e40d0a50 2023-10-11T06:14:37.728 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-10-11T06:14:37.727+0000 7feb66d4a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-11T06:14:37.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.746+0000 7f57e17fa640 1 -- 172.21.15.84:0/3577430377 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f57d4057d00 con 0x7f57e40d0a50 2023-10-11T06:14:37.747 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-11T06:14:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.748+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f57c406a830 msgr2=0x7f57c406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.748+0000 7f57ead5a640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f57c406a830 0x7f57c406cce0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f57cc036000 tx=0x7f57cc030000 comp rx=0 tx=0).stop 2023-10-11T06:14:37.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.748+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d0a50 msgr2=0x7f57e41e78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.748+0000 7f57ead5a640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d0a50 0x7f57e41e78d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d5f0 tx=0x7f57d400dab0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 shutdown_connections 2023-10-11T06:14:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f57c406a830 0x7f57c406cce0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57e415c290 0x7f57e41e3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57e40d25e0 0x7f57e40c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 --2- 172.21.15.84:0/3577430377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57e40d0a50 0x7f57e41e78d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 >> 172.21.15.84:0/3577430377 conn(0x7f57e4071cc0 msgr2=0x7f57e40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:37.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 shutdown_connections 2023-10-11T06:14:37.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.749+0000 7f57ead5a640 1 -- 172.21.15.84:0/3577430377 wait complete. 2023-10-11T06:14:37.764 INFO:tasks.ceph.mds.c.smithi084.stderr: -3267> 2023-10-11T06:13:51.893+0000 7f5dd0408640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-11T06:14:37.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-11T06:14:37.837 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-10-11T06:14:37.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.960+0000 7fdb842ec640 1 Processor -- start 2023-10-11T06:14:37.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.960+0000 7fdb842ec640 1 -- start start 2023-10-11T06:14:37.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.961+0000 7fdb842ec640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdb7c0ca9c0 0x7fdb7c070860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.961+0000 7fdb842ec640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb7c0cf7e0 0x7fdb7c070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.961+0000 7fdb842ec640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c0ce6b0 con 0x7fdb7c0ae330 2023-10-11T06:14:37.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.961+0000 7fdb842ec640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c0cca00 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.961+0000 7fdb842ec640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c0ceb40 con 0x7fdb7c0ca9c0 2023-10-11T06:14:37.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82889640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb7c0cf7e0 0x7fdb7c070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82889640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb7c0cf7e0 0x7fdb7c070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57578/0 (socket says 172.21.15.84:57578) 2023-10-11T06:14:37.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82889640 1 -- 172.21.15.84:0/3484798821 learned_addr learned my addr 172.21.15.84:0/3484798821 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:37.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82088640 1 --2- 172.21.15.84:0/3484798821 >> v2:172.21.15.99:3300/0 conn(0x7fdb7c0ca9c0 0x7fdb7c070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82889640 1 -- 172.21.15.84:0/3484798821 >> v2:172.21.15.99:3300/0 conn(0x7fdb7c0ca9c0 msgr2=0x7fdb7c070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:37.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82889640 1 --2- 172.21.15.84:0/3484798821 >> v2:172.21.15.99:3300/0 conn(0x7fdb7c0ca9c0 0x7fdb7c070860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82889640 1 -- 172.21.15.84:0/3484798821 >> v1:172.21.15.99:6789/0 conn(0x7fdb7c0ae330 legacy=0x7fdb7c15bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:37.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.962+0000 7fdb82889640 1 -- 172.21.15.84:0/3484798821 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c160af0 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.964+0000 7fdb82889640 1 --2- 172.21.15.84:0/3484798821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c070da0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c066000 tx=0x7fdb7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab62e1f4a4382b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.964+0000 7fdb81086640 1 -- 172.21.15.84:0/3484798821 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7402ca80 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.964+0000 7fdb81086640 1 -- 172.21.15.84:0/3484798821 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7402cbe0 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.964+0000 7fdb81086640 1 -- 172.21.15.84:0/3484798821 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb740343f0 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 -- 172.21.15.84:0/3484798821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 msgr2=0x7fdb7c070da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 --2- 172.21.15.84:0/3484798821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c070da0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c066000 tx=0x7fdb7402bee0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 -- 172.21.15.84:0/3484798821 shutdown_connections 2023-10-11T06:14:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 --2- 172.21.15.84:0/3484798821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c070da0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 --2- 172.21.15.84:0/3484798821 >> v2:172.21.15.99:3300/0 conn(0x7fdb7c0ca9c0 0x7fdb7c070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 -- 172.21.15.84:0/3484798821 >> 172.21.15.84:0/3484798821 conn(0x7fdb7c0b1630 msgr2=0x7fdb7c0cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:37.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 -- 172.21.15.84:0/3484798821 shutdown_connections 2023-10-11T06:14:37.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.965+0000 7fdb842ec640 1 -- 172.21.15.84:0/3484798821 wait complete. 2023-10-11T06:14:37.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.967+0000 7fdb842ec640 1 Processor -- start 2023-10-11T06:14:37.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.967+0000 7fdb842ec640 1 -- start start 2023-10-11T06:14:37.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb842ec640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb7c0ae330 0x7fdb7c0acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb842ec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb7c0ca9c0 0x7fdb7c1f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb842ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb842ec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c1f2780 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb842ec640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c159fe0 con 0x7fdb7c0ca9c0 2023-10-11T06:14:37.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb842ec640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0ad1a0 con 0x7fdb7c0ae330 2023-10-11T06:14:37.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c0b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57590/0 (socket says 172.21.15.84:57590) 2023-10-11T06:14:37.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 -- 172.21.15.84:0/1272377824 learned_addr learned my addr 172.21.15.84:0/1272377824 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:37.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb81887640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb7c0ca9c0 0x7fdb7c1f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82088640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb7c0ae330 0x7fdb7c0acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 -- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb7c0ae330 msgr2=0x7fdb7c0acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb7c0ae330 0x7fdb7c0acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 -- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb7c0ca9c0 msgr2=0x7fdb7c1f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:37.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb7c0ca9c0 0x7fdb7c1f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:37.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82889640 1 -- 172.21.15.84:0/1272377824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb74009b30 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb81887640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb7c0ca9c0 0x7fdb7c1f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:37.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.968+0000 7fdb82088640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb7c0ae330 0x7fdb7c0acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:37.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.969+0000 7fdb82889640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c0b1ca0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdb7402c4f0 tx=0x7fdb74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.969+0000 7fdb72ffd640 1 -- 172.21.15.84:0/1272377824 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7403e450 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.969+0000 7fdb72ffd640 1 -- 172.21.15.84:0/1272377824 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7403e5d0 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.969+0000 7fdb72ffd640 1 -- 172.21.15.84:0/1272377824 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7403e980 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.969+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c159c80 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.969+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c1f21c0 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.970+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c067090 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.971+0000 7fdb72ffd640 1 -- 172.21.15.84:0/1272377824 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fdb7402cd50 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.972+0000 7fdb72ffd640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdb6006a830 0x7fdb6006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:37.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.972+0000 7fdb72ffd640 1 -- 172.21.15.84:0/1272377824 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fdb740abd70 con 0x7fdb7c0cf7e0 2023-10-11T06:14:37.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.972+0000 7fdb82088640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdb6006a830 0x7fdb6006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:37.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.973+0000 7fdb82088640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdb6006a830 0x7fdb6006cce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800b820 tx=0x7fdb78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:37.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:37.977+0000 7fdb72ffd640 1 -- 172.21.15.84:0/1272377824 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fdb740770a0 con 0x7fdb7c0cf7e0 2023-10-11T06:14:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.205+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdb7c071660 con 0x7fdb7c0cf7e0 2023-10-11T06:14:38.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.748+0000 7fdb72ffd640 1 -- 172.21.15.84:0/1272377824 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdb7407a6c0 con 0x7fdb7c0cf7e0 2023-10-11T06:14:38.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdb6006a830 msgr2=0x7fdb6006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:38.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdb6006a830 0x7fdb6006cce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800b820 tx=0x7fdb78008040 comp rx=0 tx=0).stop 2023-10-11T06:14:38.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 msgr2=0x7fdb7c0b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:38.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c0b1ca0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdb7402c4f0 tx=0x7fdb74004a40 comp rx=0 tx=0).stop 2023-10-11T06:14:38.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 shutdown_connections 2023-10-11T06:14:38.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fdb6006a830 0x7fdb6006cce0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:38.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb7c0cf7e0 0x7fdb7c0b1ca0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:38.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb7c0ca9c0 0x7fdb7c1f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:38.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 --2- 172.21.15.84:0/1272377824 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb7c0ae330 0x7fdb7c0acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:38.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.752+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 >> 172.21.15.84:0/1272377824 conn(0x7fdb7c0b1630 msgr2=0x7fdb7c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:38.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.753+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 shutdown_connections 2023-10-11T06:14:38.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.753+0000 7fdb842ec640 1 -- 172.21.15.84:0/1272377824 wait complete. 2023-10-11T06:14:38.773 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-11T06:14:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.988+0000 7f0a62353640 1 Processor -- start 2023-10-11T06:14:38.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.989+0000 7f0a62353640 1 -- start start 2023-10-11T06:14:38.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.990+0000 7f0a62353640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a5c06c4c0 0x7f0a5c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.990+0000 7f0a62353640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c0b3f20 con 0x7f0a5c0ca750 2023-10-11T06:14:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.990+0000 7f0a62353640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c0caf10 con 0x7f0a5c15d510 2023-10-11T06:14:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.990+0000 7f0a62353640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c0cac30 con 0x7f0a5c06c4c0 2023-10-11T06:14:38.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.991+0000 7f0a61b52640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0a5c0ca750 0x7f0a5c0c8ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34894/0 (socket says 172.21.15.84:34894) 2023-10-11T06:14:38.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.991+0000 7f0a61b52640 1 -- 172.21.15.84:0/258392750 learned_addr learned my addr 172.21.15.84:0/258392750 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:38.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.991+0000 7f0a60b50640 1 --2- 172.21.15.84:0/258392750 >> v2:172.21.15.84:3300/0 conn(0x7f0a5c06c4c0 0x7f0a5c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.991+0000 7f0a60b50640 1 -- 172.21.15.84:0/258392750 >> v1:172.21.15.99:6789/0 conn(0x7f0a5c15d510 legacy=0x7f0a5c06bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.991+0000 7f0a60b50640 1 -- 172.21.15.84:0/258392750 >> v1:172.21.15.84:6789/0 conn(0x7f0a5c0ca750 legacy=0x7f0a5c0c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.991+0000 7f0a60b50640 1 -- 172.21.15.84:0/258392750 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c15daf0 con 0x7f0a5c06c4c0 2023-10-11T06:14:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.992+0000 7f0a60b50640 1 --2- 172.21.15.84:0/258392750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c06fda0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0a500097a0 tx=0x7f0a5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93eb8c684b4cb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.993+0000 7f0a4ffff640 1 -- 172.21.15.84:0/258392750 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50031830 con 0x7f0a5c06c4c0 2023-10-11T06:14:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.993+0000 7f0a4ffff640 1 -- 172.21.15.84:0/258392750 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a50031990 con 0x7f0a5c06c4c0 2023-10-11T06:14:38.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.993+0000 7f0a4ffff640 1 -- 172.21.15.84:0/258392750 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50031d20 con 0x7f0a5c06c4c0 2023-10-11T06:14:38.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.993+0000 7f0a62353640 1 -- 172.21.15.84:0/258392750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c06c4c0 msgr2=0x7f0a5c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:38.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.993+0000 7f0a62353640 1 --2- 172.21.15.84:0/258392750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c06fda0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0a500097a0 tx=0x7f0a5002b8d0 comp rx=0 tx=0).stop 2023-10-11T06:14:38.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.994+0000 7f0a62353640 1 -- 172.21.15.84:0/258392750 shutdown_connections 2023-10-11T06:14:38.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.994+0000 7f0a62353640 1 --2- 172.21.15.84:0/258392750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c06fda0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:38.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.994+0000 7f0a62353640 1 -- 172.21.15.84:0/258392750 >> 172.21.15.84:0/258392750 conn(0x7f0a5c069c70 msgr2=0x7f0a5c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:38.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.994+0000 7f0a62353640 1 -- 172.21.15.84:0/258392750 shutdown_connections 2023-10-11T06:14:38.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.994+0000 7f0a62353640 1 -- 172.21.15.84:0/258392750 wait complete. 2023-10-11T06:14:38.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.995+0000 7f0a62353640 1 Processor -- start 2023-10-11T06:14:39.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.996+0000 7f0a62353640 1 -- start start 2023-10-11T06:14:39.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.996+0000 7f0a62353640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:39.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.996+0000 7f0a62353640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c0ca750 0x7f0a5c1f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:39.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.996+0000 7f0a62353640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a5c15d510 0x7f0a5c0c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a62353640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0c84a0 con 0x7f0a5c0ca750 2023-10-11T06:14:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a62353640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c1f19f0 con 0x7f0a5c06c4c0 2023-10-11T06:14:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a62353640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c1f1cc0 con 0x7f0a5c15d510 2023-10-11T06:14:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c0ca750 0x7f0a5c1f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:39.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c0ca750 0x7f0a5c1f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36966/0 (socket says 172.21.15.84:36966) 2023-10-11T06:14:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 -- 172.21.15.84:0/3351271610 learned_addr learned my addr 172.21.15.84:0/3351271610 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a61351640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a61b52640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a5c15d510 0x7f0a5c0c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 -- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a5c15d510 msgr2=0x7f0a5c0c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:39.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a5c15d510 0x7f0a5c0c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:39.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 -- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c06c4c0 msgr2=0x7f0a5c0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:39.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c0c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:39.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a60b50640 1 -- 172.21.15.84:0/3351271610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a50009400 con 0x7f0a5c0ca750 2023-10-11T06:14:39.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.997+0000 7f0a61351640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:14:39.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.998+0000 7f0a60b50640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c0ca750 0x7f0a5c1f14b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0a500098c0 tx=0x7f0a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:39.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.998+0000 7f0a4dffb640 1 -- 172.21.15.84:0/3351271610 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5003ba70 con 0x7f0a5c0ca750 2023-10-11T06:14:39.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.998+0000 7f0a4dffb640 1 -- 172.21.15.84:0/3351271610 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5003bbf0 con 0x7f0a5c0ca750 2023-10-11T06:14:39.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.998+0000 7f0a4dffb640 1 -- 172.21.15.84:0/3351271610 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a500453f0 con 0x7f0a5c0ca750 2023-10-11T06:14:39.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.998+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c1f1fa0 con 0x7f0a5c0ca750 2023-10-11T06:14:39.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.998+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c1f4aa0 con 0x7f0a5c0ca750 2023-10-11T06:14:39.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:38.999+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c04e890 con 0x7f0a5c0ca750 2023-10-11T06:14:39.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:39.000+0000 7f0a4dffb640 1 -- 172.21.15.84:0/3351271610 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0a5004e450 con 0x7f0a5c0ca750 2023-10-11T06:14:39.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:39.001+0000 7f0a4dffb640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a4006a830 0x7f0a4006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:39.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:39.001+0000 7f0a4dffb640 1 -- 172.21.15.84:0/3351271610 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0a500ab290 con 0x7f0a5c0ca750 2023-10-11T06:14:39.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:39.001+0000 7f0a61351640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a4006a830 0x7f0a4006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:39.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:39.002+0000 7f0a61351640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a4006a830 0x7f0a4006cce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0a58006fd0 tx=0x7f0a58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:39.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:39.005+0000 7f0a4dffb640 1 -- 172.21.15.84:0/3351271610 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0a500765c0 con 0x7f0a5c0ca750 2023-10-11T06:14:39.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:39.194+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a5c0cb4f0 con 0x7f0a5c0ca750 2023-10-11T06:14:40.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.006+0000 7f0a4dffb640 1 -- 172.21.15.84:0/3351271610 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0a50079be0 con 0x7f0a5c0ca750 2023-10-11T06:14:40.008 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-10-11T06:14:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.010+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a4006a830 msgr2=0x7f0a4006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.010+0000 7f0a62353640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a4006a830 0x7f0a4006cce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0a58006fd0 tx=0x7f0a58008040 comp rx=0 tx=0).stop 2023-10-11T06:14:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.011+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c0ca750 msgr2=0x7f0a5c1f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:40.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.011+0000 7f0a62353640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c0ca750 0x7f0a5c1f14b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0a500098c0 tx=0x7f0a500046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.012+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 shutdown_connections 2023-10-11T06:14:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.012+0000 7f0a62353640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0a4006a830 0x7f0a4006cce0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.012+0000 7f0a62353640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a5c15d510 0x7f0a5c0c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.012+0000 7f0a62353640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c0ca750 0x7f0a5c1f14b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.012+0000 7f0a62353640 1 --2- 172.21.15.84:0/3351271610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c06c4c0 0x7f0a5c0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.013+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 >> 172.21.15.84:0/3351271610 conn(0x7f0a5c069c70 msgr2=0x7f0a5c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:40.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.013+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 shutdown_connections 2023-10-11T06:14:40.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.014+0000 7f0a62353640 1 -- 172.21.15.84:0/3351271610 wait complete. 2023-10-11T06:14:40.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-11T06:14:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.243+0000 7f0c2bd77640 1 Processor -- start 2023-10-11T06:14:40.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.243+0000 7f0c2bd77640 1 -- start start 2023-10-11T06:14:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.245+0000 7f0c2bd77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c240cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.245+0000 7f0c2bd77640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0c240c8d00 0x7f0c2406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.245+0000 7f0c2bd77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c24071380 con 0x7f0c2415a0b0 2023-10-11T06:14:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.245+0000 7f0c2bd77640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2406e0f0 con 0x7f0c240c8d00 2023-10-11T06:14:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.245+0000 7f0c2bd77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240cbae0 con 0x7f0c240ced30 2023-10-11T06:14:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.246+0000 7f0c29312640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c240cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.246+0000 7f0c2a314640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0c240c8d00 0x7f0c2406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.246+0000 7f0c29312640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c240cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:36486/0 (socket says 172.21.15.84:36486) 2023-10-11T06:14:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.246+0000 7f0c2a314640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0c240c8d00 0x7f0c2406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:59982/0 (socket says 172.21.15.84:59982) 2023-10-11T06:14:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.246+0000 7f0c29312640 1 -- 172.21.15.84:0/3736237869 learned_addr learned my addr 172.21.15.84:0/3736237869 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.247+0000 7f0c29312640 1 -- 172.21.15.84:0/3736237869 >> v2:172.21.15.99:3300/0 conn(0x7f0c240c8d00 msgr2=0x7f0c2406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.247+0000 7f0c29312640 1 --2- 172.21.15.84:0/3736237869 >> v2:172.21.15.99:3300/0 conn(0x7f0c240c8d00 0x7f0c2406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.247+0000 7f0c29312640 1 -- 172.21.15.84:0/3736237869 >> v1:172.21.15.99:6789/0 conn(0x7f0c2415a0b0 legacy=0x7f0c240ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.247+0000 7f0c29312640 1 -- 172.21.15.84:0/3736237869 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2406de10 con 0x7f0c240ced30 2023-10-11T06:14:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.248+0000 7f0c29312640 1 --2- 172.21.15.84:0/3736237869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c240cb5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009870 tx=0x7f0c1402b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85ecd1bae5f99b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.248+0000 7f0c28b11640 1 -- 172.21.15.84:0/3736237869 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c140317e0 con 0x7f0c240ced30 2023-10-11T06:14:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.248+0000 7f0c28b11640 1 -- 172.21.15.84:0/3736237869 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c14031940 con 0x7f0c240ced30 2023-10-11T06:14:40.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.248+0000 7f0c28b11640 1 -- 172.21.15.84:0/3736237869 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14031cd0 con 0x7f0c240ced30 2023-10-11T06:14:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 -- 172.21.15.84:0/3736237869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 msgr2=0x7f0c240cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/3736237869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c240cb5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009870 tx=0x7f0c1402b880 comp rx=0 tx=0).stop 2023-10-11T06:14:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 -- 172.21.15.84:0/3736237869 shutdown_connections 2023-10-11T06:14:40.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/3736237869 >> v2:172.21.15.99:3300/0 conn(0x7f0c240c8d00 0x7f0c2406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/3736237869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c240cb5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 -- 172.21.15.84:0/3736237869 >> 172.21.15.84:0/3736237869 conn(0x7f0c24069c70 msgr2=0x7f0c2406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 -- 172.21.15.84:0/3736237869 shutdown_connections 2023-10-11T06:14:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.249+0000 7f0c2bd77640 1 -- 172.21.15.84:0/3736237869 wait complete. 2023-10-11T06:14:40.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.251+0000 7f0c2bd77640 1 Processor -- start 2023-10-11T06:14:40.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.251+0000 7f0c2bd77640 1 -- start start 2023-10-11T06:14:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.252+0000 7f0c2bd77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c240c8d00 0x7f0c240b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.252+0000 7f0c2bd77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c241eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.252+0000 7f0c2bd77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c2415a0b0 0x7f0c2414db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.252+0000 7f0c2bd77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2414e610 con 0x7f0c240c8d00 2023-10-11T06:14:40.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.252+0000 7f0c2bd77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c241577d0 con 0x7f0c2415a0b0 2023-10-11T06:14:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.252+0000 7f0c2bd77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c24157ab0 con 0x7f0c240ced30 2023-10-11T06:14:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.252+0000 7f0c29b13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c240c8d00 0x7f0c240b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c240c8d00 0x7f0c240b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36974/0 (socket says 172.21.15.84:36974) 2023-10-11T06:14:40.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 -- 172.21.15.84:0/1580044352 learned_addr learned my addr 172.21.15.84:0/1580044352 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:40.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29312640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c241eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:40.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c2a314640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c2415a0b0 0x7f0c2414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:40.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 -- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 msgr2=0x7f0c241eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c241eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 -- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c2415a0b0 msgr2=0x7f0c2414db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c2415a0b0 0x7f0c2414db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:40.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 -- 172.21.15.84:0/1580044352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140094d0 con 0x7f0c240c8d00 2023-10-11T06:14:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29312640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c241eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.253+0000 7f0c29b13640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c240c8d00 0x7f0c240b15c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c400 tx=0x7f0c2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.254+0000 7f0c1a7fc640 1 -- 172.21.15.84:0/1580044352 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c200052d0 con 0x7f0c240c8d00 2023-10-11T06:14:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.254+0000 7f0c1a7fc640 1 -- 172.21.15.84:0/1580044352 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c20005430 con 0x7f0c240c8d00 2023-10-11T06:14:40.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.254+0000 7f0c1a7fc640 1 -- 172.21.15.84:0/1580044352 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20003830 con 0x7f0c240c8d00 2023-10-11T06:14:40.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.254+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c24157d90 con 0x7f0c240c8d00 2023-10-11T06:14:40.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.254+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c240b1b00 con 0x7f0c240c8d00 2023-10-11T06:14:40.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.255+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c241582b0 con 0x7f0c240c8d00 2023-10-11T06:14:40.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.256+0000 7f0c1a7fc640 1 -- 172.21.15.84:0/1580044352 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f0c20003990 con 0x7f0c240c8d00 2023-10-11T06:14:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.257+0000 7f0c1a7fc640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c0806a830 0x7f0c0806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.257+0000 7f0c1a7fc640 1 -- 172.21.15.84:0/1580044352 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f0c200884c0 con 0x7f0c240c8d00 2023-10-11T06:14:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.257+0000 7f0c29312640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c0806a830 0x7f0c0806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:40.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.258+0000 7f0c29312640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c0806a830 0x7f0c0806cce0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009870 tx=0x7f0c1403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:40.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.261+0000 7f0c1a7fc640 1 -- 172.21.15.84:0/1580044352 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f0c241582b0 con 0x7f0c240c8d00 2023-10-11T06:14:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:40.450+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c241582b0 con 0x7f0c240c8d00 2023-10-11T06:14:41.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.010+0000 7f0c1a7fc640 1 -- 172.21.15.84:0/1580044352 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0c241582b0 con 0x7f0c240c8d00 2023-10-11T06:14:41.012 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-10-11T06:14:41.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c0806a830 msgr2=0x7f0c0806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c0806a830 0x7f0c0806cce0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009870 tx=0x7f0c1403d040 comp rx=0 tx=0).stop 2023-10-11T06:14:41.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c240c8d00 msgr2=0x7f0c240b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c240c8d00 0x7f0c240b15c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c400 tx=0x7f0c2000c8c0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 shutdown_connections 2023-10-11T06:14:41.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f0c0806a830 0x7f0c0806cce0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c2415a0b0 0x7f0c2414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c240ced30 0x7f0c241eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 --2- 172.21.15.84:0/1580044352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c240c8d00 0x7f0c240b15c0 secure :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c400 tx=0x7f0c2000c8c0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.016+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 >> 172.21.15.84:0/1580044352 conn(0x7f0c24069c70 msgr2=0x7f0c24156410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:41.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.020+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 shutdown_connections 2023-10-11T06:14:41.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.022+0000 7f0c2bd77640 1 -- 172.21.15.84:0/1580044352 wait complete. 2023-10-11T06:14:41.047 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-11T06:14:41.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.253+0000 7f40a121a640 1 Processor -- start 2023-10-11T06:14:41.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.253+0000 7f40a121a640 1 -- start start 2023-10-11T06:14:41.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.254+0000 7f40a121a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f409c0cd300 0x7f409c163e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.254+0000 7f40a121a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c1593f0 con 0x7f409c0adb90 2023-10-11T06:14:41.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.254+0000 7f40a121a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c1596c0 con 0x7f409c0ce0d0 2023-10-11T06:14:41.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.254+0000 7f40a121a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c15a340 con 0x7f409c0cd300 2023-10-11T06:14:41.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.255+0000 7f409b59e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f409c0cd300 0x7f409c163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.255+0000 7f4093fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f409c0adb90 0x7f409c0caee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34896/0 (socket says 172.21.15.84:34896) 2023-10-11T06:14:41.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.255+0000 7f4093fff640 1 -- 172.21.15.84:0/639043476 learned_addr learned my addr 172.21.15.84:0/639043476 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:41.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.255+0000 7f409b59e640 1 -- 172.21.15.84:0/639043476 >> v1:172.21.15.99:6789/0 conn(0x7f409c0ce0d0 legacy=0x7f409c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:41.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.255+0000 7f409b59e640 1 -- 172.21.15.84:0/639043476 >> v1:172.21.15.84:6789/0 conn(0x7f409c0adb90 legacy=0x7f409c0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.255+0000 7f409b59e640 1 -- 172.21.15.84:0/639043476 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c1599a0 con 0x7f409c0cd300 2023-10-11T06:14:41.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.256+0000 7f409b59e640 1 --2- 172.21.15.84:0/639043476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0cd300 0x7f409c163e10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4094009d90 tx=0x7f409402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab315af3bdd1e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:41.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.256+0000 7f409a59c640 1 -- 172.21.15.84:0/639043476 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409402cbd0 con 0x7f409c0cd300 2023-10-11T06:14:41.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f409a59c640 1 -- 172.21.15.84:0/639043476 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409402cd30 con 0x7f409c0cd300 2023-10-11T06:14:41.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f409a59c640 1 -- 172.21.15.84:0/639043476 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409402cea0 con 0x7f409c0cd300 2023-10-11T06:14:41.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f40a121a640 1 -- 172.21.15.84:0/639043476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0cd300 msgr2=0x7f409c163e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f40a121a640 1 --2- 172.21.15.84:0/639043476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0cd300 0x7f409c163e10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4094009d90 tx=0x7f409402be20 comp rx=0 tx=0).stop 2023-10-11T06:14:41.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f40a121a640 1 -- 172.21.15.84:0/639043476 shutdown_connections 2023-10-11T06:14:41.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f40a121a640 1 --2- 172.21.15.84:0/639043476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0cd300 0x7f409c163e10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f40a121a640 1 -- 172.21.15.84:0/639043476 >> 172.21.15.84:0/639043476 conn(0x7f409c069c70 msgr2=0x7f409c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f40a121a640 1 -- 172.21.15.84:0/639043476 shutdown_connections 2023-10-11T06:14:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.257+0000 7f40a121a640 1 -- 172.21.15.84:0/639043476 wait complete. 2023-10-11T06:14:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.259+0000 7f40a121a640 1 Processor -- start 2023-10-11T06:14:41.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.259+0000 7f40a121a640 1 -- start start 2023-10-11T06:14:41.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f40a121a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0adb90 0x7f409c1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f40a121a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f409c0cd300 0x7f409c150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f40a121a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0ce0d0 0x7f409c0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f40a121a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b09f0 con 0x7f409c0ce0d0 2023-10-11T06:14:41.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f40a121a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b0100 con 0x7f409c0adb90 2023-10-11T06:14:41.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f40a121a640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0c8310 con 0x7f409c0cd300 2023-10-11T06:14:41.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f409b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0ce0d0 0x7f409c0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f409b59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0ce0d0 0x7f409c0c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36998/0 (socket says 172.21.15.84:36998) 2023-10-11T06:14:41.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.260+0000 7f409b59e640 1 -- 172.21.15.84:0/3917028783 learned_addr learned my addr 172.21.15.84:0/3917028783 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:41.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409b59e640 1 -- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f409c0cd300 msgr2=0x7f409c150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409b59e640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f409c0cd300 0x7f409c150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409b59e640 1 -- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0adb90 msgr2=0x7f409c1505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409ad9d640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0adb90 0x7f409c1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409b59e640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0adb90 0x7f409c1505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409b59e640 1 -- 172.21.15.84:0/3917028783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009a70 con 0x7f409c0ce0d0 2023-10-11T06:14:41.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409ad9d640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0adb90 0x7f409c1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:41.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f409b59e640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0ce0d0 0x7f409c0c6c30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4094009d90 tx=0x7f4094004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:41.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.261+0000 7f40937fe640 1 -- 172.21.15.84:0/3917028783 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409402cea0 con 0x7f409c0ce0d0 2023-10-11T06:14:41.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.262+0000 7f40937fe640 1 -- 172.21.15.84:0/3917028783 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094040450 con 0x7f409c0ce0d0 2023-10-11T06:14:41.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.262+0000 7f40937fe640 1 -- 172.21.15.84:0/3917028783 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940407e0 con 0x7f409c0ce0d0 2023-10-11T06:14:41.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.262+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c0b03a0 con 0x7f409c0ce0d0 2023-10-11T06:14:41.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.262+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c0b0e80 con 0x7f409c0ce0d0 2023-10-11T06:14:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.263+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c0b0b60 con 0x7f409c0ce0d0 2023-10-11T06:14:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.264+0000 7f40937fe640 1 -- 172.21.15.84:0/3917028783 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4094040940 con 0x7f409c0ce0d0 2023-10-11T06:14:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.264+0000 7f40937fe640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f407c06a830 0x7f407c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.265+0000 7f40937fe640 1 -- 172.21.15.84:0/3917028783 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f40940ab750 con 0x7f409c0ce0d0 2023-10-11T06:14:41.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.265+0000 7f409ad9d640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f407c06a830 0x7f407c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.265+0000 7f409ad9d640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f407c06a830 0x7f407c06cce0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f408c00a830 tx=0x7f408c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:41.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.270+0000 7f40937fe640 1 -- 172.21.15.84:0/3917028783 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f4094076d10 con 0x7f409c0ce0d0 2023-10-11T06:14:41.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.460+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f409c073ed0 con 0x7f409c0ce0d0 2023-10-11T06:14:41.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.468+0000 7f40937fe640 1 -- 172.21.15.84:0/3917028783 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f409407a330 con 0x7f409c0ce0d0 2023-10-11T06:14:41.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f407c06a830 msgr2=0x7f407c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f407c06a830 0x7f407c06cce0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f408c00a830 tx=0x7f408c008040 comp rx=0 tx=0).stop 2023-10-11T06:14:41.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0ce0d0 msgr2=0x7f409c0c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0ce0d0 0x7f409c0c6c30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4094009d90 tx=0x7f4094004a40 comp rx=0 tx=0).stop 2023-10-11T06:14:41.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 shutdown_connections 2023-10-11T06:14:41.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f407c06a830 0x7f407c06cce0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f409c0ce0d0 0x7f409c0c6c30 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f409c0cd300 0x7f409c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 --2- 172.21.15.84:0/3917028783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0adb90 0x7f409c1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 >> 172.21.15.84:0/3917028783 conn(0x7f409c069c70 msgr2=0x7f409c15cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:41.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 shutdown_connections 2023-10-11T06:14:41.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.471+0000 7f40a121a640 1 -- 172.21.15.84:0/3917028783 wait complete. 2023-10-11T06:14:41.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-10-11T06:14:41.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.704+0000 7fc95787d640 1 Processor -- start 2023-10-11T06:14:41.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.704+0000 7fc95787d640 1 -- start start 2023-10-11T06:14:41.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.706+0000 7fc95787d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc95015c290 0x7fc9500c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.706+0000 7fc95787d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500cb6c0 0x7fc9500cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.706+0000 7fc95787d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95015fe40 con 0x7fc9500c9190 2023-10-11T06:14:41.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.706+0000 7fc95787d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9500cad70 con 0x7fc95015c290 2023-10-11T06:14:41.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.706+0000 7fc95787d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95006ff30 con 0x7fc9500cb6c0 2023-10-11T06:14:41.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.707+0000 7fc955e1a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500cb6c0 0x7fc9500cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.707+0000 7fc955619640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc95015c290 0x7fc9500c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.707+0000 7fc955619640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc95015c290 0x7fc9500c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:60002/0 (socket says 172.21.15.84:60002) 2023-10-11T06:14:41.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.707+0000 7fc955e1a640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500cb6c0 0x7fc9500cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:36516/0 (socket says 172.21.15.84:36516) 2023-10-11T06:14:41.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.707+0000 7fc955619640 1 -- 172.21.15.84:0/2530140748 learned_addr learned my addr 172.21.15.84:0/2530140748 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:41.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.707+0000 7fc954e18640 1 --1- 172.21.15.84:0/2530140748 >> v1:172.21.15.99:6789/0 conn(0x7fc9500c9190 0x7fc9500cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.84:44930/0 (socket says 172.21.15.84:44930) 2023-10-11T06:14:41.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.707+0000 7fc955619640 1 -- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500cb6c0 msgr2=0x7fc9500cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:41.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.708+0000 7fc955619640 1 --2- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500cb6c0 0x7fc9500cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.708+0000 7fc955619640 1 -- 172.21.15.84:0/2530140748 >> v1:172.21.15.99:6789/0 conn(0x7fc9500c9190 legacy=0x7fc9500cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.708+0000 7fc955619640 1 -- 172.21.15.84:0/2530140748 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9501600e0 con 0x7fc95015c290 2023-10-11T06:14:41.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.708+0000 7fc947fff640 1 -- 172.21.15.84:0/2530140748 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795343836 0 0) 0x7fc95015fe40 con 0x7fc9500c9190 2023-10-11T06:14:41.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.708+0000 7fc955e1a640 1 --2- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500cb6c0 0x7fc9500cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:14:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.709+0000 7fc955619640 1 --2- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 0x7fc9500c8c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009850 tx=0x7fc94c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b067e76829da65 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc947fff640 1 -- 172.21.15.84:0/2530140748 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c004230 con 0x7fc95015c290 2023-10-11T06:14:41.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc947fff640 1 -- 172.21.15.84:0/2530140748 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c004390 con 0x7fc95015c290 2023-10-11T06:14:41.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc947fff640 1 -- 172.21.15.84:0/2530140748 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c003810 con 0x7fc95015c290 2023-10-11T06:14:41.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc95787d640 1 -- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 msgr2=0x7fc9500c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc95787d640 1 --2- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 0x7fc9500c8c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009850 tx=0x7fc94c02b960 comp rx=0 tx=0).stop 2023-10-11T06:14:41.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc95787d640 1 -- 172.21.15.84:0/2530140748 shutdown_connections 2023-10-11T06:14:41.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc95787d640 1 --2- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500cb6c0 0x7fc9500cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc95787d640 1 --2- 172.21.15.84:0/2530140748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 0x7fc9500c8c50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.710+0000 7fc95787d640 1 -- 172.21.15.84:0/2530140748 >> 172.21.15.84:0/2530140748 conn(0x7fc950069c70 msgr2=0x7fc95006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:41.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.711+0000 7fc95787d640 1 -- 172.21.15.84:0/2530140748 shutdown_connections 2023-10-11T06:14:41.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.711+0000 7fc95787d640 1 -- 172.21.15.84:0/2530140748 wait complete. 2023-10-11T06:14:41.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.712+0000 7fc95787d640 1 Processor -- start 2023-10-11T06:14:41.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.712+0000 7fc95787d640 1 -- start start 2023-10-11T06:14:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.713+0000 7fc95787d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500c9190 0x7fc95014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.713+0000 7fc95787d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9500cb6c0 0x7fc950159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.713+0000 7fc95787d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 0x7fc9500b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.713+0000 7fc95787d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950159a70 con 0x7fc9500cb6c0 2023-10-11T06:14:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.713+0000 7fc95787d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95014f790 con 0x7fc95015c290 2023-10-11T06:14:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.713+0000 7fc95787d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc950159d10 con 0x7fc9500c9190 2023-10-11T06:14:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9500cb6c0 0x7fc950159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9500cb6c0 0x7fc950159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37014/0 (socket says 172.21.15.84:37014) 2023-10-11T06:14:41.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 -- 172.21.15.84:0/378011157 learned_addr learned my addr 172.21.15.84:0/378011157 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc955619640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500c9190 0x7fc95014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 -- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500c9190 msgr2=0x7fc95014f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500c9190 0x7fc95014f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 -- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 msgr2=0x7fc9500b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc955e1a640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 0x7fc9500b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 0x7fc9500b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 -- 172.21.15.84:0/378011157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948009a70 con 0x7fc9500cb6c0 2023-10-11T06:14:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.714+0000 7fc954e18640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9500cb6c0 0x7fc950159530 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c4b0 tx=0x7fc94000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:41.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.715+0000 7fc945ffb640 1 -- 172.21.15.84:0/378011157 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9400074e0 con 0x7fc9500cb6c0 2023-10-11T06:14:41.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.715+0000 7fc945ffb640 1 -- 172.21.15.84:0/378011157 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940007660 con 0x7fc9500cb6c0 2023-10-11T06:14:41.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.715+0000 7fc945ffb640 1 -- 172.21.15.84:0/378011157 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940004a20 con 0x7fc9500cb6c0 2023-10-11T06:14:41.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.715+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c009460 con 0x7fc9500cb6c0 2023-10-11T06:14:41.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.715+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9500c32d0 con 0x7fc9500cb6c0 2023-10-11T06:14:41.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.716+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950073ed0 con 0x7fc9500cb6c0 2023-10-11T06:14:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.717+0000 7fc945ffb640 1 -- 172.21.15.84:0/378011157 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc940004b80 con 0x7fc9500cb6c0 2023-10-11T06:14:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.718+0000 7fc945ffb640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc93406a830 0x7fc93406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.718+0000 7fc945ffb640 1 -- 172.21.15.84:0/378011157 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc940087f70 con 0x7fc9500cb6c0 2023-10-11T06:14:41.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.718+0000 7fc955619640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc93406a830 0x7fc93406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:41.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.719+0000 7fc955619640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc93406a830 0x7fc93406cce0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc94c035000 tx=0x7fc94c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:41.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.722+0000 7fc945ffb640 1 -- 172.21.15.84:0/378011157 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc940053530 con 0x7fc9500cb6c0 2023-10-11T06:14:41.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:41.987+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fc9500c1cb0 con 0x7fc9500cb6c0 2023-10-11T06:14:42.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.028+0000 7fc945ffb640 1 -- 172.21.15.84:0/378011157 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v208) v1 ==== 134+0+0 (secure 0 0 0) 0x7fc9400180c0 con 0x7fc9500cb6c0 2023-10-11T06:14:42.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc93406a830 msgr2=0x7fc93406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:42.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc93406a830 0x7fc93406cce0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc94c035000 tx=0x7fc94c008030 comp rx=0 tx=0).stop 2023-10-11T06:14:42.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9500cb6c0 msgr2=0x7fc950159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:42.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9500cb6c0 0x7fc950159530 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c4b0 tx=0x7fc94000c970 comp rx=0 tx=0).stop 2023-10-11T06:14:42.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 shutdown_connections 2023-10-11T06:14:42.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc93406a830 0x7fc93406cce0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:42.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc95015c290 0x7fc9500b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:42.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9500cb6c0 0x7fc950159530 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:42.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 --2- 172.21.15.84:0/378011157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9500c9190 0x7fc95014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:42.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 >> 172.21.15.84:0/378011157 conn(0x7fc950069c70 msgr2=0x7fc9500ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:42.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 shutdown_connections 2023-10-11T06:14:42.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:42.032+0000 7fc95787d640 1 -- 172.21.15.84:0/378011157 wait complete. 2023-10-11T06:14:42.051 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2023-10-11T06:14:42.052 INFO:tasks.cephfs_test_runner: 2023-10-11T06:14:42.052 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-11T06:14:42.052 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-11T06:14:42.053 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-11T06:14:42.053 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-11T06:14:42.053 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-10-11T06:14:42.053 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-10-11T06:14:42.054 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/mount.py", line 771, in run_python 2023-10-11T06:14:42.054 INFO:tasks.cephfs_test_runner: p.wait() 2023-10-11T06:14:42.054 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-11T06:14:42.054 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-11T06:14:42.055 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-11T06:14:42.055 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-11T06:14:42.055 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi084 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-10-11T06:14:42.055 INFO:tasks.cephfs_test_runner: 2023-10-11T06:14:42.056 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-11T06:14:42.056 INFO:tasks.cephfs_test_runner:Ran 2 tests in 164.528s 2023-10-11T06:14:42.056 INFO:tasks.cephfs_test_runner: 2023-10-11T06:14:42.056 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-11T06:14:42.057 INFO:tasks.cephfs_test_runner: 2023-10-11T06:14:42.057 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-11T06:14:42.057 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-11T06:14:42.057 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-11T06:14:42.058 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-11T06:14:42.058 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-10-11T06:14:42.058 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-10-11T06:14:42.058 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs/mount.py", line 771, in run_python 2023-10-11T06:14:42.059 INFO:tasks.cephfs_test_runner: p.wait() 2023-10-11T06:14:42.059 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-11T06:14:42.059 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-11T06:14:42.059 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-11T06:14:42.059 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-11T06:14:42.060 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi084 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-10-11T06:14:42.060 INFO:tasks.cephfs_test_runner: 2023-10-11T06:14:42.060 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-11T06:14:42.122 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=43bdba28b8d24e76b4c5a7d20341f767 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a774796f0ecdd11a4b771de396e627aa1a187af/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-10-11T06:14:42.124 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-11T06:14:42.133 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-11T06:14:42.200 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-11T06:14:42.201 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:14:42.201 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:42.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053598,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T06:14:42.272 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-10-11T06:14:42.272 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:42.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053598,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-11T06:14:42.326 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:14:42.327 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:42.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:14:42.386 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2023-10-11T06:14:42.386 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-11T06:14:42.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053583,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248788k,nr_inodes=812197,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497580k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-11T06:14:42.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:42.439 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2023-10-11T06:14:42.492 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-11T06:14:42.493 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-11T06:14:42.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:42.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-11T06:14:42.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:42.587 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-11T06:14:42.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:42.680 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2023-10-11T06:14:42.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:42.810 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-10-11T06:14:42.872 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-11T06:14:42.872 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-11T06:14:42.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:42.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-11T06:14:42.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:42.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-11T06:14:43.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-11T06:14:43.065 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-10-11T06:14:43.196 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-11T06:14:43.206 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-11T06:14:43.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:14:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.398+0000 7f73dd8c7640 1 Processor -- start 2023-10-11T06:14:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.399+0000 7f73dd8c7640 1 -- start start 2023-10-11T06:14:43.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.400+0000 7f73dd8c7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f73d815d4f0 0x7f73d80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.400+0000 7f73dd8c7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d80ca300 0x7f73d80c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.400+0000 7f73dd8c7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73d80c9190 0x7f73d815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.400+0000 7f73dd8c7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d8072310 con 0x7f73d815d4f0 2023-10-11T06:14:43.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.400+0000 7f73dd8c7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d8071a90 con 0x7f73d80c9190 2023-10-11T06:14:43.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.400+0000 7f73dd8c7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8070280 con 0x7f73d80ca300 2023-10-11T06:14:43.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f73d815d4f0 0x7f73d80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f73d815d4f0 0x7f73d80cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37028/0 (socket says 172.21.15.84:37028) 2023-10-11T06:14:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 -- 172.21.15.84:0/2685948347 learned_addr learned my addr 172.21.15.84:0/2685948347 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d67fc640 1 --2- 172.21.15.84:0/2685948347 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d80ca300 0x7f73d80c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d77fe640 1 --2- 172.21.15.84:0/2685948347 >> v2:172.21.15.99:3300/0 conn(0x7f73d80c9190 0x7f73d815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 -- 172.21.15.84:0/2685948347 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d80ca300 msgr2=0x7f73d80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:43.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 --2- 172.21.15.84:0/2685948347 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d80ca300 0x7f73d80c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 -- 172.21.15.84:0/2685948347 >> v2:172.21.15.99:3300/0 conn(0x7f73d80c9190 msgr2=0x7f73d815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:43.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 --2- 172.21.15.84:0/2685948347 >> v2:172.21.15.99:3300/0 conn(0x7f73d80c9190 0x7f73d815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.401+0000 7f73d6ffd640 1 -- 172.21.15.84:0/2685948347 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8071cc0 con 0x7f73d815d4f0 2023-10-11T06:14:43.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.403+0000 7f73d6ffd640 1 --2- 172.21.15.84:0/2685948347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815d4f0 0x7f73d80cb540 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f73d80b4770 tx=0x7f73c802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cfa0846d2a889a server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:43.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.403+0000 7f73d5ffb640 1 -- 172.21.15.84:0/2685948347 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c802ca70 con 0x7f73d815d4f0 2023-10-11T06:14:43.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.403+0000 7f73d5ffb640 1 -- 172.21.15.84:0/2685948347 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c802cbd0 con 0x7f73d815d4f0 2023-10-11T06:14:43.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.403+0000 7f73d5ffb640 1 -- 172.21.15.84:0/2685948347 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c80343f0 con 0x7f73d815d4f0 2023-10-11T06:14:43.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 -- 172.21.15.84:0/2685948347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815d4f0 msgr2=0x7f73d80cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/2685948347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815d4f0 0x7f73d80cb540 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f73d80b4770 tx=0x7f73c802bed0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 -- 172.21.15.84:0/2685948347 shutdown_connections 2023-10-11T06:14:43.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/2685948347 >> v2:172.21.15.99:3300/0 conn(0x7f73d80c9190 0x7f73d815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/2685948347 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d80ca300 0x7f73d80c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/2685948347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d815d4f0 0x7f73d80cb540 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 -- 172.21.15.84:0/2685948347 >> 172.21.15.84:0/2685948347 conn(0x7f73d8069c70 msgr2=0x7f73d806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:43.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 -- 172.21.15.84:0/2685948347 shutdown_connections 2023-10-11T06:14:43.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.404+0000 7f73dd8c7640 1 -- 172.21.15.84:0/2685948347 wait complete. 2023-10-11T06:14:43.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.406+0000 7f73dd8c7640 1 Processor -- start 2023-10-11T06:14:43.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.406+0000 7f73dd8c7640 1 -- start start 2023-10-11T06:14:43.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73dd8c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d80c9190 0x7f73d81ef280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73dd8c7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d80ca300 0x7f73d80c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73dd8c7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d815d4f0 0x7f73d81591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73dd8c7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c79c0 con 0x7f73d80ca300 2023-10-11T06:14:43.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73dd8c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d81ef7c0 con 0x7f73d80c9190 2023-10-11T06:14:43.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73dd8c7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c6d40 con 0x7f73d815d4f0 2023-10-11T06:14:43.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73d67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d80ca300 0x7f73d80c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73d67fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d80ca300 0x7f73d80c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37044/0 (socket says 172.21.15.84:37044) 2023-10-11T06:14:43.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.407+0000 7f73d67fc640 1 -- 172.21.15.84:0/1814440406 learned_addr learned my addr 172.21.15.84:0/1814440406 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:43.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73d67fc640 1 -- 172.21.15.84:0/1814440406 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d815d4f0 msgr2=0x7f73d81591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73d67fc640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d815d4f0 0x7f73d81591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73d67fc640 1 -- 172.21.15.84:0/1814440406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d80c9190 msgr2=0x7f73d81ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73d67fc640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d80c9190 0x7f73d81ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73d67fc640 1 -- 172.21.15.84:0/1814440406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8009ad0 con 0x7f73d80ca300 2023-10-11T06:14:43.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73d67fc640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d80ca300 0x7f73d80c7de0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f73c000d5f0 tx=0x7f73c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:43.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73cffff640 1 -- 172.21.15.84:0/1814440406 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c00125f0 con 0x7f73d80ca300 2023-10-11T06:14:43.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73cffff640 1 -- 172.21.15.84:0/1814440406 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c0012750 con 0x7f73d80ca300 2023-10-11T06:14:43.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73cffff640 1 -- 172.21.15.84:0/1814440406 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0012b00 con 0x7f73d80ca300 2023-10-11T06:14:43.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d80c7020 con 0x7f73d80ca300 2023-10-11T06:14:43.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.408+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d80c75e0 con 0x7f73d80ca300 2023-10-11T06:14:43.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.410+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d80c7770 con 0x7f73d80ca300 2023-10-11T06:14:43.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.410+0000 7f73cffff640 1 -- 172.21.15.84:0/1814440406 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f73c0012c60 con 0x7f73d80ca300 2023-10-11T06:14:43.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.411+0000 7f73cffff640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f73b806a830 0x7f73b806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.411+0000 7f73cffff640 1 -- 172.21.15.84:0/1814440406 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f73c00891e0 con 0x7f73d80ca300 2023-10-11T06:14:43.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.412+0000 7f73d6ffd640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f73b806a830 0x7f73b806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.412+0000 7f73d6ffd640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f73b806a830 0x7f73b806cce0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f73c8035000 tx=0x7f73c802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:43.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.416+0000 7f73cffff640 1 -- 172.21.15.84:0/1814440406 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f73c00546e0 con 0x7f73d80ca300 2023-10-11T06:14:43.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.600+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73d8066fe0 con 0x7f73b806a830 2023-10-11T06:14:43.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.603+0000 7f73cffff640 1 -- 172.21.15.84:0/1814440406 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70580 (secure 0 0 0) 0x7f73d8066fe0 con 0x7f73b806a830 2023-10-11T06:14:43.605 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:14:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f73b806a830 msgr2=0x7f73b806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f73b806a830 0x7f73b806cce0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f73c8035000 tx=0x7f73c802e000 comp rx=0 tx=0).stop 2023-10-11T06:14:43.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d80ca300 msgr2=0x7f73d80c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d80ca300 0x7f73d80c7de0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f73c000d5f0 tx=0x7f73c000dab0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 shutdown_connections 2023-10-11T06:14:43.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f73b806a830 0x7f73b806cce0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d815d4f0 0x7f73d81591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d80ca300 0x7f73d80c7de0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 --2- 172.21.15.84:0/1814440406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d80c9190 0x7f73d81ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 >> 172.21.15.84:0/1814440406 conn(0x7f73d8069c70 msgr2=0x7f73d80ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:43.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 shutdown_connections 2023-10-11T06:14:43.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.606+0000 7f73dd8c7640 1 -- 172.21.15.84:0/1814440406 wait complete. 2023-10-11T06:14:43.625 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2023-10-11T06:14:42.839568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226080,"kb_used_data":6368,"kb_used_omap":0,"kb_used_meta":219648,"kb_avail":754748640,"statfs":{"total":773094113280,"available":772862607360,"internally_reserved":0,"allocated":6520832,"data_stored":3515904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":224919552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14369,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001941"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-11T06:14:38.489041+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:14:38.489041+0000","last_peered":"2023-10-11T06:14:38.489041+0000","last_clean":"2023-10-11T06:14:38.489041+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:14:38.489041+0000","last_undegraded":"2023-10-11T06:14:38.489041+0000","last_fullsized":"2023-10-11T06:14:38.489041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573324+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:13:47.573324+0000","last_peered":"2023-10-11T06:13:47.573324+0000","last_clean":"2023-10-11T06:13:47.573324+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:13:47.573324+0000","last_undegraded":"2023-10-11T06:13:47.573324+0000","last_fullsized":"2023-10-11T06:13:47.573324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.577119+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:13:47.577119+0000","last_peered":"2023-10-11T06:13:47.577119+0000","last_clean":"2023-10-11T06:13:47.577119+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:13:47.577119+0000","last_undegraded":"2023-10-11T06:13:47.577119+0000","last_fullsized":"2023-10-11T06:13:47.577119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573968+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:13:47.573968+0000","last_peered":"2023-10-11T06:13:47.573968+0000","last_clean":"2023-10-11T06:13:47.573968+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:13:47.573968+0000","last_undegraded":"2023-10-11T06:13:47.573968+0000","last_fullsized":"2023-10-11T06:13:47.573968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574755+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:13:47.574755+0000","last_peered":"2023-10-11T06:13:47.574755+0000","last_clean":"2023-10-11T06:13:47.574755+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:13:47.574755+0000","last_undegraded":"2023-10-11T06:13:47.574755+0000","last_fullsized":"2023-10-11T06:13:47.574755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-11T06:14:37.734913+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:14:37.734913+0000","last_peered":"2023-10-11T06:14:37.734913+0000","last_clean":"2023-10-11T06:14:37.734913+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:14:37.734913+0000","last_undegraded":"2023-10-11T06:14:37.734913+0000","last_fullsized":"2023-10-11T06:14:37.734913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575719+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:13:47.575719+0000","last_peered":"2023-10-11T06:13:47.575719+0000","last_clean":"2023-10-11T06:13:47.575719+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:13:47.575719+0000","last_undegraded":"2023-10-11T06:13:47.575719+0000","last_fullsized":"2023-10-11T06:13:47.575719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575109+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:13:47.575109+0000","last_peered":"2023-10-11T06:13:47.575109+0000","last_clean":"2023-10-11T06:13:47.575109+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:13:47.575109+0000","last_undegraded":"2023-10-11T06:13:47.575109+0000","last_fullsized":"2023-10-11T06:13:47.575109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575741+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:13:47.575741+0000","last_peered":"2023-10-11T06:13:47.575741+0000","last_clean":"2023-10-11T06:13:47.575741+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:13:47.575741+0000","last_undegraded":"2023-10-11T06:13:47.575741+0000","last_fullsized":"2023-10-11T06:13:47.575741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1257472,"data_stored":878291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.72599999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4379999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.171,"15min":1.161},"min":{"1min":0.78700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5569999999999999,"5min":1.788,"15min":1.788},"last":1.514}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.083,"15min":1.099},"min":{"1min":0.72499999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.383,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.8839999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.091,"15min":1.1180000000000001},"min":{"1min":0.63400000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.2529999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1040000000000001,"15min":1.1220000000000001},"min":{"1min":0.745,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4890000000000001,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.347},{"interface":"front","average":{"1min":1.167,"5min":1.26,"15min":1.284},"min":{"1min":0.92000000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.427,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.839}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.206,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.575,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.2350000000000001,"15min":1.256},"min":{"1min":0.83099999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.637,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.395}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.371,"15min":1.4059999999999999},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.645,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.071},{"interface":"front","average":{"1min":1.27,"5min":1.3080000000000001,"15min":1.3},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.7250000000000001,"5min":1.913,"15min":1.913},"last":1.4570000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3759999999999999,"15min":1.3899999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6870000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.53,"15min":1.5760000000000001},"min":{"1min":0.86199999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.8240000000000001,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.488}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4690000000000001,"15min":1.4730000000000001},"min":{"1min":0.747,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7929999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":2.02},{"interface":"front","average":{"1min":1.391,"5min":1.5269999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7829999999999999,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.766}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":655360,"data_stored":287763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.88700000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.024},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2130000000000001,"15min":1.216},"min":{"1min":0.82899999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.8,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.3300000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.038,"15min":1.038},"min":{"1min":0.69399999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.617,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.296},{"interface":"front","average":{"1min":1.167,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.76000000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.645,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.248}]},{"osd":2,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0680000000000001},"min":{"1min":0.78000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.21,"15min":1.2150000000000001},"min":{"1min":0.98999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5369999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.4139999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.874,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.542,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.236,"15min":1.2490000000000001},"min":{"1min":0.70999999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6339999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3400000000000001,"15min":1.373},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.667,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.681,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.468}]},{"osd":5,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.359,"15min":1.387},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.722,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.363},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4570000000000001,"15min":1.4670000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.748,"5min":1.992,"15min":1.992},"last":1.621}]},{"osd":7,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5609999999999999,"15min":1.589},"min":{"1min":0.81899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0979999999999999,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.51,"15min":1.538},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.792,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.5529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":288415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4590000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.032,"15min":1.022},"min":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4279999999999999,"5min":1.621,"15min":1.621},"last":1.024}]},{"osd":1,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.056},"min":{"1min":0.69899999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.052,"15min":1.0529999999999999},"min":{"1min":0.76500000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4259999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.81399999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.214,"15min":1.24},"min":{"1min":0.79000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.49,"5min":2.508,"15min":2.508},"last":1.145},{"interface":"front","average":{"1min":1.161,"5min":1.2869999999999999,"15min":1.345},"min":{"1min":0.76900000000000002,"5min":0.752,"15min":0.752},"max":{"1min":1.7689999999999999,"5min":2.524,"15min":2.524},"last":0.996}]},{"osd":3,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.82599999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5629999999999999,"5min":2.052,"15min":2.052},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.2470000000000001,"15min":1.2969999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.877,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2669999999999999,"15min":1.2829999999999999},"min":{"1min":0.746,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.516,"5min":2.242,"15min":2.242},"last":1.226},{"interface":"front","average":{"1min":1.3,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.84899999999999998,"5min":0.747,"15min":0.747},"max":{"1min":1.6619999999999999,"5min":2.427,"15min":2.427},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3720000000000001,"15min":1.4219999999999999},"min":{"1min":0.81599999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.96,"5min":2.472,"15min":2.472},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4399999999999999,"15min":1.456},"min":{"1min":1.1599999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9770000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.3400000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4179999999999999,"15min":1.4530000000000001},"min":{"1min":0.876,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.736,"5min":2.617,"15min":2.617},"last":1.266},{"interface":"front","average":{"1min":1.454,"5min":1.534,"15min":1.5740000000000001},"min":{"1min":1.0249999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.04,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.1739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":679936,"data_stored":303497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.052,"15min":1.0740000000000001},"min":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3979999999999999,"5min":1.841,"15min":1.841},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0009999999999999,"15min":0.96999999999999997},"min":{"1min":0.73499999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.103}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.105,"15min":1.087},"min":{"1min":0.77600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.194},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1040000000000001,"15min":1.127},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.4039999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0589999999999999,"15min":1.0569999999999999},"min":{"1min":0.87,"5min":0.63,"15min":0.63},"max":{"1min":1.444,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1930000000000001,"15min":1.1970000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.254}]},{"osd":3,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.79800000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.641,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.339},{"interface":"front","average":{"1min":1.165,"5min":1.194,"15min":1.2110000000000001},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.4390000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.288}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3080000000000001,"15min":1.319},"min":{"1min":0.80400000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.371,"15min":1.407},"min":{"1min":0.98699999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.746,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.2250000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.97099999999999997,"5min":0.627,"15min":0.627},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4590000000000001},"min":{"1min":1.0529999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.431}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5,"15min":1.548},"min":{"1min":1.0860000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.8400000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.046},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4610000000000001,"15min":1.4930000000000001},"min":{"1min":1.2390000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.2919999999999998,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.373}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":287819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.875,"15min":0.90100000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4910000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.93899999999999995,"15min":0.96199999999999997},"min":{"1min":0.47699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3420000000000001,"5min":2.097,"15min":2.097},"last":1.0489999999999999}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.399,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.052,"15min":1.0960000000000001},"min":{"1min":0.53400000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3129999999999999,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.165}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.99199999999999999,"15min":1.0329999999999999},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.246},{"interface":"front","average":{"1min":0.879,"5min":1.1160000000000001,"15min":1.1839999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.417,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":1.284}]},{"osd":4,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.222},"min":{"1min":0.79200000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.456,"5min":3.286,"15min":3.286},"last":1.393},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.84999999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":1.4359999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3700000000000001,"15min":1.4119999999999999},"min":{"1min":0.87,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.792,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.482},{"interface":"front","average":{"1min":1.171,"5min":1.2949999999999999,"15min":1.3320000000000001},"min":{"1min":0.78900000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.633,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.214}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.357,"15min":1.3819999999999999},"min":{"1min":0.84799999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8400000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.468,"15min":1.552},"min":{"1min":0.80800000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8999999999999999,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":1.353}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3300000000000001,"15min":1.3740000000000001},"min":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.554,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.569},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6950000000000001,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.6870000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":287819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79500000000000004,"15min":0.82699999999999996},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2649999999999999,"5min":1.494,"15min":1.494},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.53000000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.747}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.86499999999999999},"min":{"1min":0.498,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.173,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.131},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.44700000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3420000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.88300000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.91900000000000004},"min":{"1min":0.47199999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.6759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.127,"15min":1.216},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3720000000000001,"5min":3.032,"15min":3.032},"last":1.319}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2490000000000001,"15min":1.288},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.5029999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.23,"15min":1.2589999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6000000000000001,"5min":3.069,"15min":3.069},"last":1.429}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.216,"15min":1.242},"min":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5629999999999999,"5min":2.181,"15min":2.181},"last":1.159},{"interface":"front","average":{"1min":1.228,"5min":1.3240000000000001,"15min":1.385},"min":{"1min":0.84299999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6950000000000001,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.4650000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3340000000000001,"15min":1.361},"min":{"1min":0.81299999999999994,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.913,"5min":2.633,"15min":2.633},"last":1.355},{"interface":"front","average":{"1min":1.302,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6719999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.405}]},{"osd":7,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.367,"15min":1.415},"min":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7430000000000001,"5min":2.794,"15min":2.794},"last":1.111},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3120000000000001,"15min":1.335},"min":{"1min":0.95199999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5960000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.2769999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":287853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.131,"5min":1.302,"15min":1.302},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.86199999999999999},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2190000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.498,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4530000000000001,"5min":1.768,"15min":1.768},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0009999999999999},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5960000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.044},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4990000000000001,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.85799999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2609999999999999,"15min":1.2869999999999999},"min":{"1min":0.88,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8580000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.236},{"interface":"front","average":{"1min":1.272,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.873,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0259999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2609999999999999,"15min":1.2929999999999999},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7689999999999999,"5min":3.319,"15min":3.319},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.224,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.82499999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.9930000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.141}]},{"osd":6,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.292,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.931,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.284,"15min":1.2989999999999999},"min":{"1min":0.76200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7070000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.026}]},{"osd":7,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3600000000000001,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.575,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.327},{"interface":"front","average":{"1min":1.329,"5min":1.401,"15min":1.425},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.3460000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.97899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1277952,"data_stored":894447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.46200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.077},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.98799999999999999,"15min":1.0600000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.329,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":1.0249999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3300000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.49099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.278,"5min":2.444,"15min":2.444},"last":0.87}]},{"osd":3,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.84099999999999997,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.2010000000000001,"5min":1.474,"15min":1.474},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.083,"15min":1.1200000000000001},"min":{"1min":0.52600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.514,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.4159999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2969999999999999,"15min":1.3300000000000001},"min":{"1min":0.91300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.55,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.262,"15min":1.2889999999999999},"min":{"1min":0.85899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5089999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.163}]},{"osd":5,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5820000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.337},{"interface":"front","average":{"1min":1.129,"5min":1.266,"15min":1.29},"min":{"1min":0.78600000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5800000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.2350000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.327,"15min":1.3580000000000001},"min":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.605,"5min":2.552,"15min":2.552},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.325,"15min":1.373},"min":{"1min":0.874,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.716,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.341,"15min":1.3879999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6559999999999999,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3560000000000001},"min":{"1min":0.85899999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.722,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.264}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:14:43.628 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:14:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.833+0000 7f23a6977640 1 Processor -- start 2023-10-11T06:14:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.833+0000 7f23a6977640 1 -- start start 2023-10-11T06:14:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f23a6977640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a015d4f0 0x7f23a00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f23a6977640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f23a00ca300 0x7f23a00c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f23a6977640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f23a00c9190 0x7f23a015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f23a6977640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a0072310 con 0x7f23a00ca300 2023-10-11T06:14:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f23a6977640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a0071a90 con 0x7f23a00c9190 2023-10-11T06:14:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f23a6977640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0070280 con 0x7f23a015d4f0 2023-10-11T06:14:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f2397fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f23a00ca300 0x7f23a00c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f2397fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f23a00ca300 0x7f23a00c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37056/0 (socket says 172.21.15.84:37056) 2023-10-11T06:14:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.835+0000 7f2397fff640 1 -- 172.21.15.84:0/1723047043 learned_addr learned my addr 172.21.15.84:0/1723047043 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:43.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.836+0000 7f2397fff640 1 -- 172.21.15.84:0/1723047043 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a015d4f0 msgr2=0x7f23a00cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:43.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.836+0000 7f2397fff640 1 --2- 172.21.15.84:0/1723047043 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a015d4f0 0x7f23a00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.836+0000 7f2397fff640 1 -- 172.21.15.84:0/1723047043 >> v2:172.21.15.99:3300/0 conn(0x7f23a00c9190 msgr2=0x7f23a015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:43.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.836+0000 7f2397fff640 1 --2- 172.21.15.84:0/1723047043 >> v2:172.21.15.99:3300/0 conn(0x7f23a00c9190 0x7f23a015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.836+0000 7f2397fff640 1 -- 172.21.15.84:0/1723047043 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0071cc0 con 0x7f23a00ca300 2023-10-11T06:14:43.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.837+0000 7f2397fff640 1 --2- 172.21.15.84:0/1723047043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00ca300 0x7f23a00c8c50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f23a00b4770 tx=0x7f238802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e8188171a4c1bc server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:43.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.837+0000 7f239f7fe640 1 -- 172.21.15.84:0/1723047043 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388031870 con 0x7f23a00ca300 2023-10-11T06:14:43.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.837+0000 7f239f7fe640 1 -- 172.21.15.84:0/1723047043 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23880319d0 con 0x7f23a00ca300 2023-10-11T06:14:43.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.837+0000 7f239f7fe640 1 -- 172.21.15.84:0/1723047043 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388031d60 con 0x7f23a00ca300 2023-10-11T06:14:43.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 -- 172.21.15.84:0/1723047043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00ca300 msgr2=0x7f23a00c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 --2- 172.21.15.84:0/1723047043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00ca300 0x7f23a00c8c50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f23a00b4770 tx=0x7f238802b910 comp rx=0 tx=0).stop 2023-10-11T06:14:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 -- 172.21.15.84:0/1723047043 shutdown_connections 2023-10-11T06:14:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 --2- 172.21.15.84:0/1723047043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00ca300 0x7f23a00c8c50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 --2- 172.21.15.84:0/1723047043 >> v2:172.21.15.99:3300/0 conn(0x7f23a00c9190 0x7f23a015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 --2- 172.21.15.84:0/1723047043 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a015d4f0 0x7f23a00cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 -- 172.21.15.84:0/1723047043 >> 172.21.15.84:0/1723047043 conn(0x7f23a0069c70 msgr2=0x7f23a006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 -- 172.21.15.84:0/1723047043 shutdown_connections 2023-10-11T06:14:43.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.838+0000 7f23a6977640 1 -- 172.21.15.84:0/1723047043 wait complete. 2023-10-11T06:14:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.840+0000 7f23a6977640 1 Processor -- start 2023-10-11T06:14:43.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.840+0000 7f23a6977640 1 -- start start 2023-10-11T06:14:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.841+0000 7f23a6977640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00c9190 0x7f23a014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.841+0000 7f23a6977640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a00ca300 0x7f23a00ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.841+0000 7f23a6977640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23a015d4f0 0x7f23a00ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.841+0000 7f23a6977640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00cd5c0 con 0x7f23a00c9190 2023-10-11T06:14:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.841+0000 7f23a6977640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00cd8a0 con 0x7f23a015d4f0 2023-10-11T06:14:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.841+0000 7f23a6977640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00cdb80 con 0x7f23a00ca300 2023-10-11T06:14:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.841+0000 7f239ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00c9190 0x7f23a014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00c9190 0x7f23a014e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37072/0 (socket says 172.21.15.84:37072) 2023-10-11T06:14:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 -- 172.21.15.84:0/1753202020 learned_addr learned my addr 172.21.15.84:0/1753202020 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 -- 172.21.15.84:0/1753202020 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a00ca300 msgr2=0x7f23a00ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a00ca300 0x7f23a00ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 -- 172.21.15.84:0/1753202020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23a015d4f0 msgr2=0x7f23a00ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23a015d4f0 0x7f23a00ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 -- 172.21.15.84:0/1753202020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880094d0 con 0x7f23a00c9190 2023-10-11T06:14:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239ffff640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00c9190 0x7f23a014e900 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f239000b3f0 tx=0x7f239000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:43.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239d7fa640 1 -- 172.21.15.84:0/1753202020 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390005970 con 0x7f23a00c9190 2023-10-11T06:14:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239d7fa640 1 -- 172.21.15.84:0/1753202020 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2390005ad0 con 0x7f23a00c9190 2023-10-11T06:14:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a00cde60 con 0x7f23a00c9190 2023-10-11T06:14:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.842+0000 7f239d7fa640 1 -- 172.21.15.84:0/1753202020 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390005e80 con 0x7f23a00c9190 2023-10-11T06:14:43.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.843+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a0156e10 con 0x7f23a00c9190 2023-10-11T06:14:43.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.844+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a0073ed0 con 0x7f23a00c9190 2023-10-11T06:14:43.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.845+0000 7f239d7fa640 1 -- 172.21.15.84:0/1753202020 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2390011630 con 0x7f23a00c9190 2023-10-11T06:14:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.846+0000 7f239d7fa640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f238006a830 0x7f238006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.846+0000 7f239d7fa640 1 -- 172.21.15.84:0/1753202020 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f23900889d0 con 0x7f23a00c9190 2023-10-11T06:14:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.846+0000 7f2397fff640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f238006a830 0x7f238006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:43.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.847+0000 7f2397fff640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f238006a830 0x7f238006cce0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2388036000 tx=0x7f2388030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:43.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:43.851+0000 7f239d7fa640 1 -- 172.21.15.84:0/1753202020 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2390053f60 con 0x7f23a00c9190 2023-10-11T06:14:44.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.036+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23a015d8c0 con 0x7f238006a830 2023-10-11T06:14:44.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.040+0000 7f239d7fa640 1 -- 172.21.15.84:0/1753202020 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70580 (secure 0 0 0) 0x7f2388031c40 con 0x7f238006a830 2023-10-11T06:14:44.043 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:44.044 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:14:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f238006a830 msgr2=0x7f238006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f238006a830 0x7f238006cce0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2388036000 tx=0x7f2388030000 comp rx=0 tx=0).stop 2023-10-11T06:14:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00c9190 msgr2=0x7f23a014e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00c9190 0x7f23a014e900 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f239000b3f0 tx=0x7f239000b8b0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 shutdown_connections 2023-10-11T06:14:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f238006a830 0x7f238006cce0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23a015d4f0 0x7f23a00ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23a00ca300 0x7f23a00ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 --2- 172.21.15.84:0/1753202020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23a00c9190 0x7f23a014e900 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 >> 172.21.15.84:0/1753202020 conn(0x7f23a0069c70 msgr2=0x7f23a00b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.043+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 shutdown_connections 2023-10-11T06:14:44.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.044+0000 7f23a6977640 1 -- 172.21.15.84:0/1753202020 wait complete. 2023-10-11T06:14:44.063 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2023-10-11T06:14:42.839568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226080,"kb_used_data":6368,"kb_used_omap":0,"kb_used_meta":219648,"kb_avail":754748640,"statfs":{"total":773094113280,"available":772862607360,"internally_reserved":0,"allocated":6520832,"data_stored":3515904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":224919552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14369,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001941"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-11T06:14:38.489041+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:14:38.489041+0000","last_peered":"2023-10-11T06:14:38.489041+0000","last_clean":"2023-10-11T06:14:38.489041+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:14:38.489041+0000","last_undegraded":"2023-10-11T06:14:38.489041+0000","last_fullsized":"2023-10-11T06:14:38.489041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573324+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:13:47.573324+0000","last_peered":"2023-10-11T06:13:47.573324+0000","last_clean":"2023-10-11T06:13:47.573324+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:13:47.573324+0000","last_undegraded":"2023-10-11T06:13:47.573324+0000","last_fullsized":"2023-10-11T06:13:47.573324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.577119+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:13:47.577119+0000","last_peered":"2023-10-11T06:13:47.577119+0000","last_clean":"2023-10-11T06:13:47.577119+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:13:47.577119+0000","last_undegraded":"2023-10-11T06:13:47.577119+0000","last_fullsized":"2023-10-11T06:13:47.577119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573968+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:13:47.573968+0000","last_peered":"2023-10-11T06:13:47.573968+0000","last_clean":"2023-10-11T06:13:47.573968+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:13:47.573968+0000","last_undegraded":"2023-10-11T06:13:47.573968+0000","last_fullsized":"2023-10-11T06:13:47.573968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574755+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:13:47.574755+0000","last_peered":"2023-10-11T06:13:47.574755+0000","last_clean":"2023-10-11T06:13:47.574755+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:13:47.574755+0000","last_undegraded":"2023-10-11T06:13:47.574755+0000","last_fullsized":"2023-10-11T06:13:47.574755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-11T06:14:37.734913+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:14:37.734913+0000","last_peered":"2023-10-11T06:14:37.734913+0000","last_clean":"2023-10-11T06:14:37.734913+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:14:37.734913+0000","last_undegraded":"2023-10-11T06:14:37.734913+0000","last_fullsized":"2023-10-11T06:14:37.734913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575719+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:13:47.575719+0000","last_peered":"2023-10-11T06:13:47.575719+0000","last_clean":"2023-10-11T06:13:47.575719+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:13:47.575719+0000","last_undegraded":"2023-10-11T06:13:47.575719+0000","last_fullsized":"2023-10-11T06:13:47.575719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575109+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:13:47.575109+0000","last_peered":"2023-10-11T06:13:47.575109+0000","last_clean":"2023-10-11T06:13:47.575109+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:13:47.575109+0000","last_undegraded":"2023-10-11T06:13:47.575109+0000","last_fullsized":"2023-10-11T06:13:47.575109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575741+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:13:47.575741+0000","last_peered":"2023-10-11T06:13:47.575741+0000","last_clean":"2023-10-11T06:13:47.575741+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:13:47.575741+0000","last_undegraded":"2023-10-11T06:13:47.575741+0000","last_fullsized":"2023-10-11T06:13:47.575741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1257472,"data_stored":878291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.72599999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4379999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.171,"15min":1.161},"min":{"1min":0.78700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5569999999999999,"5min":1.788,"15min":1.788},"last":1.514}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.083,"15min":1.099},"min":{"1min":0.72499999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.383,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.8839999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.091,"15min":1.1180000000000001},"min":{"1min":0.63400000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.2529999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1040000000000001,"15min":1.1220000000000001},"min":{"1min":0.745,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4890000000000001,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.347},{"interface":"front","average":{"1min":1.167,"5min":1.26,"15min":1.284},"min":{"1min":0.92000000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.427,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.839}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.206,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.575,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.2350000000000001,"15min":1.256},"min":{"1min":0.83099999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.637,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.395}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.371,"15min":1.4059999999999999},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.645,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.071},{"interface":"front","average":{"1min":1.27,"5min":1.3080000000000001,"15min":1.3},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.7250000000000001,"5min":1.913,"15min":1.913},"last":1.4570000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3759999999999999,"15min":1.3899999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6870000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.53,"15min":1.5760000000000001},"min":{"1min":0.86199999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.8240000000000001,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.488}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4690000000000001,"15min":1.4730000000000001},"min":{"1min":0.747,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7929999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":2.02},{"interface":"front","average":{"1min":1.391,"5min":1.5269999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7829999999999999,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.766}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":655360,"data_stored":287763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.88700000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.024},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2130000000000001,"15min":1.216},"min":{"1min":0.82899999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.8,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.3300000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.038,"15min":1.038},"min":{"1min":0.69399999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.617,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.296},{"interface":"front","average":{"1min":1.167,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.76000000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.645,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.248}]},{"osd":2,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0680000000000001},"min":{"1min":0.78000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.21,"15min":1.2150000000000001},"min":{"1min":0.98999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5369999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.4139999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.874,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.542,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.236,"15min":1.2490000000000001},"min":{"1min":0.70999999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6339999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3400000000000001,"15min":1.373},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.667,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.681,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.468}]},{"osd":5,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.359,"15min":1.387},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.722,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.363},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4570000000000001,"15min":1.4670000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.748,"5min":1.992,"15min":1.992},"last":1.621}]},{"osd":7,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5609999999999999,"15min":1.589},"min":{"1min":0.81899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0979999999999999,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.51,"15min":1.538},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.792,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.5529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":288415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4590000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.032,"15min":1.022},"min":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4279999999999999,"5min":1.621,"15min":1.621},"last":1.024}]},{"osd":1,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.056},"min":{"1min":0.69899999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.052,"15min":1.0529999999999999},"min":{"1min":0.76500000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4259999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.81399999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.214,"15min":1.24},"min":{"1min":0.79000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.49,"5min":2.508,"15min":2.508},"last":1.145},{"interface":"front","average":{"1min":1.161,"5min":1.2869999999999999,"15min":1.345},"min":{"1min":0.76900000000000002,"5min":0.752,"15min":0.752},"max":{"1min":1.7689999999999999,"5min":2.524,"15min":2.524},"last":0.996}]},{"osd":3,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.82599999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5629999999999999,"5min":2.052,"15min":2.052},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.2470000000000001,"15min":1.2969999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.877,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2669999999999999,"15min":1.2829999999999999},"min":{"1min":0.746,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.516,"5min":2.242,"15min":2.242},"last":1.226},{"interface":"front","average":{"1min":1.3,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.84899999999999998,"5min":0.747,"15min":0.747},"max":{"1min":1.6619999999999999,"5min":2.427,"15min":2.427},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3720000000000001,"15min":1.4219999999999999},"min":{"1min":0.81599999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.96,"5min":2.472,"15min":2.472},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4399999999999999,"15min":1.456},"min":{"1min":1.1599999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9770000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.3400000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4179999999999999,"15min":1.4530000000000001},"min":{"1min":0.876,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.736,"5min":2.617,"15min":2.617},"last":1.266},{"interface":"front","average":{"1min":1.454,"5min":1.534,"15min":1.5740000000000001},"min":{"1min":1.0249999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.04,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.1739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":679936,"data_stored":303497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.052,"15min":1.0740000000000001},"min":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3979999999999999,"5min":1.841,"15min":1.841},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0009999999999999,"15min":0.96999999999999997},"min":{"1min":0.73499999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.103}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.105,"15min":1.087},"min":{"1min":0.77600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.194},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1040000000000001,"15min":1.127},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.4039999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0589999999999999,"15min":1.0569999999999999},"min":{"1min":0.87,"5min":0.63,"15min":0.63},"max":{"1min":1.444,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1930000000000001,"15min":1.1970000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.254}]},{"osd":3,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.79800000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.641,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.339},{"interface":"front","average":{"1min":1.165,"5min":1.194,"15min":1.2110000000000001},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.4390000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.288}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3080000000000001,"15min":1.319},"min":{"1min":0.80400000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.371,"15min":1.407},"min":{"1min":0.98699999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.746,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.2250000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.97099999999999997,"5min":0.627,"15min":0.627},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4590000000000001},"min":{"1min":1.0529999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.431}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5,"15min":1.548},"min":{"1min":1.0860000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.8400000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.046},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4610000000000001,"15min":1.4930000000000001},"min":{"1min":1.2390000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.2919999999999998,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.373}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":287819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.875,"15min":0.90100000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4910000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.93899999999999995,"15min":0.96199999999999997},"min":{"1min":0.47699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3420000000000001,"5min":2.097,"15min":2.097},"last":1.0489999999999999}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.399,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.052,"15min":1.0960000000000001},"min":{"1min":0.53400000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3129999999999999,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.165}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.99199999999999999,"15min":1.0329999999999999},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.246},{"interface":"front","average":{"1min":0.879,"5min":1.1160000000000001,"15min":1.1839999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.417,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":1.284}]},{"osd":4,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.222},"min":{"1min":0.79200000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.456,"5min":3.286,"15min":3.286},"last":1.393},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.84999999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":1.4359999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3700000000000001,"15min":1.4119999999999999},"min":{"1min":0.87,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.792,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.482},{"interface":"front","average":{"1min":1.171,"5min":1.2949999999999999,"15min":1.3320000000000001},"min":{"1min":0.78900000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.633,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.214}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.357,"15min":1.3819999999999999},"min":{"1min":0.84799999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8400000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.468,"15min":1.552},"min":{"1min":0.80800000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8999999999999999,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":1.353}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3300000000000001,"15min":1.3740000000000001},"min":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.554,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.569},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6950000000000001,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.6870000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":287819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79500000000000004,"15min":0.82699999999999996},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2649999999999999,"5min":1.494,"15min":1.494},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.53000000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.747}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.86499999999999999},"min":{"1min":0.498,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.173,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.131},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.44700000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3420000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.88300000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.91900000000000004},"min":{"1min":0.47199999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.6759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.127,"15min":1.216},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3720000000000001,"5min":3.032,"15min":3.032},"last":1.319}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2490000000000001,"15min":1.288},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.5029999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.23,"15min":1.2589999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6000000000000001,"5min":3.069,"15min":3.069},"last":1.429}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.216,"15min":1.242},"min":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5629999999999999,"5min":2.181,"15min":2.181},"last":1.159},{"interface":"front","average":{"1min":1.228,"5min":1.3240000000000001,"15min":1.385},"min":{"1min":0.84299999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6950000000000001,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.4650000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3340000000000001,"15min":1.361},"min":{"1min":0.81299999999999994,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.913,"5min":2.633,"15min":2.633},"last":1.355},{"interface":"front","average":{"1min":1.302,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6719999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.405}]},{"osd":7,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.367,"15min":1.415},"min":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7430000000000001,"5min":2.794,"15min":2.794},"last":1.111},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3120000000000001,"15min":1.335},"min":{"1min":0.95199999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5960000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.2769999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":287853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.131,"5min":1.302,"15min":1.302},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.86199999999999999},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2190000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.498,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4530000000000001,"5min":1.768,"15min":1.768},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0009999999999999},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5960000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.044},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4990000000000001,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.85799999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2609999999999999,"15min":1.2869999999999999},"min":{"1min":0.88,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8580000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.236},{"interface":"front","average":{"1min":1.272,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.873,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0259999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2609999999999999,"15min":1.2929999999999999},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7689999999999999,"5min":3.319,"15min":3.319},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.224,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.82499999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.9930000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.141}]},{"osd":6,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.292,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.931,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.284,"15min":1.2989999999999999},"min":{"1min":0.76200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7070000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.026}]},{"osd":7,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3600000000000001,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.575,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.327},{"interface":"front","average":{"1min":1.329,"5min":1.401,"15min":1.425},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.3460000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.97899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1277952,"data_stored":894447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.46200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.077},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.98799999999999999,"15min":1.0600000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.329,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":1.0249999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3300000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.49099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.278,"5min":2.444,"15min":2.444},"last":0.87}]},{"osd":3,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.84099999999999997,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.2010000000000001,"5min":1.474,"15min":1.474},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.083,"15min":1.1200000000000001},"min":{"1min":0.52600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.514,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.4159999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2969999999999999,"15min":1.3300000000000001},"min":{"1min":0.91300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.55,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.262,"15min":1.2889999999999999},"min":{"1min":0.85899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5089999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.163}]},{"osd":5,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5820000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.337},{"interface":"front","average":{"1min":1.129,"5min":1.266,"15min":1.29},"min":{"1min":0.78600000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5800000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.2350000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.327,"15min":1.3580000000000001},"min":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.605,"5min":2.552,"15min":2.552},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.325,"15min":1.373},"min":{"1min":0.874,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.716,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.341,"15min":1.3879999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6559999999999999,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3560000000000001},"min":{"1min":0.85899999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.722,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.264}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:14:44.065 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-11T06:14:44.065 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:14:44.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.269+0000 7f9453281640 1 Processor -- start 2023-10-11T06:14:44.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.269+0000 7f9453281640 1 -- start start 2023-10-11T06:14:44.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.270+0000 7f9453281640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f944c15c290 0x7f944c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.270+0000 7f9453281640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f944c0d25e0 0x7f944c0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.270+0000 7f9453281640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944c0d2a60 con 0x7f944c0d0a50 2023-10-11T06:14:44.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f9453281640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c0d0e90 con 0x7f944c15c290 2023-10-11T06:14:44.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f9453281640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c06cd10 con 0x7f944c0d25e0 2023-10-11T06:14:44.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945101d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f944c15c290 0x7f944c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945181e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f944c0d0a50 0x7f944c06b0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34910/0 (socket says 172.21.15.84:34910) 2023-10-11T06:14:44.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945181e640 1 -- 172.21.15.84:0/1791302871 learned_addr learned my addr 172.21.15.84:0/1791302871 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:44.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945101d640 1 -- 172.21.15.84:0/1791302871 >> v2:172.21.15.99:3300/0 conn(0x7f944c0d25e0 msgr2=0x7f944c0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:44.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945081c640 1 --2- 172.21.15.84:0/1791302871 >> v2:172.21.15.99:3300/0 conn(0x7f944c0d25e0 0x7f944c0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945101d640 1 --2- 172.21.15.84:0/1791302871 >> v2:172.21.15.99:3300/0 conn(0x7f944c0d25e0 0x7f944c0d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945101d640 1 -- 172.21.15.84:0/1791302871 >> v1:172.21.15.84:6789/0 conn(0x7f944c0d0a50 legacy=0x7f944c06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f945101d640 1 -- 172.21.15.84:0/1791302871 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c0afb60 con 0x7f944c15c290 2023-10-11T06:14:44.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.271+0000 7f9443fff640 1 -- 172.21.15.84:0/1791302871 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851713330 0 0) 0x7f944c0d2a60 con 0x7f944c0d0a50 2023-10-11T06:14:44.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.272+0000 7f945101d640 1 --2- 172.21.15.84:0/1791302871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c15c290 0x7f944c0af4d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f944c0d0380 tx=0x7f944802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfa6f2f074ca3be server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:44.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9443fff640 1 -- 172.21.15.84:0/1791302871 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944802e070 con 0x7f944c15c290 2023-10-11T06:14:44.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9443fff640 1 -- 172.21.15.84:0/1791302871 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94480315a0 con 0x7f944c15c290 2023-10-11T06:14:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9443fff640 1 -- 172.21.15.84:0/1791302871 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448031930 con 0x7f944c15c290 2023-10-11T06:14:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 -- 172.21.15.84:0/1791302871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c15c290 msgr2=0x7f944c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 --2- 172.21.15.84:0/1791302871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c15c290 0x7f944c0af4d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f944c0d0380 tx=0x7f944802b840 comp rx=0 tx=0).stop 2023-10-11T06:14:44.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 -- 172.21.15.84:0/1791302871 shutdown_connections 2023-10-11T06:14:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 --2- 172.21.15.84:0/1791302871 >> v2:172.21.15.99:3300/0 conn(0x7f944c0d25e0 0x7f944c0d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 --2- 172.21.15.84:0/1791302871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c15c290 0x7f944c0af4d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 -- 172.21.15.84:0/1791302871 >> 172.21.15.84:0/1791302871 conn(0x7f944c071cc0 msgr2=0x7f944c09bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:44.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 -- 172.21.15.84:0/1791302871 shutdown_connections 2023-10-11T06:14:44.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.273+0000 7f9453281640 1 -- 172.21.15.84:0/1791302871 wait complete. 2023-10-11T06:14:44.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.275+0000 7f9453281640 1 Processor -- start 2023-10-11T06:14:44.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.275+0000 7f9453281640 1 -- start start 2023-10-11T06:14:44.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f9453281640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c0d0a50 0x7f944c1f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f9453281640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f944c0d25e0 0x7f944c0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f9453281640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f944c15c290 0x7f944c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f9453281640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c1f1420 con 0x7f944c0d0a50 2023-10-11T06:14:44.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f9453281640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c09ed20 con 0x7f944c15c290 2023-10-11T06:14:44.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f9453281640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c09f000 con 0x7f944c0d25e0 2023-10-11T06:14:44.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f945101d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c0d0a50 0x7f944c1f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f945101d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c0d0a50 0x7f944c1f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37090/0 (socket says 172.21.15.84:37090) 2023-10-11T06:14:44.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.276+0000 7f945101d640 1 -- 172.21.15.84:0/1554836318 learned_addr learned my addr 172.21.15.84:0/1554836318 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:44.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f945101d640 1 -- 172.21.15.84:0/1554836318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f944c0d25e0 msgr2=0x7f944c0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f945101d640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f944c0d25e0 0x7f944c0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f945101d640 1 -- 172.21.15.84:0/1554836318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f944c15c290 msgr2=0x7f944c1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f945101d640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f944c15c290 0x7f944c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f945101d640 1 -- 172.21.15.84:0/1554836318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9448009400 con 0x7f944c0d0a50 2023-10-11T06:14:44.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f945101d640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c0d0a50 0x7f944c1f0ee0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9448009890 tx=0x7f94480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:44.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f9441ffb640 1 -- 172.21.15.84:0/1554836318 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448032900 con 0x7f944c0d0a50 2023-10-11T06:14:44.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f9441ffb640 1 -- 172.21.15.84:0/1554836318 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448032a60 con 0x7f944c0d0a50 2023-10-11T06:14:44.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c09f2e0 con 0x7f944c0d0a50 2023-10-11T06:14:44.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.277+0000 7f9441ffb640 1 -- 172.21.15.84:0/1554836318 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448032e10 con 0x7f944c0d0a50 2023-10-11T06:14:44.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.278+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c0c9f50 con 0x7f944c0d0a50 2023-10-11T06:14:44.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.279+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c1f18d0 con 0x7f944c0d0a50 2023-10-11T06:14:44.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.279+0000 7f9441ffb640 1 -- 172.21.15.84:0/1554836318 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f944803b3f0 con 0x7f944c0d0a50 2023-10-11T06:14:44.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.281+0000 7f9441ffb640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f943006a860 0x7f943006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.281+0000 7f9441ffb640 1 -- 172.21.15.84:0/1554836318 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f94480aae70 con 0x7f944c0d0a50 2023-10-11T06:14:44.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.281+0000 7f945081c640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f943006a860 0x7f943006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.282+0000 7f945081c640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f943006a860 0x7f943006cd10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f943c006fd0 tx=0x7f943c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:44.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.286+0000 7f9441ffb640 1 -- 172.21.15.84:0/1554836318 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f944c1f18d0 con 0x7f944c0d0a50 2023-10-11T06:14:44.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.459+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f944c0b0120 con 0x7f943006a860 2023-10-11T06:14:44.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.463+0000 7f9441ffb640 1 -- 172.21.15.84:0/1554836318 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70580 (secure 0 0 0) 0x7f943c00ae20 con 0x7f943006a860 2023-10-11T06:14:44.466 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:44.467 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:14:44.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f943006a860 msgr2=0x7f943006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f943006a860 0x7f943006cd10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f943c006fd0 tx=0x7f943c008040 comp rx=0 tx=0).stop 2023-10-11T06:14:44.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c0d0a50 msgr2=0x7f944c1f0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c0d0a50 0x7f944c1f0ee0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9448009890 tx=0x7f94480046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 shutdown_connections 2023-10-11T06:14:44.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f943006a860 0x7f943006cd10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f944c15c290 0x7f944c1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f944c0d25e0 0x7f944c0c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 --2- 172.21.15.84:0/1554836318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f944c0d0a50 0x7f944c1f0ee0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 >> 172.21.15.84:0/1554836318 conn(0x7f944c071cc0 msgr2=0x7f944c159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:44.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 shutdown_connections 2023-10-11T06:14:44.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.466+0000 7f9453281640 1 -- 172.21.15.84:0/1554836318 wait complete. 2023-10-11T06:14:44.487 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2023-10-11T06:14:42.839568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226080,"kb_used_data":6368,"kb_used_omap":0,"kb_used_meta":219648,"kb_avail":754748640,"statfs":{"total":773094113280,"available":772862607360,"internally_reserved":0,"allocated":6520832,"data_stored":3515904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":224919552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14369,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001941"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-11T06:14:38.489041+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:14:38.489041+0000","last_peered":"2023-10-11T06:14:38.489041+0000","last_clean":"2023-10-11T06:14:38.489041+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:14:38.489041+0000","last_undegraded":"2023-10-11T06:14:38.489041+0000","last_fullsized":"2023-10-11T06:14:38.489041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573324+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:13:47.573324+0000","last_peered":"2023-10-11T06:13:47.573324+0000","last_clean":"2023-10-11T06:13:47.573324+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:13:47.573324+0000","last_undegraded":"2023-10-11T06:13:47.573324+0000","last_fullsized":"2023-10-11T06:13:47.573324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.577119+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:13:47.577119+0000","last_peered":"2023-10-11T06:13:47.577119+0000","last_clean":"2023-10-11T06:13:47.577119+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:13:47.577119+0000","last_undegraded":"2023-10-11T06:13:47.577119+0000","last_fullsized":"2023-10-11T06:13:47.577119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.573968+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:13:47.573968+0000","last_peered":"2023-10-11T06:13:47.573968+0000","last_clean":"2023-10-11T06:13:47.573968+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:13:47.573968+0000","last_undegraded":"2023-10-11T06:13:47.573968+0000","last_fullsized":"2023-10-11T06:13:47.573968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.574755+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:13:47.574755+0000","last_peered":"2023-10-11T06:13:47.574755+0000","last_clean":"2023-10-11T06:13:47.574755+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:13:47.574755+0000","last_undegraded":"2023-10-11T06:13:47.574755+0000","last_fullsized":"2023-10-11T06:13:47.574755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-11T06:14:37.734913+0000","last_change":"2023-10-11T06:11:08.323505+0000","last_active":"2023-10-11T06:14:37.734913+0000","last_peered":"2023-10-11T06:14:37.734913+0000","last_clean":"2023-10-11T06:14:37.734913+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:14:37.734913+0000","last_undegraded":"2023-10-11T06:14:37.734913+0000","last_fullsized":"2023-10-11T06:14:37.734913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575719+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:13:47.575719+0000","last_peered":"2023-10-11T06:13:47.575719+0000","last_clean":"2023-10-11T06:13:47.575719+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:13:47.575719+0000","last_undegraded":"2023-10-11T06:13:47.575719+0000","last_fullsized":"2023-10-11T06:13:47.575719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575109+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:13:47.575109+0000","last_peered":"2023-10-11T06:13:47.575109+0000","last_clean":"2023-10-11T06:13:47.575109+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:13:47.575109+0000","last_undegraded":"2023-10-11T06:13:47.575109+0000","last_fullsized":"2023-10-11T06:13:47.575109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-11T06:13:47.575741+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:13:47.575741+0000","last_peered":"2023-10-11T06:13:47.575741+0000","last_clean":"2023-10-11T06:13:47.575741+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:13:47.575741+0000","last_undegraded":"2023-10-11T06:13:47.575741+0000","last_fullsized":"2023-10-11T06:13:47.575741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1257472,"data_stored":878291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.72599999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4379999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.171,"15min":1.161},"min":{"1min":0.78700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5569999999999999,"5min":1.788,"15min":1.788},"last":1.514}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.083,"15min":1.099},"min":{"1min":0.72499999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.383,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.8839999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.091,"15min":1.1180000000000001},"min":{"1min":0.63400000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.2529999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1040000000000001,"15min":1.1220000000000001},"min":{"1min":0.745,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4890000000000001,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.347},{"interface":"front","average":{"1min":1.167,"5min":1.26,"15min":1.284},"min":{"1min":0.92000000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.427,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.839}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.206,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.575,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.2350000000000001,"15min":1.256},"min":{"1min":0.83099999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.637,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.395}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.371,"15min":1.4059999999999999},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.645,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.071},{"interface":"front","average":{"1min":1.27,"5min":1.3080000000000001,"15min":1.3},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.7250000000000001,"5min":1.913,"15min":1.913},"last":1.4570000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3759999999999999,"15min":1.3899999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6870000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.53,"15min":1.5760000000000001},"min":{"1min":0.86199999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.8240000000000001,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.488}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4690000000000001,"15min":1.4730000000000001},"min":{"1min":0.747,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7929999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":2.02},{"interface":"front","average":{"1min":1.391,"5min":1.5269999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7829999999999999,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.766}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":655360,"data_stored":287763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.88700000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.024},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2130000000000001,"15min":1.216},"min":{"1min":0.82899999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.8,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.3300000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.038,"15min":1.038},"min":{"1min":0.69399999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.617,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.296},{"interface":"front","average":{"1min":1.167,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.76000000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.645,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.248}]},{"osd":2,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0680000000000001},"min":{"1min":0.78000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.21,"15min":1.2150000000000001},"min":{"1min":0.98999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5369999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.4139999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.874,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.542,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.236,"15min":1.2490000000000001},"min":{"1min":0.70999999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6339999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3400000000000001,"15min":1.373},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.667,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.681,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.468}]},{"osd":5,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.359,"15min":1.387},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.722,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.363},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4570000000000001,"15min":1.4670000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.748,"5min":1.992,"15min":1.992},"last":1.621}]},{"osd":7,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5609999999999999,"15min":1.589},"min":{"1min":0.81899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0979999999999999,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.51,"15min":1.538},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.792,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.5529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":288415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4590000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.032,"15min":1.022},"min":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4279999999999999,"5min":1.621,"15min":1.621},"last":1.024}]},{"osd":1,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.056},"min":{"1min":0.69899999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.052,"15min":1.0529999999999999},"min":{"1min":0.76500000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4259999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.81399999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.214,"15min":1.24},"min":{"1min":0.79000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.49,"5min":2.508,"15min":2.508},"last":1.145},{"interface":"front","average":{"1min":1.161,"5min":1.2869999999999999,"15min":1.345},"min":{"1min":0.76900000000000002,"5min":0.752,"15min":0.752},"max":{"1min":1.7689999999999999,"5min":2.524,"15min":2.524},"last":0.996}]},{"osd":3,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.82599999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5629999999999999,"5min":2.052,"15min":2.052},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.2470000000000001,"15min":1.2969999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.877,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2669999999999999,"15min":1.2829999999999999},"min":{"1min":0.746,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.516,"5min":2.242,"15min":2.242},"last":1.226},{"interface":"front","average":{"1min":1.3,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.84899999999999998,"5min":0.747,"15min":0.747},"max":{"1min":1.6619999999999999,"5min":2.427,"15min":2.427},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3720000000000001,"15min":1.4219999999999999},"min":{"1min":0.81599999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.96,"5min":2.472,"15min":2.472},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4399999999999999,"15min":1.456},"min":{"1min":1.1599999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9770000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.3400000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4179999999999999,"15min":1.4530000000000001},"min":{"1min":0.876,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.736,"5min":2.617,"15min":2.617},"last":1.266},{"interface":"front","average":{"1min":1.454,"5min":1.534,"15min":1.5740000000000001},"min":{"1min":1.0249999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.04,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.1739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":679936,"data_stored":303497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.052,"15min":1.0740000000000001},"min":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3979999999999999,"5min":1.841,"15min":1.841},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0009999999999999,"15min":0.96999999999999997},"min":{"1min":0.73499999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.103}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.105,"15min":1.087},"min":{"1min":0.77600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.194},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1040000000000001,"15min":1.127},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.4039999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0589999999999999,"15min":1.0569999999999999},"min":{"1min":0.87,"5min":0.63,"15min":0.63},"max":{"1min":1.444,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1930000000000001,"15min":1.1970000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.254}]},{"osd":3,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.79800000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.641,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.339},{"interface":"front","average":{"1min":1.165,"5min":1.194,"15min":1.2110000000000001},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.4390000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.288}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3080000000000001,"15min":1.319},"min":{"1min":0.80400000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.371,"15min":1.407},"min":{"1min":0.98699999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.746,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.2250000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.97099999999999997,"5min":0.627,"15min":0.627},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4590000000000001},"min":{"1min":1.0529999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.431}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5,"15min":1.548},"min":{"1min":1.0860000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.8400000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.046},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4610000000000001,"15min":1.4930000000000001},"min":{"1min":1.2390000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.2919999999999998,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.373}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":287819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.875,"15min":0.90100000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4910000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.93899999999999995,"15min":0.96199999999999997},"min":{"1min":0.47699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3420000000000001,"5min":2.097,"15min":2.097},"last":1.0489999999999999}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.399,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.052,"15min":1.0960000000000001},"min":{"1min":0.53400000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3129999999999999,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.165}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.99199999999999999,"15min":1.0329999999999999},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.246},{"interface":"front","average":{"1min":0.879,"5min":1.1160000000000001,"15min":1.1839999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.417,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":1.284}]},{"osd":4,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.222},"min":{"1min":0.79200000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.456,"5min":3.286,"15min":3.286},"last":1.393},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.84999999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":1.4359999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3700000000000001,"15min":1.4119999999999999},"min":{"1min":0.87,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.792,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.482},{"interface":"front","average":{"1min":1.171,"5min":1.2949999999999999,"15min":1.3320000000000001},"min":{"1min":0.78900000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.633,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.214}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.357,"15min":1.3819999999999999},"min":{"1min":0.84799999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8400000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.468,"15min":1.552},"min":{"1min":0.80800000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8999999999999999,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":1.353}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3300000000000001,"15min":1.3740000000000001},"min":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.554,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.569},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6950000000000001,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.6870000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":663552,"data_stored":287819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79500000000000004,"15min":0.82699999999999996},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2649999999999999,"5min":1.494,"15min":1.494},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.53000000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.747}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.86499999999999999},"min":{"1min":0.498,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.173,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.131},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.44700000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3420000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.88300000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.91900000000000004},"min":{"1min":0.47199999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.6759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.127,"15min":1.216},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3720000000000001,"5min":3.032,"15min":3.032},"last":1.319}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2490000000000001,"15min":1.288},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.5029999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.23,"15min":1.2589999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6000000000000001,"5min":3.069,"15min":3.069},"last":1.429}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.216,"15min":1.242},"min":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5629999999999999,"5min":2.181,"15min":2.181},"last":1.159},{"interface":"front","average":{"1min":1.228,"5min":1.3240000000000001,"15min":1.385},"min":{"1min":0.84299999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6950000000000001,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.4650000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3340000000000001,"15min":1.361},"min":{"1min":0.81299999999999994,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.913,"5min":2.633,"15min":2.633},"last":1.355},{"interface":"front","average":{"1min":1.302,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6719999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.405}]},{"osd":7,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.367,"15min":1.415},"min":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7430000000000001,"5min":2.794,"15min":2.794},"last":1.111},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3120000000000001,"15min":1.335},"min":{"1min":0.95199999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5960000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.2769999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":287853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.131,"5min":1.302,"15min":1.302},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.86199999999999999},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2190000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.498,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4530000000000001,"5min":1.768,"15min":1.768},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0009999999999999},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5960000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.044},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4990000000000001,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.85799999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2609999999999999,"15min":1.2869999999999999},"min":{"1min":0.88,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8580000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.236},{"interface":"front","average":{"1min":1.272,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.873,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0259999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2609999999999999,"15min":1.2929999999999999},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7689999999999999,"5min":3.319,"15min":3.319},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.224,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.82499999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.9930000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.141}]},{"osd":6,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.292,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.931,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.284,"15min":1.2989999999999999},"min":{"1min":0.76200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7070000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.026}]},{"osd":7,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3600000000000001,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.575,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.327},{"interface":"front","average":{"1min":1.329,"5min":1.401,"15min":1.425},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.3460000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.97899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1277952,"data_stored":894447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.46200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.077},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.98799999999999999,"15min":1.0600000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.329,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":1.0249999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3300000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.49099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.278,"5min":2.444,"15min":2.444},"last":0.87}]},{"osd":3,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.84099999999999997,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.2010000000000001,"5min":1.474,"15min":1.474},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.083,"15min":1.1200000000000001},"min":{"1min":0.52600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.514,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.4159999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2969999999999999,"15min":1.3300000000000001},"min":{"1min":0.91300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.55,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.262,"15min":1.2889999999999999},"min":{"1min":0.85899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5089999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.163}]},{"osd":5,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5820000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.337},{"interface":"front","average":{"1min":1.129,"5min":1.266,"15min":1.29},"min":{"1min":0.78600000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5800000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.2350000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.327,"15min":1.3580000000000001},"min":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.605,"5min":2.552,"15min":2.552},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.325,"15min":1.373},"min":{"1min":0.874,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.716,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.341,"15min":1.3879999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6559999999999999,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3560000000000001},"min":{"1min":0.85899999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.722,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.264}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:14:44.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-11T06:14:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.697+0000 7ffb3ddbb640 1 Processor -- start 2023-10-11T06:14:44.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.697+0000 7ffb3ddbb640 1 -- start start 2023-10-11T06:14:44.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.698+0000 7ffb3ddbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 0x7ffb380caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.698+0000 7ffb3ddbb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb381593f0 con 0x7ffb380ce0d0 2023-10-11T06:14:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.698+0000 7ffb3ddbb640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb381596c0 con 0x7ffb380cd300 2023-10-11T06:14:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.698+0000 7ffb3ddbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815a340 con 0x7ffb380adb90 2023-10-11T06:14:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb377fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffb380ce0d0 0x7ffb380c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34916/0 (socket says 172.21.15.84:34916) 2023-10-11T06:14:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb377fe640 1 -- 172.21.15.84:0/4229294822 learned_addr learned my addr 172.21.15.84:0/4229294822 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738162875 0 0) 0x7ffb381593f0 con 0x7ffb380ce0d0 2023-10-11T06:14:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb20003520 con 0x7ffb380ce0d0 2023-10-11T06:14:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb36ffd640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 0x7ffb380caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3706562093 0 0) 0x7ffb20003520 con 0x7ffb380ce0d0 2023-10-11T06:14:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 msgr2=0x7ffb380caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 0x7ffb380caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 >> v1:172.21.15.99:6789/0 conn(0x7ffb380cd300 legacy=0x7ffb3815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb381599a0 con 0x7ffb380ce0d0 2023-10-11T06:14:44.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7ffb280017f0 con 0x7ffb380ce0d0 2023-10-11T06:14:44.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 >> v1:172.21.15.84:6789/0 conn(0x7ffb380ce0d0 legacy=0x7ffb380c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 0x7ffb200039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb20004d00 0x7ffb20005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.699+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815a340 con 0x7ffb20003fb0 2023-10-11T06:14:44.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2000bb10 con 0x7ffb20004d00 2023-10-11T06:14:44.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2000bcd0 con 0x7ffb380adb90 2023-10-11T06:14:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb36ffd640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb377fe640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 0x7ffb200039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb36ffd640 1 -- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 msgr2=0x7ffb200039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb36ffd640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 0x7ffb200039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb36ffd640 1 -- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb20004d00 msgr2=0x7ffb20005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb37fff640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb20004d00 0x7ffb20005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb36ffd640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb20004d00 0x7ffb20005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb36ffd640 1 -- 172.21.15.84:0/4229294822 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20003520 con 0x7ffb20003fb0 2023-10-11T06:14:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb377fe640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb380adb90 0x7ffb200039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.700+0000 7ffb37fff640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb20004d00 0x7ffb20005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.701+0000 7ffb36ffd640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb200047c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c007360 tx=0x7ffb200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef53a0fef992ce56 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:44.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c02ccb0 con 0x7ffb20003fb0 2023-10-11T06:14:44.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c02ce10 con 0x7ffb20003fb0 2023-10-11T06:14:44.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb367fc640 1 -- 172.21.15.84:0/4229294822 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c034660 con 0x7ffb20003fb0 2023-10-11T06:14:44.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/4229294822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 msgr2=0x7ffb200047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb3ddbb640 1 --2- 172.21.15.84:0/4229294822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb200047c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c007360 tx=0x7ffb200057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb37fff640 1 -- 172.21.15.84:0/4229294822 reap_dead start 2023-10-11T06:14:44.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/4229294822 shutdown_connections 2023-10-11T06:14:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/4229294822 >> 172.21.15.84:0/4229294822 conn(0x7ffb38069c70 msgr2=0x7ffb3806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/4229294822 shutdown_connections 2023-10-11T06:14:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.702+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/4229294822 wait complete. 2023-10-11T06:14:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.704+0000 7ffb3ddbb640 1 Processor -- start 2023-10-11T06:14:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.704+0000 7ffb3ddbb640 1 -- start start 2023-10-11T06:14:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.705+0000 7ffb3ddbb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb380c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.705+0000 7ffb3ddbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb381537a0 0x7ffb380c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.705+0000 7ffb3ddbb640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb38152fc0 0x7ffb380b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.705+0000 7ffb3ddbb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c67d0 con 0x7ffb20003fb0 2023-10-11T06:14:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.705+0000 7ffb3ddbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c7a60 con 0x7ffb381537a0 2023-10-11T06:14:44.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.705+0000 7ffb3ddbb640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380c7d40 con 0x7ffb38152fc0 2023-10-11T06:14:44.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb380c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb380c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37112/0 (socket says 172.21.15.84:37112) 2023-10-11T06:14:44.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 -- 172.21.15.84:0/694576013 learned_addr learned my addr 172.21.15.84:0/694576013 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 -- 172.21.15.84:0/694576013 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb38152fc0 msgr2=0x7ffb380b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb38152fc0 0x7ffb380b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 -- 172.21.15.84:0/694576013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb381537a0 msgr2=0x7ffb380c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb381537a0 0x7ffb380c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 -- 172.21.15.84:0/694576013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380c6180 con 0x7ffb20003fb0 2023-10-11T06:14:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.706+0000 7ffb377fe640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb380c6c60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800d800 tx=0x7ffb2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.707+0000 7ffb1ffff640 1 -- 172.21.15.84:0/694576013 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28007480 con 0x7ffb20003fb0 2023-10-11T06:14:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.707+0000 7ffb1ffff640 1 -- 172.21.15.84:0/694576013 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb280075e0 con 0x7ffb20003fb0 2023-10-11T06:14:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.707+0000 7ffb1ffff640 1 -- 172.21.15.84:0/694576013 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280059a0 con 0x7ffb20003fb0 2023-10-11T06:14:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.707+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb380c6460 con 0x7ffb20003fb0 2023-10-11T06:14:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.707+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb380b3270 con 0x7ffb20003fb0 2023-10-11T06:14:44.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.708+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb380c7f00 con 0x7ffb20003fb0 2023-10-11T06:14:44.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.709+0000 7ffb1ffff640 1 -- 172.21.15.84:0/694576013 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ffb28005b00 con 0x7ffb20003fb0 2023-10-11T06:14:44.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.710+0000 7ffb1ffff640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffb2406a830 0x7ffb2406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:44.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.710+0000 7ffb1ffff640 1 -- 172.21.15.84:0/694576013 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffb28088ef0 con 0x7ffb20003fb0 2023-10-11T06:14:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.710+0000 7ffb36ffd640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffb2406a830 0x7ffb2406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.711+0000 7ffb36ffd640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffb2406a830 0x7ffb2406cce0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c036000 tx=0x7ffb2c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.715+0000 7ffb1ffff640 1 -- 172.21.15.84:0/694576013 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ffb28054500 con 0x7ffb20003fb0 2023-10-11T06:14:44.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.905+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb38073ed0 con 0x7ffb20003fb0 2023-10-11T06:14:44.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.907+0000 7ffb1ffff640 1 -- 172.21.15.84:0/694576013 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+14252 (secure 0 0 0) 0x7ffb28005db0 con 0x7ffb20003fb0 2023-10-11T06:14:44.910 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:44.911 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":57,"fsid":"c84dc637-9184-44da-bf67-6cb4224adc5a","created":"2023-10-11T06:10:45.889257+0000","modified":"2023-10-11T06:14:41.002097+0000","last_up_change":"2023-10-11T06:10:59.440438+0000","last_in_change":"2023-10-11T06:10:50.161835+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-11T06:11:00.844920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-11T06:11:06.509213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f04c5f33-5c2b-4fe0-ac57-f6deda5f70df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":87856},{"type":"v1","addr":"172.21.15.84:6803","nonce":87856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":87856},{"type":"v1","addr":"172.21.15.84:6805","nonce":87856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":87856},{"type":"v1","addr":"172.21.15.84:6809","nonce":87856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":87856},{"type":"v1","addr":"172.21.15.84:6807","nonce":87856}]},"public_addr":"172.21.15.84:6803/87856","cluster_addr":"172.21.15.84:6805/87856","heartbeat_back_addr":"172.21.15.84:6809/87856","heartbeat_front_addr":"172.21.15.84:6807/87856","state":["exists","up"]},{"osd":1,"uuid":"713cb503-ae2a-45af-b3bd-5133ee34f7cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":87859},{"type":"v1","addr":"172.21.15.84:6811","nonce":87859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":87859},{"type":"v1","addr":"172.21.15.84:6813","nonce":87859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":87859},{"type":"v1","addr":"172.21.15.84:6821","nonce":87859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":87859},{"type":"v1","addr":"172.21.15.84:6815","nonce":87859}]},"public_addr":"172.21.15.84:6811/87859","cluster_addr":"172.21.15.84:6813/87859","heartbeat_back_addr":"172.21.15.84:6821/87859","heartbeat_front_addr":"172.21.15.84:6815/87859","state":["exists","up"]},{"osd":2,"uuid":"faeec611-1e2f-466e-addc-fc7e02102e37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":87861},{"type":"v1","addr":"172.21.15.84:6819","nonce":87861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":87861},{"type":"v1","addr":"172.21.15.84:6824","nonce":87861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":87861},{"type":"v1","addr":"172.21.15.84:6832","nonce":87861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":87861},{"type":"v1","addr":"172.21.15.84:6828","nonce":87861}]},"public_addr":"172.21.15.84:6819/87861","cluster_addr":"172.21.15.84:6824/87861","heartbeat_back_addr":"172.21.15.84:6832/87861","heartbeat_front_addr":"172.21.15.84:6828/87861","state":["exists","up"]},{"osd":3,"uuid":"7485b46e-6701-4a5d-825f-66b3a7841ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6817","nonce":87864},{"type":"v1","addr":"172.21.15.84:6820","nonce":87864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6823","nonce":87864},{"type":"v1","addr":"172.21.15.84:6825","nonce":87864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6831","nonce":87864},{"type":"v1","addr":"172.21.15.84:6833","nonce":87864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6827","nonce":87864},{"type":"v1","addr":"172.21.15.84:6829","nonce":87864}]},"public_addr":"172.21.15.84:6820/87864","cluster_addr":"172.21.15.84:6825/87864","heartbeat_back_addr":"172.21.15.84:6833/87864","heartbeat_front_addr":"172.21.15.84:6829/87864","state":["exists","up"]},{"osd":4,"uuid":"2e7af1a8-0fa7-4e5c-942e-e7ed7a589bf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":88014},{"type":"v1","addr":"172.21.15.99:6801","nonce":88014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":88014},{"type":"v1","addr":"172.21.15.99:6803","nonce":88014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":88014},{"type":"v1","addr":"172.21.15.99:6807","nonce":88014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":88014},{"type":"v1","addr":"172.21.15.99:6805","nonce":88014}]},"public_addr":"172.21.15.99:6801/88014","cluster_addr":"172.21.15.99:6803/88014","heartbeat_back_addr":"172.21.15.99:6807/88014","heartbeat_front_addr":"172.21.15.99:6805/88014","state":["exists","up"]},{"osd":5,"uuid":"62ae3f3f-d323-46cd-8f29-289419f9f189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":88015},{"type":"v1","addr":"172.21.15.99:6809","nonce":88015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":88015},{"type":"v1","addr":"172.21.15.99:6811","nonce":88015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":88015},{"type":"v1","addr":"172.21.15.99:6815","nonce":88015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":88015},{"type":"v1","addr":"172.21.15.99:6813","nonce":88015}]},"public_addr":"172.21.15.99:6809/88015","cluster_addr":"172.21.15.99:6811/88015","heartbeat_back_addr":"172.21.15.99:6815/88015","heartbeat_front_addr":"172.21.15.99:6813/88015","state":["exists","up"]},{"osd":6,"uuid":"73564779-506c-4fd7-9fa7-d75e37a4be36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":88045},{"type":"v1","addr":"172.21.15.99:6822","nonce":88045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":88045},{"type":"v1","addr":"172.21.15.99:6826","nonce":88045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":88045},{"type":"v1","addr":"172.21.15.99:6831","nonce":88045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":88045},{"type":"v1","addr":"172.21.15.99:6829","nonce":88045}]},"public_addr":"172.21.15.99:6822/88045","cluster_addr":"172.21.15.99:6826/88045","heartbeat_back_addr":"172.21.15.99:6831/88045","heartbeat_front_addr":"172.21.15.99:6829/88045","state":["exists","up"]},{"osd":7,"uuid":"22b1d502-ce7e-439a-8c3e-a7c63e4e3e1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":88032},{"type":"v1","addr":"172.21.15.99:6817","nonce":88032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":88032},{"type":"v1","addr":"172.21.15.99:6819","nonce":88032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":88032},{"type":"v1","addr":"172.21.15.99:6827","nonce":88032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":88032},{"type":"v1","addr":"172.21.15.99:6823","nonce":88032}]},"public_addr":"172.21.15.99:6817/88032","cluster_addr":"172.21.15.99:6819/88032","heartbeat_back_addr":"172.21.15.99:6827/88032","heartbeat_front_addr":"172.21.15.99:6823/88032","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.688099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.488618+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.503978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.522338+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.755494+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.520156+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.544254+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-11T06:10:55.528204+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/2048038445":"2023-10-12T06:14:37.727714+0000","172.21.15.84:6835/2048038445":"2023-10-12T06:14:37.727714+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-11T06:14:44.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffb2406a830 msgr2=0x7ffb2406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffb2406a830 0x7ffb2406cce0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c036000 tx=0x7ffb2c02e000 comp rx=0 tx=0).stop 2023-10-11T06:14:44.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 msgr2=0x7ffb380c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:44.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb380c6c60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800d800 tx=0x7ffb2800dcc0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 shutdown_connections 2023-10-11T06:14:44.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ffb2406a830 0x7ffb2406cce0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb38152fc0 0x7ffb380b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb381537a0 0x7ffb380c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 --2- 172.21.15.84:0/694576013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb20003fb0 0x7ffb380c6c60 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:44.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.910+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 >> 172.21.15.84:0/694576013 conn(0x7ffb38069c70 msgr2=0x7ffb3815ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:44.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.911+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 shutdown_connections 2023-10-11T06:14:44.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:44.911+0000 7ffb3ddbb640 1 -- 172.21.15.84:0/694576013 wait complete. 2023-10-11T06:14:45.934 INFO:tasks.ceph:Scrubbing osd.0 2023-10-11T06:14:45.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.162+0000 7f86c9f33640 1 Processor -- start 2023-10-11T06:14:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.162+0000 7f86c9f33640 1 -- start start 2023-10-11T06:14:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.163+0000 7f86c9f33640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c406b700 0x7f86c406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.163+0000 7f86c9f33640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86c40d2f20 0x7f86c40cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.163+0000 7f86c9f33640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c40d0ed0 con 0x7f86c40d0450 2023-10-11T06:14:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.163+0000 7f86c9f33640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c415c6f0 con 0x7f86c40d2f20 2023-10-11T06:14:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.163+0000 7f86c9f33640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40d0bf0 con 0x7f86c406b700 2023-10-11T06:14:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.164+0000 7f86c8f31640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c406b700 0x7f86c406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.164+0000 7f86c9732640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86c40d2f20 0x7f86c40cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.164+0000 7f86c8f31640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c406b700 0x7f86c406bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:36640/0 (socket says 172.21.15.84:36640) 2023-10-11T06:14:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.164+0000 7f86c8f31640 1 -- 172.21.15.84:0/1820176424 learned_addr learned my addr 172.21.15.84:0/1820176424 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.165+0000 7f86c8f31640 1 -- 172.21.15.84:0/1820176424 >> v2:172.21.15.99:3300/0 conn(0x7f86c40d2f20 msgr2=0x7f86c40cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.165+0000 7f86c8f31640 1 --2- 172.21.15.84:0/1820176424 >> v2:172.21.15.99:3300/0 conn(0x7f86c40d2f20 0x7f86c40cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.165+0000 7f86c8f31640 1 -- 172.21.15.84:0/1820176424 >> v1:172.21.15.99:6789/0 conn(0x7f86c40d0450 legacy=0x7f86c40d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.165+0000 7f86c8f31640 1 -- 172.21.15.84:0/1820176424 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40ccf70 con 0x7f86c406b700 2023-10-11T06:14:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.167+0000 7f86c8f31640 1 --2- 172.21.15.84:0/1820176424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c406b700 0x7f86c406bad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009e50 tx=0x7f86b402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5aae67a57715c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.167+0000 7f86c37fe640 1 -- 172.21.15.84:0/1820176424 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b402cb00 con 0x7f86c406b700 2023-10-11T06:14:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.167+0000 7f86c37fe640 1 -- 172.21.15.84:0/1820176424 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b402cc60 con 0x7f86c406b700 2023-10-11T06:14:46.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.167+0000 7f86c37fe640 1 -- 172.21.15.84:0/1820176424 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40343f0 con 0x7f86c406b700 2023-10-11T06:14:46.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 -- 172.21.15.84:0/1820176424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c406b700 msgr2=0x7f86c406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 --2- 172.21.15.84:0/1820176424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c406b700 0x7f86c406bad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009e50 tx=0x7f86b402bf60 comp rx=0 tx=0).stop 2023-10-11T06:14:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 -- 172.21.15.84:0/1820176424 shutdown_connections 2023-10-11T06:14:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 --2- 172.21.15.84:0/1820176424 >> v2:172.21.15.99:3300/0 conn(0x7f86c40d2f20 0x7f86c40cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 --2- 172.21.15.84:0/1820176424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c406b700 0x7f86c406bad0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 -- 172.21.15.84:0/1820176424 >> 172.21.15.84:0/1820176424 conn(0x7f86c4071cc0 msgr2=0x7f86c409bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 -- 172.21.15.84:0/1820176424 shutdown_connections 2023-10-11T06:14:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.168+0000 7f86c9f33640 1 -- 172.21.15.84:0/1820176424 wait complete. 2023-10-11T06:14:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.170+0000 7f86c9f33640 1 Processor -- start 2023-10-11T06:14:46.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.170+0000 7f86c9f33640 1 -- start start 2023-10-11T06:14:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c9f33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c406b700 0x7f86c4155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c9f33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86c40d0450 0x7f86c40c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c9f33640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c40d2f20 0x7f86c40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c9f33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c41f1720 con 0x7f86c40d0450 2023-10-11T06:14:46.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c9f33640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c415d870 con 0x7f86c406b700 2023-10-11T06:14:46.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c9f33640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c415daf0 con 0x7f86c40d2f20 2023-10-11T06:14:46.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86c40d0450 0x7f86c40c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86c40d0450 0x7f86c40c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37128/0 (socket says 172.21.15.84:37128) 2023-10-11T06:14:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c3fff640 1 -- 172.21.15.84:0/89136570 learned_addr learned my addr 172.21.15.84:0/89136570 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.171+0000 7f86c8f31640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c406b700 0x7f86c4155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c9732640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c40d2f20 0x7f86c40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c3fff640 1 -- 172.21.15.84:0/89136570 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c40d2f20 msgr2=0x7f86c40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c3fff640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86c40d2f20 0x7f86c40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c3fff640 1 -- 172.21.15.84:0/89136570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c406b700 msgr2=0x7f86c4155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c3fff640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86c406b700 0x7f86c4155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c3fff640 1 -- 172.21.15.84:0/89136570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4009b30 con 0x7f86c40d0450 2023-10-11T06:14:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c3fff640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86c40d0450 0x7f86c40c8ae0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c3c0 tx=0x7f86ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.172+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0052d0 con 0x7f86c40d0450 2023-10-11T06:14:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.173+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac005450 con 0x7f86c40d0450 2023-10-11T06:14:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.173+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c415ddd0 con 0x7f86c40d0450 2023-10-11T06:14:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.173+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac007260 con 0x7f86c40d0450 2023-10-11T06:14:46.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.173+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c415e390 con 0x7f86c40d0450 2023-10-11T06:14:46.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.174+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86c40cd530 con 0x7f86c40d0450 2023-10-11T06:14:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.175+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f86ac0073c0 con 0x7f86c40d0450 2023-10-11T06:14:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.175+0000 7f86c17fa640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f86a406a830 0x7f86a406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.176+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f86ac088040 con 0x7f86c40d0450 2023-10-11T06:14:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.176+0000 7f86c8f31640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f86a406a830 0x7f86a406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.176+0000 7f86c17fa640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f86a4070ca0 0x7f86a40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.176+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 --> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86a4073760 con 0x7f86a4070ca0 2023-10-11T06:14:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.176+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86ac08a5c0 con 0x7f86c40d0450 2023-10-11T06:14:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.176+0000 7f86c9732640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f86a4070ca0 0x7f86a40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.176+0000 7f86c8f31640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f86a406a830 0x7f86a406cce0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86b4035000 tx=0x7f86b4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.177+0000 7f86c9732640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f86a4070ca0 0x7f86a40730b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.177+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== osd.0 v2:172.21.15.84:6802/87856 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f86a4073760 con 0x7f86a4070ca0 2023-10-11T06:14:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.191+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 --> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f86c40cd250 con 0x7f86a4070ca0 2023-10-11T06:14:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.191+0000 7f86c17fa640 1 -- 172.21.15.84:0/89136570 <== osd.0 v2:172.21.15.84:6802/87856 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f86bc009800 con 0x7f86a4070ca0 2023-10-11T06:14:46.194 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:46.194 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:46.195 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:46.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f86a4070ca0 msgr2=0x7f86a40730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6802/87856,v1:172.21.15.84:6803/87856] conn(0x7f86a4070ca0 0x7f86a40730b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f86a406a830 msgr2=0x7f86a406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f86a406a830 0x7f86a406cce0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86b4035000 tx=0x7f86b4030000 comp rx=0 tx=0).stop 2023-10-11T06:14:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86c40d0450 msgr2=0x7f86c40c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 --2- 172.21.15.84:0/89136570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86c40d0450 0x7f86c40c8ae0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c3c0 tx=0x7f86ac00c880 comp rx=0 tx=0).stop 2023-10-11T06:14:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9732640 1 -- 172.21.15.84:0/89136570 reap_dead start 2023-10-11T06:14:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 shutdown_connections 2023-10-11T06:14:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 >> 172.21.15.84:0/89136570 conn(0x7f86c4071cc0 msgr2=0x7f86c406ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 shutdown_connections 2023-10-11T06:14:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.192+0000 7f86c9f33640 1 -- 172.21.15.84:0/89136570 wait complete. 2023-10-11T06:14:46.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-11T06:14:46.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.423+0000 7fba07fff640 1 Processor -- start 2023-10-11T06:14:46.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.423+0000 7fba07fff640 1 -- start start 2023-10-11T06:14:46.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.425+0000 7fba07fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba080b3e70 0x7fba080ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.425+0000 7fba07fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fba080ca750 0x7fba080ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.425+0000 7fba07fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0815da50 con 0x7fba0806fd80 2023-10-11T06:14:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.425+0000 7fba07fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0815b480 con 0x7fba080ca750 2023-10-11T06:14:46.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.425+0000 7fba07fff640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba080cad10 con 0x7fba080b3e70 2023-10-11T06:14:46.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.425+0000 7fb9fffff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fba0806fd80 0x7fba0806bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34930/0 (socket says 172.21.15.84:34930) 2023-10-11T06:14:46.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.426+0000 7fb9fffff640 1 -- 172.21.15.84:0/2418281783 learned_addr learned my addr 172.21.15.84:0/2418281783 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:46.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.425+0000 7fba077fe640 1 --2- 172.21.15.84:0/2418281783 >> v2:172.21.15.84:3300/0 conn(0x7fba080ca750 0x7fba080ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.426+0000 7fba06ffd640 1 --2- 172.21.15.84:0/2418281783 >> v2:172.21.15.99:3300/0 conn(0x7fba080b3e70 0x7fba080ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.426+0000 7fba077fe640 1 -- 172.21.15.84:0/2418281783 >> v2:172.21.15.99:3300/0 conn(0x7fba080b3e70 msgr2=0x7fba080ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:46.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.426+0000 7fba077fe640 1 --2- 172.21.15.84:0/2418281783 >> v2:172.21.15.99:3300/0 conn(0x7fba080b3e70 0x7fba080ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.426+0000 7fba077fe640 1 -- 172.21.15.84:0/2418281783 >> v1:172.21.15.84:6789/0 conn(0x7fba0806fd80 legacy=0x7fba0806bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.426+0000 7fba077fe640 1 -- 172.21.15.84:0/2418281783 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0815dca0 con 0x7fba080ca750 2023-10-11T06:14:46.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.427+0000 7fba077fe640 1 --2- 172.21.15.84:0/2418281783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba080ca750 0x7fba080ce910 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fba080cf520 tx=0x7fba0002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fca1a183bd14c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.427+0000 7fba067fc640 1 -- 172.21.15.84:0/2418281783 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0002cc70 con 0x7fba080ca750 2023-10-11T06:14:46.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.427+0000 7fba067fc640 1 -- 172.21.15.84:0/2418281783 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0002cdd0 con 0x7fba080ca750 2023-10-11T06:14:46.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.427+0000 7fba067fc640 1 -- 172.21.15.84:0/2418281783 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00034660 con 0x7fba080ca750 2023-10-11T06:14:46.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 -- 172.21.15.84:0/2418281783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba080ca750 msgr2=0x7fba080ce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 --2- 172.21.15.84:0/2418281783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba080ca750 0x7fba080ce910 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fba080cf520 tx=0x7fba0002bec0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 -- 172.21.15.84:0/2418281783 shutdown_connections 2023-10-11T06:14:46.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 --2- 172.21.15.84:0/2418281783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba080ca750 0x7fba080ce910 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 --2- 172.21.15.84:0/2418281783 >> v2:172.21.15.99:3300/0 conn(0x7fba080b3e70 0x7fba080ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 -- 172.21.15.84:0/2418281783 >> 172.21.15.84:0/2418281783 conn(0x7fba08069c70 msgr2=0x7fba0806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:46.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 -- 172.21.15.84:0/2418281783 shutdown_connections 2023-10-11T06:14:46.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.428+0000 7fba07fff640 1 -- 172.21.15.84:0/2418281783 wait complete. 2023-10-11T06:14:46.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.430+0000 7fba07fff640 1 Processor -- start 2023-10-11T06:14:46.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.430+0000 7fba07fff640 1 -- start start 2023-10-11T06:14:46.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.431+0000 7fba07fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba0806fd80 0x7fba0814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.431+0000 7fba07fff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba080b3e70 0x7fba080ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.431+0000 7fba07fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba080ca750 0x7fba08158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.431+0000 7fba07fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0814be80 con 0x7fba0806fd80 2023-10-11T06:14:46.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.431+0000 7fba07fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080abe20 con 0x7fba080ca750 2023-10-11T06:14:46.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.431+0000 7fba07fff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba081f52f0 con 0x7fba080b3e70 2023-10-11T06:14:46.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba0806fd80 0x7fba0814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba0806fd80 0x7fba0814b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37150/0 (socket says 172.21.15.84:37150) 2023-10-11T06:14:46.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 -- 172.21.15.84:0/1674335327 learned_addr learned my addr 172.21.15.84:0/1674335327 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:46.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba077fe640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba080ca750 0x7fba08158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 -- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba080b3e70 msgr2=0x7fba080ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fb9fffff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba080b3e70 0x7fba080ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba080b3e70 0x7fba080ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 -- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba080ca750 msgr2=0x7fba08158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba080ca750 0x7fba08158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 -- 172.21.15.84:0/1674335327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00009a70 con 0x7fba0806fd80 2023-10-11T06:14:46.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fb9fffff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba080b3e70 0x7fba080ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:46.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.432+0000 7fba06ffd640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba0806fd80 0x7fba0814b940 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800d800 tx=0x7fb9f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.433+0000 7fba04ff9640 1 -- 172.21.15.84:0/1674335327 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800b660 con 0x7fba0806fd80 2023-10-11T06:14:46.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.433+0000 7fba04ff9640 1 -- 172.21.15.84:0/1674335327 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f800b7c0 con 0x7fba0806fd80 2023-10-11T06:14:46.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.433+0000 7fba04ff9640 1 -- 172.21.15.84:0/1674335327 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800bb50 con 0x7fba0806fd80 2023-10-11T06:14:46.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.433+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba081f55d0 con 0x7fba0806fd80 2023-10-11T06:14:46.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.433+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba081f5b90 con 0x7fba0806fd80 2023-10-11T06:14:46.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.434+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0804e890 con 0x7fba0806fd80 2023-10-11T06:14:46.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.435+0000 7fba04ff9640 1 -- 172.21.15.84:0/1674335327 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fb9f800bcb0 con 0x7fba0806fd80 2023-10-11T06:14:46.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.436+0000 7fba04ff9640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb9e806a830 0x7fb9e806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.436+0000 7fba04ff9640 1 -- 172.21.15.84:0/1674335327 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb9f80891d0 con 0x7fba0806fd80 2023-10-11T06:14:46.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.436+0000 7fb9fffff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb9e806a830 0x7fb9e806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.437+0000 7fb9fffff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb9e806a830 0x7fb9e806cce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fba080b2950 tx=0x7fb9f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.441+0000 7fba04ff9640 1 -- 172.21.15.84:0/1674335327 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fb9f8054760 con 0x7fba0806fd80 2023-10-11T06:14:46.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.632+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fba0815e260 con 0x7fb9e806a830 2023-10-11T06:14:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.633+0000 7fba04ff9640 1 -- 172.21.15.84:0/1674335327 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb9f0001d00 con 0x7fb9e806a830 2023-10-11T06:14:46.635 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 0 to deep-scrub 2023-10-11T06:14:46.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb9e806a830 msgr2=0x7fb9e806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb9e806a830 0x7fb9e806cce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fba080b2950 tx=0x7fb9f0009040 comp rx=0 tx=0).stop 2023-10-11T06:14:46.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba0806fd80 msgr2=0x7fba0814b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba0806fd80 0x7fba0814b940 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800d800 tx=0x7fb9f800dcc0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 shutdown_connections 2023-10-11T06:14:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fb9e806a830 0x7fb9e806cce0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba080ca750 0x7fba08158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba080b3e70 0x7fba080ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 --2- 172.21.15.84:0/1674335327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba0806fd80 0x7fba0814b940 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 >> 172.21.15.84:0/1674335327 conn(0x7fba08069c70 msgr2=0x7fba08159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 shutdown_connections 2023-10-11T06:14:46.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.635+0000 7fba07fff640 1 -- 172.21.15.84:0/1674335327 wait complete. 2023-10-11T06:14:46.655 INFO:tasks.ceph:Scrubbing osd.1 2023-10-11T06:14:46.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:46.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.864+0000 7f650b17d640 1 Processor -- start 2023-10-11T06:14:46.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.864+0000 7f650b17d640 1 -- start start 2023-10-11T06:14:46.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.865+0000 7f650b17d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f650414ff30 0x7f65040c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.865+0000 7f650b17d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040c9b90 con 0x7f65040ce510 2023-10-11T06:14:46.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f650b17d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650415da10 con 0x7f6504150300 2023-10-11T06:14:46.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f650b17d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040cc990 con 0x7f650414ff30 2023-10-11T06:14:46.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f6508f19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f650414ff30 0x7f65040c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f6508f19640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f650414ff30 0x7f65040c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37154/0 (socket says 172.21.15.84:37154) 2023-10-11T06:14:46.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f6508f19640 1 -- 172.21.15.84:0/1477998402 learned_addr learned my addr 172.21.15.84:0/1477998402 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:46.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f6508f19640 1 -- 172.21.15.84:0/1477998402 >> v1:172.21.15.99:6789/0 conn(0x7f6504150300 legacy=0x7f65040cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:46.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f6508f19640 1 -- 172.21.15.84:0/1477998402 >> v1:172.21.15.84:6789/0 conn(0x7f65040ce510 legacy=0x7f65040ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.866+0000 7f6508f19640 1 -- 172.21.15.84:0/1477998402 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650415ce00 con 0x7f650414ff30 2023-10-11T06:14:46.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.867+0000 7f6508f19640 1 --2- 172.21.15.84:0/1477998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f650414ff30 0x7f65040c9650 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009870 tx=0x7f64f802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23154fe58c0c3020 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f65037fe640 1 -- 172.21.15.84:0/1477998402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8031860 con 0x7f650414ff30 2023-10-11T06:14:46.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f65037fe640 1 -- 172.21.15.84:0/1477998402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f80319c0 con 0x7f650414ff30 2023-10-11T06:14:46.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f65037fe640 1 -- 172.21.15.84:0/1477998402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8031d50 con 0x7f650414ff30 2023-10-11T06:14:46.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f650b17d640 1 -- 172.21.15.84:0/1477998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f650414ff30 msgr2=0x7f65040c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f650b17d640 1 --2- 172.21.15.84:0/1477998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f650414ff30 0x7f65040c9650 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009870 tx=0x7f64f802b900 comp rx=0 tx=0).stop 2023-10-11T06:14:46.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f650b17d640 1 -- 172.21.15.84:0/1477998402 shutdown_connections 2023-10-11T06:14:46.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f650b17d640 1 --2- 172.21.15.84:0/1477998402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f650414ff30 0x7f65040c9650 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.868+0000 7f650b17d640 1 -- 172.21.15.84:0/1477998402 >> 172.21.15.84:0/1477998402 conn(0x7f6504069c70 msgr2=0x7f650406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:46.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.869+0000 7f650b17d640 1 -- 172.21.15.84:0/1477998402 shutdown_connections 2023-10-11T06:14:46.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.869+0000 7f650b17d640 1 -- 172.21.15.84:0/1477998402 wait complete. 2023-10-11T06:14:46.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.870+0000 7f650b17d640 1 Processor -- start 2023-10-11T06:14:46.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.870+0000 7f650b17d640 1 -- start start 2023-10-11T06:14:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.871+0000 7f650b17d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65040ce510 0x7f6504154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.871+0000 7f650b17d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f650414ff30 0x7f65041f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f650b17d640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6504150300 0x7f65041f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f650b17d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65041f6170 con 0x7f65040ce510 2023-10-11T06:14:46.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f650b17d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65041f5bb0 con 0x7f650414ff30 2023-10-11T06:14:46.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f650b17d640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65041f5e90 con 0x7f6504150300 2023-10-11T06:14:46.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65040ce510 0x7f6504154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65040ce510 0x7f6504154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37158/0 (socket says 172.21.15.84:37158) 2023-10-11T06:14:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 -- 172.21.15.84:0/458106179 learned_addr learned my addr 172.21.15.84:0/458106179 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 -- 172.21.15.84:0/458106179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6504150300 msgr2=0x7f65041f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6504150300 0x7f65041f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 -- 172.21.15.84:0/458106179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f650414ff30 msgr2=0x7f65041f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f650414ff30 0x7f65041f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 -- 172.21.15.84:0/458106179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80094d0 con 0x7f65040ce510 2023-10-11T06:14:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.872+0000 7f6508f19640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65040ce510 0x7f6504154ba0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009870 tx=0x7f64f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.873+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8031810 con 0x7f65040ce510 2023-10-11T06:14:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.873+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f80443f0 con 0x7f65040ce510 2023-10-11T06:14:46.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.873+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80447a0 con 0x7f65040ce510 2023-10-11T06:14:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.873+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65041f1e60 con 0x7f65040ce510 2023-10-11T06:14:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.873+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65041f2ba0 con 0x7f65040ce510 2023-10-11T06:14:46.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.874+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f650415d3c0 con 0x7f65040ce510 2023-10-11T06:14:46.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.875+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f64f804d450 con 0x7f65040ce510 2023-10-11T06:14:46.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.876+0000 7f65017fa640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f64e806a830 0x7f64e806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.876+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f64f80ab480 con 0x7f65040ce510 2023-10-11T06:14:46.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.876+0000 7f6503fff640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f64e806a830 0x7f64e806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.876+0000 7f65017fa640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f64e8070ca0 0x7f64e80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.876+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 --> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64e8073760 con 0x7f64e8070ca0 2023-10-11T06:14:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.876+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f80ada60 con 0x7f65040ce510 2023-10-11T06:14:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.876+0000 7f650971a640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f64e8070ca0 0x7f64e80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:46.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.877+0000 7f6503fff640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f64e806a830 0x7f64e806cce0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6504073060 tx=0x7f64f0006ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.877+0000 7f650971a640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f64e8070ca0 0x7f64e80730b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:46.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.878+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== osd.1 v2:172.21.15.84:6810/87859 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f64e8073760 con 0x7f64e8070ca0 2023-10-11T06:14:46.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.891+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 --> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f650415d0e0 con 0x7f64e8070ca0 2023-10-11T06:14:46.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f65017fa640 1 -- 172.21.15.84:0/458106179 <== osd.1 v2:172.21.15.84:6810/87859 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f64fc001d80 con 0x7f64e8070ca0 2023-10-11T06:14:46.894 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:46.895 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:46.895 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:46.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f64e8070ca0 msgr2=0x7f64e80730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650b17d640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6810/87859,v1:172.21.15.84:6811/87859] conn(0x7f64e8070ca0 0x7f64e80730b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f64e806a830 msgr2=0x7f64e806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650b17d640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f64e806a830 0x7f64e806cce0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6504073060 tx=0x7f64f0006ba0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65040ce510 msgr2=0x7f6504154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:46.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650b17d640 1 --2- 172.21.15.84:0/458106179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65040ce510 0x7f6504154ba0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009870 tx=0x7f64f80046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:46.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650971a640 1 -- 172.21.15.84:0/458106179 reap_dead start 2023-10-11T06:14:46.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.892+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 shutdown_connections 2023-10-11T06:14:46.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.893+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 >> 172.21.15.84:0/458106179 conn(0x7f6504069c70 msgr2=0x7f65040b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:46.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.893+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 shutdown_connections 2023-10-11T06:14:46.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:46.893+0000 7f650b17d640 1 -- 172.21.15.84:0/458106179 wait complete. 2023-10-11T06:14:46.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-11T06:14:47.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.124+0000 7f998b53c640 1 Processor -- start 2023-10-11T06:14:47.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.125+0000 7f998b53c640 1 -- start start 2023-10-11T06:14:47.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.126+0000 7f998b53c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 0x7f99840af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.126+0000 7f998b53c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840d2a30 con 0x7f99840d0c20 2023-10-11T06:14:47.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.126+0000 7f998b53c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998406cd10 con 0x7f99840d25e0 2023-10-11T06:14:47.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.126+0000 7f998b53c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840afb60 con 0x7f998415c290 2023-10-11T06:14:47.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.126+0000 7f9989ad9640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f99840d0c20 0x7f99840b0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34944/0 (socket says 172.21.15.84:34944) 2023-10-11T06:14:47.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.126+0000 7f9989ad9640 1 -- 172.21.15.84:0/1156670469 learned_addr learned my addr 172.21.15.84:0/1156670469 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:47.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f99892d8640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 0x7f99840af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513254109 0 0) 0x7f99840d2a30 con 0x7f99840d0c20 2023-10-11T06:14:47.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9970003520 con 0x7f99840d0c20 2023-10-11T06:14:47.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688434581 0 0) 0x7f9970003520 con 0x7f99840d0c20 2023-10-11T06:14:47.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 msgr2=0x7f99840af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 0x7f99840af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 >> v1:172.21.15.99:6789/0 conn(0x7f99840d25e0 legacy=0x7f99840d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840afe40 con 0x7f99840d0c20 2023-10-11T06:14:47.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f997c0019f0 con 0x7f99840d0c20 2023-10-11T06:14:47.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 >> v1:172.21.15.84:6789/0 conn(0x7f99840d0c20 legacy=0x7f99840b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 0x7f99700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.127+0000 7f997bfff640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9970003f70 0x7f99700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f997bfff640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9970004d00 0x7f9970005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99840afb60 con 0x7f9970003f70 2023-10-11T06:14:47.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997000bb40 con 0x7f9970004d00 2023-10-11T06:14:47.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997000bd00 con 0x7f998415c290 2023-10-11T06:14:47.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f9988ad7640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9970003f70 0x7f99700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f99892d8640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 0x7f99700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f9989ad9640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9970004d00 0x7f9970005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f9988ad7640 1 -- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 msgr2=0x7f99700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f9988ad7640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998415c290 0x7f99700039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f9988ad7640 1 -- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9970004d00 msgr2=0x7f9970005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f9988ad7640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9970004d00 0x7f9970005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.128+0000 7f9988ad7640 1 -- 172.21.15.84:0/1156670469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970003520 con 0x7f9970003f70 2023-10-11T06:14:47.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.130+0000 7f9988ad7640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9970003f70 0x7f99700047c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9974001750 tx=0x7f99700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5c229eee5cc00c server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.130+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740052d0 con 0x7f9970003f70 2023-10-11T06:14:47.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.130+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997402e050 con 0x7f9970003f70 2023-10-11T06:14:47.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.130+0000 7f997bfff640 1 -- 172.21.15.84:0/1156670469 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974003830 con 0x7f9970003f70 2023-10-11T06:14:47.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.131+0000 7f998b53c640 1 -- 172.21.15.84:0/1156670469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9970003f70 msgr2=0x7f99700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.131+0000 7f998b53c640 1 --2- 172.21.15.84:0/1156670469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9970003f70 0x7f99700047c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9974001750 tx=0x7f99700057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.131+0000 7f9989ad9640 1 -- 172.21.15.84:0/1156670469 reap_dead start 2023-10-11T06:14:47.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.131+0000 7f998b53c640 1 -- 172.21.15.84:0/1156670469 shutdown_connections 2023-10-11T06:14:47.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.131+0000 7f998b53c640 1 -- 172.21.15.84:0/1156670469 >> 172.21.15.84:0/1156670469 conn(0x7f9984071cc0 msgr2=0x7f998409bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:47.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.131+0000 7f998b53c640 1 -- 172.21.15.84:0/1156670469 shutdown_connections 2023-10-11T06:14:47.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.131+0000 7f998b53c640 1 -- 172.21.15.84:0/1156670469 wait complete. 2023-10-11T06:14:47.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.133+0000 7f998b53c640 1 Processor -- start 2023-10-11T06:14:47.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.133+0000 7f998b53c640 1 -- start start 2023-10-11T06:14:47.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f998b53c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f998414eea0 0x7f998409c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f998b53c640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998409cb20 0x7f998409e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f998b53c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998409e540 0x7f99841f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f998b53c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998409e9b0 con 0x7f998414eea0 2023-10-11T06:14:47.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f998b53c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998409dbe0 con 0x7f998409e540 2023-10-11T06:14:47.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f998b53c640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998409d060 con 0x7f998409cb20 2023-10-11T06:14:47.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f99892d8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f998414eea0 0x7f998409c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f99892d8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f998414eea0 0x7f998409c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37174/0 (socket says 172.21.15.84:37174) 2023-10-11T06:14:47.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.134+0000 7f99892d8640 1 -- 172.21.15.84:0/290789103 learned_addr learned my addr 172.21.15.84:0/290789103 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:47.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f99892d8640 1 -- 172.21.15.84:0/290789103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998409cb20 msgr2=0x7f998409e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f99892d8640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998409cb20 0x7f998409e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f99892d8640 1 -- 172.21.15.84:0/290789103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998409e540 msgr2=0x7f99841f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f99892d8640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998409e540 0x7f99841f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f99892d8640 1 -- 172.21.15.84:0/290789103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998409d340 con 0x7f998414eea0 2023-10-11T06:14:47.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f99892d8640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f998414eea0 0x7f998409c5e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9980011600 tx=0x7f9980011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f9979ffb640 1 -- 172.21.15.84:0/290789103 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000f920 con 0x7f998414eea0 2023-10-11T06:14:47.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f9979ffb640 1 -- 172.21.15.84:0/290789103 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998000fa80 con 0x7f998414eea0 2023-10-11T06:14:47.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f9979ffb640 1 -- 172.21.15.84:0/290789103 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000fdf0 con 0x7f998414eea0 2023-10-11T06:14:47.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998409d620 con 0x7f998414eea0 2023-10-11T06:14:47.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.135+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998414f270 con 0x7f998414eea0 2023-10-11T06:14:47.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.136+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9984073ed0 con 0x7f998414eea0 2023-10-11T06:14:47.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.137+0000 7f9979ffb640 1 -- 172.21.15.84:0/290789103 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f9980015460 con 0x7f998414eea0 2023-10-11T06:14:47.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.138+0000 7f9979ffb640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f996806a860 0x7f996806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.138+0000 7f9979ffb640 1 -- 172.21.15.84:0/290789103 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9980089270 con 0x7f998414eea0 2023-10-11T06:14:47.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.139+0000 7f9988ad7640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f996806a860 0x7f996806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.139+0000 7f9988ad7640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f996806a860 0x7f996806cd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9974033000 tx=0x7f997402f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.143+0000 7f9979ffb640 1 -- 172.21.15.84:0/290789103 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f9980054770 con 0x7f998414eea0 2023-10-11T06:14:47.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.333+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f99840b0400 con 0x7f996806a860 2023-10-11T06:14:47.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.334+0000 7f9979ffb640 1 -- 172.21.15.84:0/290789103 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9974007a30 con 0x7f996806a860 2023-10-11T06:14:47.336 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 1 to deep-scrub 2023-10-11T06:14:47.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f996806a860 msgr2=0x7f996806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f996806a860 0x7f996806cd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9974033000 tx=0x7f997402f000 comp rx=0 tx=0).stop 2023-10-11T06:14:47.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f998414eea0 msgr2=0x7f998409c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f998414eea0 0x7f998409c5e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9980011600 tx=0x7f9980011ac0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 shutdown_connections 2023-10-11T06:14:47.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f996806a860 0x7f996806cd10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998409e540 0x7f99841f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f998409cb20 0x7f998409e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 --2- 172.21.15.84:0/290789103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f998414eea0 0x7f998409c5e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 >> 172.21.15.84:0/290789103 conn(0x7f9984071cc0 msgr2=0x7f99840b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:47.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 shutdown_connections 2023-10-11T06:14:47.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.336+0000 7f998b53c640 1 -- 172.21.15.84:0/290789103 wait complete. 2023-10-11T06:14:47.356 INFO:tasks.ceph:Scrubbing osd.2 2023-10-11T06:14:47.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:47.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.575+0000 7f20e6c77640 1 Processor -- start 2023-10-11T06:14:47.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.575+0000 7f20e6c77640 1 -- start start 2023-10-11T06:14:47.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.576+0000 7f20e6c77640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20e00d2ac0 0x7f20e00d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.576+0000 7f20e6c77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d0940 0x7f20e00d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.576+0000 7f20e6c77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00cf220 con 0x7f20e015f7e0 2023-10-11T06:14:47.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.576+0000 7f20e6c77640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e00d3480 con 0x7f20e00d2ac0 2023-10-11T06:14:47.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.576+0000 7f20e6c77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00cf500 con 0x7f20e00d0940 2023-10-11T06:14:47.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.577+0000 7f20dffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20e00d2ac0 0x7f20e00d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.577+0000 7f20dffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20e00d2ac0 0x7f20e00d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:60170/0 (socket says 172.21.15.84:60170) 2023-10-11T06:14:47.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.577+0000 7f20dffff640 1 -- 172.21.15.84:0/3983364262 learned_addr learned my addr 172.21.15.84:0/3983364262 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:47.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.577+0000 7f20e5214640 1 --2- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d0940 0x7f20e00d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.578+0000 7f20df7fe640 1 -- 172.21.15.84:0/3983364262 <== mon.1 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486723480 0 0) 0x7f20e00cf220 con 0x7f20e015f7e0 2023-10-11T06:14:47.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.578+0000 7f20df7fe640 1 -- 172.21.15.84:0/3983364262 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20cc003520 con 0x7f20e015f7e0 2023-10-11T06:14:47.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.578+0000 7f20dffff640 1 -- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d0940 msgr2=0x7f20e00d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.578+0000 7f20dffff640 1 --2- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d0940 0x7f20e00d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.578+0000 7f20dffff640 1 -- 172.21.15.84:0/3983364262 >> v1:172.21.15.99:6789/0 conn(0x7f20e015f7e0 legacy=0x7f20e00cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.578+0000 7f20dffff640 1 -- 172.21.15.84:0/3983364262 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00cf7e0 con 0x7f20e00d2ac0 2023-10-11T06:14:47.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.578+0000 7f20e5214640 1 --2- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d0940 0x7f20e00d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:14:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.580+0000 7f20dffff640 1 --2- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d2ac0 0x7f20e00d2f40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009870 tx=0x7f20c802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb9a7e05c92953f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.580+0000 7f20df7fe640 1 -- 172.21.15.84:0/3983364262 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8030840 con 0x7f20e00d2ac0 2023-10-11T06:14:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.580+0000 7f20df7fe640 1 -- 172.21.15.84:0/3983364262 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c80309a0 con 0x7f20e00d2ac0 2023-10-11T06:14:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.580+0000 7f20df7fe640 1 -- 172.21.15.84:0/3983364262 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8030d10 con 0x7f20e00d2ac0 2023-10-11T06:14:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 -- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d2ac0 msgr2=0x7f20e00d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 --2- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d2ac0 0x7f20e00d2f40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009870 tx=0x7f20c802b980 comp rx=0 tx=0).stop 2023-10-11T06:14:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 -- 172.21.15.84:0/3983364262 shutdown_connections 2023-10-11T06:14:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 --2- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d0940 0x7f20e00d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 --2- 172.21.15.84:0/3983364262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d2ac0 0x7f20e00d2f40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 -- 172.21.15.84:0/3983364262 >> 172.21.15.84:0/3983364262 conn(0x7f20e00713a0 msgr2=0x7f20e00717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 -- 172.21.15.84:0/3983364262 shutdown_connections 2023-10-11T06:14:47.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.581+0000 7f20e6c77640 1 -- 172.21.15.84:0/3983364262 wait complete. 2023-10-11T06:14:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.584+0000 7f20e6c77640 1 Processor -- start 2023-10-11T06:14:47.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.584+0000 7f20e6c77640 1 -- start start 2023-10-11T06:14:47.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.585+0000 7f20e6c77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d0940 0x7f20e01f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.585+0000 7f20e6c77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d2ac0 0x7f20e00caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.585+0000 7f20e6c77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e015f7e0 0x7f20e00b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.585+0000 7f20e6c77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00b6b30 con 0x7f20e015f7e0 2023-10-11T06:14:47.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.585+0000 7f20e6c77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00b6850 con 0x7f20e00d0940 2023-10-11T06:14:47.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.585+0000 7f20e6c77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00ca400 con 0x7f20e00d2ac0 2023-10-11T06:14:47.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e015f7e0 0x7f20e00b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e015f7e0 0x7f20e00b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37190/0 (socket says 172.21.15.84:37190) 2023-10-11T06:14:47.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 -- 172.21.15.84:0/3406487742 learned_addr learned my addr 172.21.15.84:0/3406487742 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:47.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20dffff640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d2ac0 0x7f20e00caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e4a13640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d0940 0x7f20e01f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 -- 172.21.15.84:0/3406487742 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d2ac0 msgr2=0x7f20e00caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20e00d2ac0 0x7f20e00caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 -- 172.21.15.84:0/3406487742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d0940 msgr2=0x7f20e01f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20e00d0940 0x7f20e01f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 -- 172.21.15.84:0/3406487742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009400 con 0x7f20e015f7e0 2023-10-11T06:14:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.586+0000 7f20e5214640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e015f7e0 0x7f20e00b34e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f20d800e930 tx=0x7f20d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.587+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8009700 con 0x7f20e015f7e0 2023-10-11T06:14:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.587+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d800ca00 con 0x7f20e015f7e0 2023-10-11T06:14:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.587+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c80094d0 con 0x7f20e015f7e0 2023-10-11T06:14:47.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.587+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d800cd90 con 0x7f20e015f7e0 2023-10-11T06:14:47.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.587+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e00ca9c0 con 0x7f20e015f7e0 2023-10-11T06:14:47.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.588+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20e00cfda0 con 0x7f20e015f7e0 2023-10-11T06:14:47.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.589+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f20d8013470 con 0x7f20e015f7e0 2023-10-11T06:14:47.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.590+0000 7f20dd7fa640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f20c006a830 0x7f20c006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.590+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f20d80889c0 con 0x7f20e015f7e0 2023-10-11T06:14:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.590+0000 7f20dd7fa640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7f20c0070ca0 0x7f20c00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.590+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 --> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20c0073760 con 0x7f20c0070ca0 2023-10-11T06:14:47.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.590+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20d808d2a0 con 0x7f20e015f7e0 2023-10-11T06:14:47.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.590+0000 7f20e4a13640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f20c006a830 0x7f20c006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.591+0000 7f20dffff640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7f20c0070ca0 0x7f20c00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.591+0000 7f20e4a13640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f20c006a830 0x7f20c006cce0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001cb0 tx=0x7f20d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.591+0000 7f20dffff640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7f20c0070ca0 0x7f20c00730b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.592+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== osd.2 v2:172.21.15.84:6816/87861 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f20c0073760 con 0x7f20c0070ca0 2023-10-11T06:14:47.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.606+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 --> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f20e00cfac0 con 0x7f20c0070ca0 2023-10-11T06:14:47.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.606+0000 7f20dd7fa640 1 -- 172.21.15.84:0/3406487742 <== osd.2 v2:172.21.15.84:6816/87861 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f20c8030b90 con 0x7f20c0070ca0 2023-10-11T06:14:47.609 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:47.609 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:47.609 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:47.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7f20c0070ca0 msgr2=0x7f20c00730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6816/87861,v1:172.21.15.84:6819/87861] conn(0x7f20c0070ca0 0x7f20c00730b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f20c006a830 msgr2=0x7f20c006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f20c006a830 0x7f20c006cce0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001cb0 tx=0x7f20d0009040 comp rx=0 tx=0).stop 2023-10-11T06:14:47.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e015f7e0 msgr2=0x7f20e00b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 --2- 172.21.15.84:0/3406487742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20e015f7e0 0x7f20e00b34e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f20d800e930 tx=0x7f20d800edf0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e5214640 1 -- 172.21.15.84:0/3406487742 reap_dead start 2023-10-11T06:14:47.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 shutdown_connections 2023-10-11T06:14:47.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 >> 172.21.15.84:0/3406487742 conn(0x7f20e00713a0 msgr2=0x7f20e00cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:47.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 shutdown_connections 2023-10-11T06:14:47.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.607+0000 7f20e6c77640 1 -- 172.21.15.84:0/3406487742 wait complete. 2023-10-11T06:14:47.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-11T06:14:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.842+0000 7fafcaf1b640 1 Processor -- start 2023-10-11T06:14:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.842+0000 7fafcaf1b640 1 -- start start 2023-10-11T06:14:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafcaf1b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafc415c290 0x7fafc40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafcaf1b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc40cb6c0 0x7fafc40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafcaf1b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc415fe40 con 0x7fafc40c9190 2023-10-11T06:14:47.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafcaf1b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc40cad70 con 0x7fafc415c290 2023-10-11T06:14:47.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafcaf1b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc406ff30 con 0x7fafc40cb6c0 2023-10-11T06:14:47.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafc3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fafc40c9190 0x7fafc40cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34950/0 (socket says 172.21.15.84:34950) 2023-10-11T06:14:47.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafc3fff640 1 -- 172.21.15.84:0/3527613209 learned_addr learned my addr 172.21.15.84:0/3527613209 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafc8cb7640 1 --2- 172.21.15.84:0/3527613209 >> v2:172.21.15.99:3300/0 conn(0x7fafc415c290 0x7fafc40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafc94b8640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc40cb6c0 0x7fafc40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.844+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938800544 0 0) 0x7fafc415fe40 con 0x7fafc40c9190 2023-10-11T06:14:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb4003520 con 0x7fafc40c9190 2023-10-11T06:14:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 898788602 0 0) 0x7fafb4003520 con 0x7fafc40c9190 2023-10-11T06:14:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc40cb6c0 msgr2=0x7fafc40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc40cb6c0 0x7fafc40cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 >> v2:172.21.15.99:3300/0 conn(0x7fafc415c290 msgr2=0x7fafc40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 --2- 172.21.15.84:0/3527613209 >> v2:172.21.15.99:3300/0 conn(0x7fafc415c290 0x7fafc40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc41600e0 con 0x7fafc40c9190 2023-10-11T06:14:47.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7fafb0001670 con 0x7fafc40c9190 2023-10-11T06:14:47.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 >> v1:172.21.15.84:6789/0 conn(0x7fafc40c9190 legacy=0x7fafc40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb4003970 0x7fafb4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb40043a0 0x7fafb4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc40cb6c0 0x7fafb4004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40cad70 con 0x7fafb40043a0 2023-10-11T06:14:47.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc406ff30 con 0x7fafb4003970 2023-10-11T06:14:47.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb400b980 con 0x7fafc40cb6c0 2023-10-11T06:14:47.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.845+0000 7fafc3fff640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb40043a0 0x7fafb4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.846+0000 7fafc3fff640 1 -- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc40cb6c0 msgr2=0x7fafb4004f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.846+0000 7fafc3fff640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafc40cb6c0 0x7fafb4004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.846+0000 7fafc3fff640 1 -- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb4003970 msgr2=0x7fafb4003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:47.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.846+0000 7fafc3fff640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb4003970 0x7fafb4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.846+0000 7fafc3fff640 1 -- 172.21.15.84:0/3527613209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4003520 con 0x7fafb40043a0 2023-10-11T06:14:47.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.847+0000 7fafc3fff640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb40043a0 0x7fafb4004860 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c210 tx=0x7fafb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1b05ae07b95712 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.847+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0005ad0 con 0x7fafb40043a0 2023-10-11T06:14:47.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.847+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0005c30 con 0x7fafb40043a0 2023-10-11T06:14:47.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.847+0000 7fafc37fe640 1 -- 172.21.15.84:0/3527613209 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb002c8d0 con 0x7fafb40043a0 2023-10-11T06:14:47.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.848+0000 7fafcaf1b640 1 -- 172.21.15.84:0/3527613209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb40043a0 msgr2=0x7fafb4004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.848+0000 7fafcaf1b640 1 --2- 172.21.15.84:0/3527613209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb40043a0 0x7fafb4004860 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c210 tx=0x7fafb40057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.848+0000 7fafc94b8640 1 -- 172.21.15.84:0/3527613209 reap_dead start 2023-10-11T06:14:47.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.848+0000 7fafcaf1b640 1 -- 172.21.15.84:0/3527613209 shutdown_connections 2023-10-11T06:14:47.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.848+0000 7fafcaf1b640 1 -- 172.21.15.84:0/3527613209 >> 172.21.15.84:0/3527613209 conn(0x7fafc4069c70 msgr2=0x7fafc406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:47.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.848+0000 7fafcaf1b640 1 -- 172.21.15.84:0/3527613209 shutdown_connections 2023-10-11T06:14:47.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.848+0000 7fafcaf1b640 1 -- 172.21.15.84:0/3527613209 wait complete. 2023-10-11T06:14:47.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.850+0000 7fafcaf1b640 1 Processor -- start 2023-10-11T06:14:47.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.850+0000 7fafcaf1b640 1 -- start start 2023-10-11T06:14:47.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafcaf1b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafb40043a0 0x7fafc40b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafcaf1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc4159130 0x7fafc41f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafcaf1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc41f5000 0x7fafc41f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafcaf1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc415ca80 con 0x7fafc4159130 2023-10-11T06:14:47.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafcaf1b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc41f59b0 con 0x7fafc41f5000 2023-10-11T06:14:47.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafcaf1b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40ce790 con 0x7fafb40043a0 2023-10-11T06:14:47.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc4159130 0x7fafc41f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc4159130 0x7fafc41f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37206/0 (socket says 172.21.15.84:37206) 2023-10-11T06:14:47.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 -- 172.21.15.84:0/593167061 learned_addr learned my addr 172.21.15.84:0/593167061 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:47.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc94b8640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc41f5000 0x7fafc41f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc8cb7640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafb40043a0 0x7fafc40b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 -- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafb40043a0 msgr2=0x7fafc40b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafb40043a0 0x7fafc40b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 -- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc41f5000 msgr2=0x7fafc41f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:47.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc41f5000 0x7fafc41f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:47.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.851+0000 7fafc3fff640 1 -- 172.21.15.84:0/593167061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40cc440 con 0x7fafc4159130 2023-10-11T06:14:47.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.852+0000 7fafc3fff640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc4159130 0x7fafc41f1260 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c330 tx=0x7fafb0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.852+0000 7fafc17fa640 1 -- 172.21.15.84:0/593167061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0039070 con 0x7fafc4159130 2023-10-11T06:14:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.852+0000 7fafc17fa640 1 -- 172.21.15.84:0/593167061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb002cce0 con 0x7fafc4159130 2023-10-11T06:14:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.852+0000 7fafc17fa640 1 -- 172.21.15.84:0/593167061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb003c040 con 0x7fafc4159130 2023-10-11T06:14:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.852+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc40cc720 con 0x7fafc4159130 2023-10-11T06:14:47.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.852+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc40ccce0 con 0x7fafc4159130 2023-10-11T06:14:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.853+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc4073ed0 con 0x7fafc4159130 2023-10-11T06:14:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.854+0000 7fafc17fa640 1 -- 172.21.15.84:0/593167061 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fafb0032070 con 0x7fafc4159130 2023-10-11T06:14:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.855+0000 7fafc17fa640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafa806a830 0x7fafa806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:47.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.855+0000 7fafc17fa640 1 -- 172.21.15.84:0/593167061 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fafb00aa110 con 0x7fafc4159130 2023-10-11T06:14:47.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.855+0000 7fafc8cb7640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafa806a830 0x7fafa806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:47.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.856+0000 7fafc8cb7640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafa806a830 0x7fafa806cce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009960 tx=0x7fafb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:47.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:47.860+0000 7fafc17fa640 1 -- 172.21.15.84:0/593167061 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fafb0075840 con 0x7fafc4159130 2023-10-11T06:14:48.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.049+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fafc41606a0 con 0x7fafa806a830 2023-10-11T06:14:48.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.049+0000 7fafc17fa640 1 -- 172.21.15.84:0/593167061 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fafb80095b0 con 0x7fafa806a830 2023-10-11T06:14:48.052 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 2 to deep-scrub 2023-10-11T06:14:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafa806a830 msgr2=0x7fafa806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafa806a830 0x7fafa806cce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009960 tx=0x7fafb8008040 comp rx=0 tx=0).stop 2023-10-11T06:14:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc4159130 msgr2=0x7fafc41f1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc4159130 0x7fafc41f1260 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c330 tx=0x7fafb0001a00 comp rx=0 tx=0).stop 2023-10-11T06:14:48.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 shutdown_connections 2023-10-11T06:14:48.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fafa806a830 0x7fafa806cce0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc41f5000 0x7fafc41f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc4159130 0x7fafc41f1260 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 --2- 172.21.15.84:0/593167061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fafb40043a0 0x7fafc40b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 >> 172.21.15.84:0/593167061 conn(0x7fafc4069c70 msgr2=0x7fafc4070070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:48.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.052+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 shutdown_connections 2023-10-11T06:14:48.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.053+0000 7fafcaf1b640 1 -- 172.21.15.84:0/593167061 wait complete. 2023-10-11T06:14:48.073 INFO:tasks.ceph:Scrubbing osd.3 2023-10-11T06:14:48.073 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.291+0000 7ff262bf6640 1 Processor -- start 2023-10-11T06:14:48.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.291+0000 7ff262bf6640 1 -- start start 2023-10-11T06:14:48.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.292+0000 7ff262bf6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff25c15f7e0 0x7ff25c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.292+0000 7ff262bf6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff25c0cd6c0 0x7ff25c0d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.292+0000 7ff262bf6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c0d3420 con 0x7ff25c0d3000 2023-10-11T06:14:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.292+0000 7ff262bf6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0cf220 con 0x7ff25c15f7e0 2023-10-11T06:14:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.292+0000 7ff262bf6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0cf500 con 0x7ff25c0cd6c0 2023-10-11T06:14:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.293+0000 7ff260992640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff25c15f7e0 0x7ff25c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.293+0000 7ff260992640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff25c15f7e0 0x7ff25c0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37222/0 (socket says 172.21.15.84:37222) 2023-10-11T06:14:48.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.293+0000 7ff260992640 1 -- 172.21.15.84:0/308435965 learned_addr learned my addr 172.21.15.84:0/308435965 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:48.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.293+0000 7ff260992640 1 -- 172.21.15.84:0/308435965 >> v2:172.21.15.99:3300/0 conn(0x7ff25c0cd6c0 msgr2=0x7ff25c0d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.293+0000 7ff260992640 1 --2- 172.21.15.84:0/308435965 >> v2:172.21.15.99:3300/0 conn(0x7ff25c0cd6c0 0x7ff25c0d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.293+0000 7ff260992640 1 -- 172.21.15.84:0/308435965 >> v1:172.21.15.99:6789/0 conn(0x7ff25c0d3000 legacy=0x7ff25c15fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.293+0000 7ff260992640 1 -- 172.21.15.84:0/308435965 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0cf7e0 con 0x7ff25c15f7e0 2023-10-11T06:14:48.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.294+0000 7ff260992640 1 --2- 172.21.15.84:0/308435965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c0cd180 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff25c08adb0 tx=0x7ff24c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe34bae176e34092 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.294+0000 7ff25b7fe640 1 -- 172.21.15.84:0/308435965 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c031870 con 0x7ff25c15f7e0 2023-10-11T06:14:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.294+0000 7ff25b7fe640 1 -- 172.21.15.84:0/308435965 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c0319d0 con 0x7ff25c15f7e0 2023-10-11T06:14:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.294+0000 7ff25b7fe640 1 -- 172.21.15.84:0/308435965 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c031d60 con 0x7ff25c15f7e0 2023-10-11T06:14:48.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 -- 172.21.15.84:0/308435965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 msgr2=0x7ff25c0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 --2- 172.21.15.84:0/308435965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c0cd180 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff25c08adb0 tx=0x7ff24c02b910 comp rx=0 tx=0).stop 2023-10-11T06:14:48.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 -- 172.21.15.84:0/308435965 shutdown_connections 2023-10-11T06:14:48.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 --2- 172.21.15.84:0/308435965 >> v2:172.21.15.99:3300/0 conn(0x7ff25c0cd6c0 0x7ff25c0d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 --2- 172.21.15.84:0/308435965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c0cd180 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 -- 172.21.15.84:0/308435965 >> 172.21.15.84:0/308435965 conn(0x7ff25c0713a0 msgr2=0x7ff25c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:48.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 -- 172.21.15.84:0/308435965 shutdown_connections 2023-10-11T06:14:48.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.295+0000 7ff262bf6640 1 -- 172.21.15.84:0/308435965 wait complete. 2023-10-11T06:14:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.297+0000 7ff262bf6640 1 Processor -- start 2023-10-11T06:14:48.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.298+0000 7ff262bf6640 1 -- start start 2023-10-11T06:14:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff262bf6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cd6c0 0x7ff25c0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff262bf6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0d3000 0x7ff25c15fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff262bf6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff262bf6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0c8350 con 0x7ff25c15f7e0 2023-10-11T06:14:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff262bf6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c15ebf0 con 0x7ff25c0cd6c0 2023-10-11T06:14:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff262bf6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0c79b0 con 0x7ff25c0d3000 2023-10-11T06:14:48.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff261193640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff261193640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37230/0 (socket says 172.21.15.84:37230) 2023-10-11T06:14:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.299+0000 7ff261193640 1 -- 172.21.15.84:0/4053357788 learned_addr learned my addr 172.21.15.84:0/4053357788 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff261193640 1 -- 172.21.15.84:0/4053357788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0d3000 msgr2=0x7ff25c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff261193640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff25c0d3000 0x7ff25c15fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff261193640 1 -- 172.21.15.84:0/4053357788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cd6c0 msgr2=0x7ff25c0c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff261193640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25c0cd6c0 0x7ff25c0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff261193640 1 -- 172.21.15.84:0/4053357788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0094d0 con 0x7ff25c15f7e0 2023-10-11T06:14:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff261193640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c160300 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c390 tx=0x7ff25400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2540035f0 con 0x7ff25c15f7e0 2023-10-11T06:14:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254003750 con 0x7ff25c15f7e0 2023-10-11T06:14:48.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c0c7c90 con 0x7ff25c15f7e0 2023-10-11T06:14:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254003b00 con 0x7ff25c15f7e0 2023-10-11T06:14:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.300+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c151ac0 con 0x7ff25c15f7e0 2023-10-11T06:14:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.301+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff25c0cfda0 con 0x7ff25c15f7e0 2023-10-11T06:14:48.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.302+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff254003c60 con 0x7ff25c15f7e0 2023-10-11T06:14:48.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.303+0000 7ff2597fa640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff23c06a860 0x7ff23c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.303+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff2540881f0 con 0x7ff25c15f7e0 2023-10-11T06:14:48.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.304+0000 7ff2597fa640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7ff23c070cd0 0x7ff23c0730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.304+0000 7ff260992640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff23c06a860 0x7ff23c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.304+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 --> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff23c073790 con 0x7ff23c070cd0 2023-10-11T06:14:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.304+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff25408a890 con 0x7ff25c15f7e0 2023-10-11T06:14:48.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.304+0000 7ff25bfff640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7ff23c070cd0 0x7ff23c0730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.304+0000 7ff260992640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff23c06a860 0x7ff23c06cd10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff24c036000 tx=0x7ff24c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.305+0000 7ff25bfff640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7ff23c070cd0 0x7ff23c0730e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:48.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.305+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== osd.3 v2:172.21.15.84:6817/87864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff23c073790 con 0x7ff23c070cd0 2023-10-11T06:14:48.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.319+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 --> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff25c0cfac0 con 0x7ff23c070cd0 2023-10-11T06:14:48.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff2597fa640 1 -- 172.21.15.84:0/4053357788 <== osd.3 v2:172.21.15.84:6817/87864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff24400acc0 con 0x7ff23c070cd0 2023-10-11T06:14:48.322 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:48.323 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:48.323 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7ff23c070cd0 msgr2=0x7ff23c0730e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff262bf6640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6817/87864,v1:172.21.15.84:6820/87864] conn(0x7ff23c070cd0 0x7ff23c0730e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff23c06a860 msgr2=0x7ff23c06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff262bf6640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff23c06a860 0x7ff23c06cd10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff24c036000 tx=0x7ff24c030000 comp rx=0 tx=0).stop 2023-10-11T06:14:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 msgr2=0x7ff25c160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff262bf6640 1 --2- 172.21.15.84:0/4053357788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff25c15f7e0 0x7ff25c160300 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c390 tx=0x7ff25400c850 comp rx=0 tx=0).stop 2023-10-11T06:14:48.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.320+0000 7ff261193640 1 -- 172.21.15.84:0/4053357788 reap_dead start 2023-10-11T06:14:48.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.321+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 shutdown_connections 2023-10-11T06:14:48.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.321+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 >> 172.21.15.84:0/4053357788 conn(0x7ff25c0713a0 msgr2=0x7ff25c071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:48.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.321+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 shutdown_connections 2023-10-11T06:14:48.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.321+0000 7ff262bf6640 1 -- 172.21.15.84:0/4053357788 wait complete. 2023-10-11T06:14:48.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-11T06:14:48.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.556+0000 7fc9a759e640 1 Processor -- start 2023-10-11T06:14:48.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.556+0000 7fc9a759e640 1 -- start start 2023-10-11T06:14:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9a80c9330 0x7fc9a8158730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a759e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a806c3d0 0x7fc9a8158c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a759e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80adc40 con 0x7fc9a806c000 2023-10-11T06:14:48.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a759e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a81591b0 con 0x7fc9a80c9330 2023-10-11T06:14:48.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a759e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80c97e0 con 0x7fc9a806c3d0 2023-10-11T06:14:48.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a5d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9a80c9330 0x7fc9a8158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a5d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc9a80c9330 0x7fc9a8158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37242/0 (socket says 172.21.15.84:37242) 2023-10-11T06:14:48.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/2409564298 learned_addr learned my addr 172.21.15.84:0/2409564298 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:48.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a6d9d640 1 --2- 172.21.15.84:0/2409564298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a806c3d0 0x7fc9a8158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/2409564298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a806c3d0 msgr2=0x7fc9a8158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:48.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a5d9b640 1 --2- 172.21.15.84:0/2409564298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a806c3d0 0x7fc9a8158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/2409564298 >> v1:172.21.15.99:6789/0 conn(0x7fc9a806c000 legacy=0x7fc9a80c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:48.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/2409564298 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a80c9ac0 con 0x7fc9a80c9330 2023-10-11T06:14:48.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.558+0000 7fc9a6d9d640 1 --2- 172.21.15.84:0/2409564298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a806c3d0 0x7fc9a8158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:48.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.559+0000 7fc9a5d9b640 1 --2- 172.21.15.84:0/2409564298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a80c9330 0x7fc9a8158730 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009870 tx=0x7fc99c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0da3338674bcc7e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:48.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.560+0000 7fc9a559a640 1 -- 172.21.15.84:0/2409564298 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c031640 con 0x7fc9a80c9330 2023-10-11T06:14:48.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.560+0000 7fc9a559a640 1 -- 172.21.15.84:0/2409564298 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c0317a0 con 0x7fc9a80c9330 2023-10-11T06:14:48.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.560+0000 7fc9a559a640 1 -- 172.21.15.84:0/2409564298 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c031b30 con 0x7fc9a80c9330 2023-10-11T06:14:48.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.560+0000 7fc9a759e640 1 -- 172.21.15.84:0/2409564298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a80c9330 msgr2=0x7fc9a8158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.560+0000 7fc9a759e640 1 --2- 172.21.15.84:0/2409564298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a80c9330 0x7fc9a8158730 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009870 tx=0x7fc99c02b8f0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.561+0000 7fc9a759e640 1 -- 172.21.15.84:0/2409564298 shutdown_connections 2023-10-11T06:14:48.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.561+0000 7fc9a759e640 1 --2- 172.21.15.84:0/2409564298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a806c3d0 0x7fc9a8158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.561+0000 7fc9a759e640 1 --2- 172.21.15.84:0/2409564298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a80c9330 0x7fc9a8158730 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.561+0000 7fc9a759e640 1 -- 172.21.15.84:0/2409564298 >> 172.21.15.84:0/2409564298 conn(0x7fc9a8069c70 msgr2=0x7fc9a806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:48.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.561+0000 7fc9a759e640 1 -- 172.21.15.84:0/2409564298 shutdown_connections 2023-10-11T06:14:48.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.561+0000 7fc9a759e640 1 -- 172.21.15.84:0/2409564298 wait complete. 2023-10-11T06:14:48.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.562+0000 7fc9a759e640 1 Processor -- start 2023-10-11T06:14:48.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.563+0000 7fc9a759e640 1 -- start start 2023-10-11T06:14:48.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.563+0000 7fc9a759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a806c000 0x7fc9a80c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.563+0000 7fc9a759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a806c3d0 0x7fc9a81523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.563+0000 7fc9a759e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a80c9330 0x7fc9a80af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.563+0000 7fc9a759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81f51c0 con 0x7fc9a806c3d0 2023-10-11T06:14:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.563+0000 7fc9a759e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80c2d00 con 0x7fc9a806c000 2023-10-11T06:14:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.563+0000 7fc9a759e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81f4890 con 0x7fc9a80c9330 2023-10-11T06:14:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a806c3d0 0x7fc9a81523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a806c3d0 0x7fc9a81523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37244/0 (socket says 172.21.15.84:37244) 2023-10-11T06:14:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/919725411 learned_addr learned my addr 172.21.15.84:0/919725411 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a659c640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a806c000 0x7fc9a80c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a80c9330 msgr2=0x7fc9a80af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a6d9d640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a80c9330 0x7fc9a80af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a80c9330 0x7fc9a80af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a806c000 msgr2=0x7fc9a80c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a806c000 0x7fc9a80c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.564+0000 7fc9a5d9b640 1 -- 172.21.15.84:0/919725411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0094d0 con 0x7fc9a806c3d0 2023-10-11T06:14:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.565+0000 7fc9a5d9b640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a806c3d0 0x7fc9a81523b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0090b0 tx=0x7fc99c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.565+0000 7fc996ffd640 1 -- 172.21.15.84:0/919725411 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c03ac60 con 0x7fc9a806c3d0 2023-10-11T06:14:48.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.565+0000 7fc996ffd640 1 -- 172.21.15.84:0/919725411 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c03ade0 con 0x7fc9a806c3d0 2023-10-11T06:14:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.565+0000 7fc996ffd640 1 -- 172.21.15.84:0/919725411 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c044630 con 0x7fc9a806c3d0 2023-10-11T06:14:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.565+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a81f4b70 con 0x7fc9a806c3d0 2023-10-11T06:14:48.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.565+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a80c2350 con 0x7fc9a806c3d0 2023-10-11T06:14:48.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.566+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a81f5330 con 0x7fc9a806c3d0 2023-10-11T06:14:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.567+0000 7fc996ffd640 1 -- 172.21.15.84:0/919725411 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc99c02f050 con 0x7fc9a806c3d0 2023-10-11T06:14:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.568+0000 7fc996ffd640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc98806a830 0x7fc98806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.568+0000 7fc996ffd640 1 -- 172.21.15.84:0/919725411 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc99c0aadd0 con 0x7fc9a806c3d0 2023-10-11T06:14:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.568+0000 7fc9a659c640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc98806a830 0x7fc98806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.569+0000 7fc9a659c640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc98806a830 0x7fc98806cce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc998003d90 tx=0x7fc998009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:48.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.572+0000 7fc996ffd640 1 -- 172.21.15.84:0/919725411 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc99c076380 con 0x7fc9a806c3d0 2023-10-11T06:14:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.764+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9a80706c0 con 0x7fc98806a830 2023-10-11T06:14:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.765+0000 7fc996ffd640 1 -- 172.21.15.84:0/919725411 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc998003a30 con 0x7fc98806a830 2023-10-11T06:14:48.768 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 3 to deep-scrub 2023-10-11T06:14:48.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc98806a830 msgr2=0x7fc98806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc98806a830 0x7fc98806cce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc998003d90 tx=0x7fc998009040 comp rx=0 tx=0).stop 2023-10-11T06:14:48.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a806c3d0 msgr2=0x7fc9a81523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:48.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a806c3d0 0x7fc9a81523b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0090b0 tx=0x7fc99c0046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 shutdown_connections 2023-10-11T06:14:48.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc98806a830 0x7fc98806cce0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc9a80c9330 0x7fc9a80af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9a806c3d0 0x7fc9a81523b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 --2- 172.21.15.84:0/919725411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a806c000 0x7fc9a80c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:48.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 >> 172.21.15.84:0/919725411 conn(0x7fc9a8069c70 msgr2=0x7fc9a8071460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:48.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 shutdown_connections 2023-10-11T06:14:48.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:48.768+0000 7fc9a759e640 1 -- 172.21.15.84:0/919725411 wait complete. 2023-10-11T06:14:48.788 INFO:tasks.ceph:Scrubbing osd.4 2023-10-11T06:14:48.789 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:49.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.005+0000 7f87406f7640 1 Processor -- start 2023-10-11T06:14:49.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.006+0000 7f87406f7640 1 -- start start 2023-10-11T06:14:49.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.007+0000 7f87406f7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f873815f7e0 0x7f87380cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.007+0000 7f87406f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380d31d0 0x7f87380d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.007+0000 7f87406f7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87380cf220 con 0x7f87380cd6c0 2023-10-11T06:14:49.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.007+0000 7f87406f7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87380cf500 con 0x7f873815f7e0 2023-10-11T06:14:49.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.007+0000 7f87406f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87380cf7e0 con 0x7f87380d31d0 2023-10-11T06:14:49.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.008+0000 7f873e493640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f873815f7e0 0x7f87380cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.008+0000 7f873ec94640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380d31d0 0x7f87380d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.008+0000 7f873e493640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f873815f7e0 0x7f87380cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.84:33236/0 (socket says 172.21.15.84:33236) 2023-10-11T06:14:49.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.008+0000 7f873e493640 1 -- 172.21.15.84:0/455928782 learned_addr learned my addr 172.21.15.84:0/455928782 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:49.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.008+0000 7f873ec94640 1 --2- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380d31d0 0x7f87380d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.84:54116/0 (socket says 172.21.15.84:54116) 2023-10-11T06:14:49.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.009+0000 7f873e493640 1 -- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380d31d0 msgr2=0x7f87380d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:49.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.009+0000 7f873e493640 1 --2- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380d31d0 0x7f87380d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.009+0000 7f873e493640 1 -- 172.21.15.84:0/455928782 >> v1:172.21.15.99:6789/0 conn(0x7f87380cd6c0 legacy=0x7f87380d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.009+0000 7f873e493640 1 -- 172.21.15.84:0/455928782 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380cfac0 con 0x7f873815f7e0 2023-10-11T06:14:49.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.009+0000 7f873ec94640 1 --2- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380d31d0 0x7f87380d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-11T06:14:49.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.010+0000 7f873e493640 1 --2- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f873815f7e0 0x7f87380cd180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8734009e50 tx=0x7f873402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63dd720c2f60ca26 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f873d491640 1 -- 172.21.15.84:0/455928782 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873402cb00 con 0x7f873815f7e0 2023-10-11T06:14:49.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f873d491640 1 -- 172.21.15.84:0/455928782 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873402cc60 con 0x7f873815f7e0 2023-10-11T06:14:49.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f873d491640 1 -- 172.21.15.84:0/455928782 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340343f0 con 0x7f873815f7e0 2023-10-11T06:14:49.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f87406f7640 1 -- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f873815f7e0 msgr2=0x7f87380cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f87406f7640 1 --2- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f873815f7e0 0x7f87380cd180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8734009e50 tx=0x7f873402bf60 comp rx=0 tx=0).stop 2023-10-11T06:14:49.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f87406f7640 1 -- 172.21.15.84:0/455928782 shutdown_connections 2023-10-11T06:14:49.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f87406f7640 1 --2- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380d31d0 0x7f87380d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f87406f7640 1 --2- 172.21.15.84:0/455928782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f873815f7e0 0x7f87380cd180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.011+0000 7f87406f7640 1 -- 172.21.15.84:0/455928782 >> 172.21.15.84:0/455928782 conn(0x7f87380713a0 msgr2=0x7f87380717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:49.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.012+0000 7f87406f7640 1 -- 172.21.15.84:0/455928782 shutdown_connections 2023-10-11T06:14:49.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.012+0000 7f87406f7640 1 -- 172.21.15.84:0/455928782 wait complete. 2023-10-11T06:14:49.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.013+0000 7f87406f7640 1 Processor -- start 2023-10-11T06:14:49.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.013+0000 7f87406f7640 1 -- start start 2023-10-11T06:14:49.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.014+0000 7f87406f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380cd6c0 0x7f87380c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.014+0000 7f87406f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87380d31d0 0x7f87381f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.014+0000 7f87406f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873815f7e0 0x7f87380b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.014+0000 7f87406f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87380cae40 con 0x7f873815f7e0 2023-10-11T06:14:49.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.014+0000 7f87406f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738149b10 con 0x7f87380d31d0 2023-10-11T06:14:49.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.014+0000 7f87406f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87380ca300 con 0x7f87380cd6c0 2023-10-11T06:14:49.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873815f7e0 0x7f87380b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873815f7e0 0x7f87380b69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41542/0 (socket says 172.21.15.84:41542) 2023-10-11T06:14:49.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 -- 172.21.15.84:0/3527509372 learned_addr learned my addr 172.21.15.84:0/3527509372 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:49.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873dc92640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87380d31d0 0x7f87381f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873e493640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380cd6c0 0x7f87380c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 -- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380cd6c0 msgr2=0x7f87380c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f87380cd6c0 0x7f87380c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 -- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87380d31d0 msgr2=0x7f87381f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87380d31d0 0x7f87381f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 -- 172.21.15.84:0/3527509372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009400 con 0x7f873815f7e0 2023-10-11T06:14:49.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873dc92640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87380d31d0 0x7f87381f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:49.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.015+0000 7f873ec94640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873815f7e0 0x7f87380b69b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f873000e900 tx=0x7f873000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.016+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300096e0 con 0x7f873815f7e0 2023-10-11T06:14:49.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.016+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873000ca00 con 0x7f873815f7e0 2023-10-11T06:14:49.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.016+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8734009b30 con 0x7f873815f7e0 2023-10-11T06:14:49.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.016+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000cd70 con 0x7f873815f7e0 2023-10-11T06:14:49.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.016+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8738149d20 con 0x7f873815f7e0 2023-10-11T06:14:49.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.017+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87380cfda0 con 0x7f873815f7e0 2023-10-11T06:14:49.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.018+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f8730013470 con 0x7f873815f7e0 2023-10-11T06:14:49.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.019+0000 7f872effd640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f871c06a830 0x7f871c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.019+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8730088fe0 con 0x7f873815f7e0 2023-10-11T06:14:49.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.019+0000 7f873e493640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f871c06a830 0x7f871c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.019+0000 7f872effd640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7f871c070ca0 0x7f871c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.019+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 --> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f871c073760 con 0x7f871c070ca0 2023-10-11T06:14:49.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.019+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f873008b5c0 con 0x7f873815f7e0 2023-10-11T06:14:49.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.020+0000 7f873e493640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f871c06a830 0x7f871c06cce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8734035000 tx=0x7f8734030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.020+0000 7f873dc92640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7f871c070ca0 0x7f871c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.021+0000 7f873dc92640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7f871c070ca0 0x7f871c0730b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.023+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== osd.4 v2:172.21.15.99:6800/88014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f871c073760 con 0x7f871c070ca0 2023-10-11T06:14:49.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.037+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 --> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f87380cee40 con 0x7f871c070ca0 2023-10-11T06:14:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.038+0000 7f872effd640 1 -- 172.21.15.84:0/3527509372 <== osd.4 v2:172.21.15.99:6800/88014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f87280050f0 con 0x7f871c070ca0 2023-10-11T06:14:49.041 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:49.041 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:49.041 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:49.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7f871c070ca0 msgr2=0x7f871c0730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.99:6800/88014,v1:172.21.15.99:6801/88014] conn(0x7f871c070ca0 0x7f871c0730b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f871c06a830 msgr2=0x7f871c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f871c06a830 0x7f871c06cce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8734035000 tx=0x7f8734030000 comp rx=0 tx=0).stop 2023-10-11T06:14:49.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873815f7e0 msgr2=0x7f87380b69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 --2- 172.21.15.84:0/3527509372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f873815f7e0 0x7f87380b69b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f873000e900 tx=0x7f873000edc0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f873ec94640 1 -- 172.21.15.84:0/3527509372 reap_dead start 2023-10-11T06:14:49.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 shutdown_connections 2023-10-11T06:14:49.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.039+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 >> 172.21.15.84:0/3527509372 conn(0x7f87380713a0 msgr2=0x7f8738071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:49.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.040+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 shutdown_connections 2023-10-11T06:14:49.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.040+0000 7f87406f7640 1 -- 172.21.15.84:0/3527509372 wait complete. 2023-10-11T06:14:49.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-11T06:14:49.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.272+0000 7fe848ef2640 1 Processor -- start 2023-10-11T06:14:49.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.272+0000 7fe848ef2640 1 -- start start 2023-10-11T06:14:49.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.273+0000 7fe848ef2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8440afce0 0x7fe8440b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.273+0000 7fe848ef2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe84415dc40 0x7fe8440cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.273+0000 7fe848ef2640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84415a530 con 0x7fe8440cdd60 2023-10-11T06:14:49.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.273+0000 7fe848ef2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8440b1590 con 0x7fe8440afce0 2023-10-11T06:14:49.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.273+0000 7fe848ef2640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8440ce130 con 0x7fe84415dc40 2023-10-11T06:14:49.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.273+0000 7fe84259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8440afce0 0x7fe8440b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.274+0000 7fe84259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8440afce0 0x7fe8440b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41558/0 (socket says 172.21.15.84:41558) 2023-10-11T06:14:49.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.274+0000 7fe84259c640 1 -- 172.21.15.84:0/4101137418 learned_addr learned my addr 172.21.15.84:0/4101137418 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:49.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.274+0000 7fe84259c640 1 -- 172.21.15.84:0/4101137418 >> v2:172.21.15.99:3300/0 conn(0x7fe84415dc40 msgr2=0x7fe8440cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:49.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.274+0000 7fe84259c640 1 --2- 172.21.15.84:0/4101137418 >> v2:172.21.15.99:3300/0 conn(0x7fe84415dc40 0x7fe8440cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.274+0000 7fe84259c640 1 -- 172.21.15.84:0/4101137418 >> v1:172.21.15.99:6789/0 conn(0x7fe8440cdd60 legacy=0x7fe8440cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:49.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.274+0000 7fe84259c640 1 -- 172.21.15.84:0/4101137418 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8440b0cf0 con 0x7fe8440afce0 2023-10-11T06:14:49.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.275+0000 7fe84259c640 1 --2- 172.21.15.84:0/4101137418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8440afce0 0x7fe8440b05f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe844072140 tx=0x7fe83402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2205e8d25bd968e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.275+0000 7fe84159a640 1 -- 172.21.15.84:0/4101137418 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83402ca50 con 0x7fe8440afce0 2023-10-11T06:14:49.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.275+0000 7fe84159a640 1 -- 172.21.15.84:0/4101137418 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83402cbb0 con 0x7fe8440afce0 2023-10-11T06:14:49.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.275+0000 7fe84159a640 1 -- 172.21.15.84:0/4101137418 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340343f0 con 0x7fe8440afce0 2023-10-11T06:14:49.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 -- 172.21.15.84:0/4101137418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8440afce0 msgr2=0x7fe8440b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 --2- 172.21.15.84:0/4101137418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8440afce0 0x7fe8440b05f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe844072140 tx=0x7fe83402beb0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 -- 172.21.15.84:0/4101137418 shutdown_connections 2023-10-11T06:14:49.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 --2- 172.21.15.84:0/4101137418 >> v2:172.21.15.99:3300/0 conn(0x7fe84415dc40 0x7fe8440cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 --2- 172.21.15.84:0/4101137418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8440afce0 0x7fe8440b05f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 -- 172.21.15.84:0/4101137418 >> 172.21.15.84:0/4101137418 conn(0x7fe84406a590 msgr2=0x7fe844099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:49.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 -- 172.21.15.84:0/4101137418 shutdown_connections 2023-10-11T06:14:49.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.276+0000 7fe848ef2640 1 -- 172.21.15.84:0/4101137418 wait complete. 2023-10-11T06:14:49.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.279+0000 7fe848ef2640 1 Processor -- start 2023-10-11T06:14:49.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.279+0000 7fe848ef2640 1 -- start start 2023-10-11T06:14:49.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.280+0000 7fe848ef2640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8440afce0 0x7fe8441e4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.280+0000 7fe848ef2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8440cdd60 0x7fe8441e6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.280+0000 7fe848ef2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84415dc40 0x7fe8440cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.280+0000 7fe848ef2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441f6120 con 0x7fe84415dc40 2023-10-11T06:14:49.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.280+0000 7fe848ef2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441f55a0 con 0x7fe8440cdd60 2023-10-11T06:14:49.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.280+0000 7fe848ef2640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441f5880 con 0x7fe8440afce0 2023-10-11T06:14:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84415dc40 0x7fe8440cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84415dc40 0x7fe8440cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41572/0 (socket says 172.21.15.84:41572) 2023-10-11T06:14:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 -- 172.21.15.84:0/1503233619 learned_addr learned my addr 172.21.15.84:0/1503233619 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:49.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 -- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8440afce0 msgr2=0x7fe8441e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8440afce0 0x7fe8441e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 -- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8440cdd60 msgr2=0x7fe8441e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe841d9b640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8440cdd60 0x7fe8441e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8440cdd60 0x7fe8441e6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 -- 172.21.15.84:0/1503233619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009b30 con 0x7fe84415dc40 2023-10-11T06:14:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe841d9b640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8440cdd60 0x7fe8441e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.281+0000 7fe842d9d640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84415dc40 0x7fe8440cbd40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c390 tx=0x7fe83c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.282+0000 7fe83affd640 1 -- 172.21.15.84:0/1503233619 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0035f0 con 0x7fe84415dc40 2023-10-11T06:14:49.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.282+0000 7fe83affd640 1 -- 172.21.15.84:0/1503233619 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c003750 con 0x7fe84415dc40 2023-10-11T06:14:49.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.282+0000 7fe83affd640 1 -- 172.21.15.84:0/1503233619 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c003b00 con 0x7fe84415dc40 2023-10-11T06:14:49.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.282+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8441f5b60 con 0x7fe84415dc40 2023-10-11T06:14:49.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.282+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8440b4260 con 0x7fe84415dc40 2023-10-11T06:14:49.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.283+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8440b43f0 con 0x7fe84415dc40 2023-10-11T06:14:49.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.284+0000 7fe83affd640 1 -- 172.21.15.84:0/1503233619 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fe83c003c60 con 0x7fe84415dc40 2023-10-11T06:14:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.285+0000 7fe83affd640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe82406a830 0x7fe82406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.285+0000 7fe83affd640 1 -- 172.21.15.84:0/1503233619 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe83c087f30 con 0x7fe84415dc40 2023-10-11T06:14:49.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.285+0000 7fe84259c640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe82406a830 0x7fe82406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.285+0000 7fe84259c640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe82406a830 0x7fe82406cce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe834035000 tx=0x7fe83402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.289+0000 7fe83affd640 1 -- 172.21.15.84:0/1503233619 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fe8440b43f0 con 0x7fe84415dc40 2023-10-11T06:14:49.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.478+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8440b12b0 con 0x7fe82406a830 2023-10-11T06:14:49.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.479+0000 7fe83affd640 1 -- 172.21.15.84:0/1503233619 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe834009510 con 0x7fe82406a830 2023-10-11T06:14:49.482 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 4 to deep-scrub 2023-10-11T06:14:49.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe82406a830 msgr2=0x7fe82406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe82406a830 0x7fe82406cce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe834035000 tx=0x7fe83402e000 comp rx=0 tx=0).stop 2023-10-11T06:14:49.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84415dc40 msgr2=0x7fe8440cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84415dc40 0x7fe8440cbd40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c390 tx=0x7fe83c00c850 comp rx=0 tx=0).stop 2023-10-11T06:14:49.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 shutdown_connections 2023-10-11T06:14:49.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fe82406a830 0x7fe82406cce0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe84415dc40 0x7fe8440cbd40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8440cdd60 0x7fe8441e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 --2- 172.21.15.84:0/1503233619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8440afce0 0x7fe8441e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 >> 172.21.15.84:0/1503233619 conn(0x7fe84406a590 msgr2=0x7fe844099ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 shutdown_connections 2023-10-11T06:14:49.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.482+0000 7fe848ef2640 1 -- 172.21.15.84:0/1503233619 wait complete. 2023-10-11T06:14:49.503 INFO:tasks.ceph:Scrubbing osd.5 2023-10-11T06:14:49.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:49.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.723+0000 7f4328ab6640 1 Processor -- start 2023-10-11T06:14:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.723+0000 7f4328ab6640 1 -- start start 2023-10-11T06:14:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f4328ab6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324150300 0x7f43240c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f4328ab6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43240ce510 0x7f43240ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f4328ab6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240c9b90 con 0x7f432414ff30 2023-10-11T06:14:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f4328ab6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432415da10 con 0x7f43240ce510 2023-10-11T06:14:49.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f4328ab6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240ceef0 con 0x7f4324150300 2023-10-11T06:14:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f432259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f432414ff30 0x7f43240cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60094/0 (socket says 172.21.15.84:60094) 2023-10-11T06:14:49.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f432259c640 1 -- 172.21.15.84:0/3842506161 learned_addr learned my addr 172.21.15.84:0/3842506161 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f4322d9d640 1 --2- 172.21.15.84:0/3842506161 >> v2:172.21.15.99:3300/0 conn(0x7f43240ce510 0x7f43240ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150555958 0 0) 0x7f43240c9b90 con 0x7f432414ff30 2023-10-11T06:14:49.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.725+0000 7f4321d9b640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324150300 0x7f43240c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f430c003520 con 0x7f432414ff30 2023-10-11T06:14:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434578228 0 0) 0x7f430c003520 con 0x7f432414ff30 2023-10-11T06:14:49.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324150300 msgr2=0x7f43240c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324150300 0x7f43240c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 >> v2:172.21.15.99:3300/0 conn(0x7f43240ce510 msgr2=0x7f43240ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:49.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 --2- 172.21.15.84:0/3842506161 >> v2:172.21.15.99:3300/0 conn(0x7f43240ce510 0x7f43240ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240cc990 con 0x7f432414ff30 2023-10-11T06:14:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7f4314001670 con 0x7f432414ff30 2023-10-11T06:14:49.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f4322d9d640 1 --2- 172.21.15.84:0/3842506161 >> v2:172.21.15.99:3300/0 conn(0x7f43240ce510 0x7f43240ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-11T06:14:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 >> v1:172.21.15.84:6789/0 conn(0x7f432414ff30 legacy=0x7f43240cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f430c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324150300 0x7f430c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f430c004d00 0x7f430c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432415da10 con 0x7f430c003970 2023-10-11T06:14:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240ceef0 con 0x7f430c004d00 2023-10-11T06:14:49.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430c00ba00 con 0x7f4324150300 2023-10-11T06:14:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.726+0000 7f432259c640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f430c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.727+0000 7f432259c640 1 -- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324150300 msgr2=0x7f430c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.727+0000 7f432259c640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324150300 0x7f430c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.727+0000 7f4322d9d640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f430c004d00 0x7f430c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.727+0000 7f432259c640 1 -- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f430c004d00 msgr2=0x7f430c005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:49.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.727+0000 7f432259c640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f430c004d00 0x7f430c005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.727+0000 7f432259c640 1 -- 172.21.15.84:0/3842506161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c003520 con 0x7f430c003970 2023-10-11T06:14:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.728+0000 7f432259c640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f430c003d80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f431400c240 tx=0x7f430c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cc11c02613be0d server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.728+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314005ad0 con 0x7f430c003970 2023-10-11T06:14:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.728+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314005c50 con 0x7f430c003970 2023-10-11T06:14:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.728+0000 7f432159a640 1 -- 172.21.15.84:0/3842506161 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431402c8d0 con 0x7f430c003970 2023-10-11T06:14:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.728+0000 7f4328ab6640 1 -- 172.21.15.84:0/3842506161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 msgr2=0x7f430c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.728+0000 7f4328ab6640 1 --2- 172.21.15.84:0/3842506161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f430c003d80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f431400c240 tx=0x7f430c0057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.729+0000 7f4322d9d640 1 -- 172.21.15.84:0/3842506161 reap_dead start 2023-10-11T06:14:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.729+0000 7f4328ab6640 1 -- 172.21.15.84:0/3842506161 shutdown_connections 2023-10-11T06:14:49.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.729+0000 7f4328ab6640 1 -- 172.21.15.84:0/3842506161 >> 172.21.15.84:0/3842506161 conn(0x7f4324069c70 msgr2=0x7f432406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:49.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.729+0000 7f4328ab6640 1 -- 172.21.15.84:0/3842506161 shutdown_connections 2023-10-11T06:14:49.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.729+0000 7f4328ab6640 1 -- 172.21.15.84:0/3842506161 wait complete. 2023-10-11T06:14:49.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.730+0000 7f4328ab6640 1 Processor -- start 2023-10-11T06:14:49.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.731+0000 7f4328ab6640 1 -- start start 2023-10-11T06:14:49.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.731+0000 7f4328ab6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f4324070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.731+0000 7f4328ab6640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324145a10 0x7f43241438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f4328ab6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4324143de0 0x7f4324155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f4328ab6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324158ea0 con 0x7f430c003970 2023-10-11T06:14:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f4328ab6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324154b10 con 0x7f4324143de0 2023-10-11T06:14:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f4328ab6640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4324154df0 con 0x7f4324145a10 2023-10-11T06:14:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f4324070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f4324070210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41594/0 (socket says 172.21.15.84:41594) 2023-10-11T06:14:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 -- 172.21.15.84:0/3761406426 learned_addr learned my addr 172.21.15.84:0/3761406426 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 -- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324145a10 msgr2=0x7f43241438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4324145a10 0x7f43241438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 -- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4324143de0 msgr2=0x7f4324155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4324143de0 0x7f4324155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 -- 172.21.15.84:0/3761406426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43241550d0 con 0x7f430c003970 2023-10-11T06:14:49.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.732+0000 7f432259c640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f4324070210 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f431400c240 tx=0x7f432415db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.733+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431403c040 con 0x7f430c003970 2023-10-11T06:14:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.733+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431402dd70 con 0x7f430c003970 2023-10-11T06:14:49.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.733+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314007290 con 0x7f430c003970 2023-10-11T06:14:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.733+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43241553b0 con 0x7f430c003970 2023-10-11T06:14:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.733+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4324155970 con 0x7f430c003970 2023-10-11T06:14:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.734+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f432415d0e0 con 0x7f430c003970 2023-10-11T06:14:49.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.735+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f4314032070 con 0x7f430c003970 2023-10-11T06:14:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.736+0000 7f4312ffd640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f430406a830 0x7f430406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.736+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f43140aa550 con 0x7f430c003970 2023-10-11T06:14:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.736+0000 7f4312ffd640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f4304070ca0 0x7f43040730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.736+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 --> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4304073760 con 0x7f4304070ca0 2023-10-11T06:14:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.736+0000 7f4321d9b640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f430406a830 0x7f430406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.736+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43140acab0 con 0x7f430c003970 2023-10-11T06:14:49.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.737+0000 7f4321d9b640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f430406a830 0x7f430406cce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4318006a90 tx=0x7f4318006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.737+0000 7f4322d9d640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f4304070ca0 0x7f43040730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.738+0000 7f4322d9d640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f4304070ca0 0x7f43040730b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.740+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== osd.5 v2:172.21.15.99:6808/88015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4304073760 con 0x7f4304070ca0 2023-10-11T06:14:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.753+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 --> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f432415ce00 con 0x7f4304070ca0 2023-10-11T06:14:49.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4312ffd640 1 -- 172.21.15.84:0/3761406426 <== osd.5 v2:172.21.15.99:6808/88015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f431c003dc0 con 0x7f4304070ca0 2023-10-11T06:14:49.757 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:49.757 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:49.758 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:49.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f4304070ca0 msgr2=0x7f43040730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4328ab6640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.99:6808/88015,v1:172.21.15.99:6809/88015] conn(0x7f4304070ca0 0x7f43040730b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f430406a830 msgr2=0x7f430406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4328ab6640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f430406a830 0x7f430406cce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4318006a90 tx=0x7f4318006a20 comp rx=0 tx=0).stop 2023-10-11T06:14:49.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 msgr2=0x7f4324070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4328ab6640 1 --2- 172.21.15.84:0/3761406426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f430c003970 0x7f4324070210 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f431400c240 tx=0x7f432415db50 comp rx=0 tx=0).stop 2023-10-11T06:14:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4322d9d640 1 -- 172.21.15.84:0/3761406426 reap_dead start 2023-10-11T06:14:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.755+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 shutdown_connections 2023-10-11T06:14:49.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.756+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 >> 172.21.15.84:0/3761406426 conn(0x7f4324069c70 msgr2=0x7f43240cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.756+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 shutdown_connections 2023-10-11T06:14:49.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.756+0000 7f4328ab6640 1 -- 172.21.15.84:0/3761406426 wait complete. 2023-10-11T06:14:49.774 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-11T06:14:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.992+0000 7f55d0700640 1 Processor -- start 2023-10-11T06:14:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.993+0000 7f55d0700640 1 -- start start 2023-10-11T06:14:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.994+0000 7f55d0700640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f55c80cafb0 0x7f55c80cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.994+0000 7f55d0700640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c80cb970 con 0x7f55c815c290 2023-10-11T06:14:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.994+0000 7f55d0700640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c815fe40 con 0x7f55c80cc410 2023-10-11T06:14:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.994+0000 7f55d0700640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c80cad70 con 0x7f55c80cafb0 2023-10-11T06:14:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.994+0000 7f55ce49c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f55c815c290 0x7f55c80c8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60108/0 (socket says 172.21.15.84:60108) 2023-10-11T06:14:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.995+0000 7f55ce49c640 1 -- 172.21.15.84:0/753266521 learned_addr learned my addr 172.21.15.84:0/753266521 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.995+0000 7f55cdc9b640 1 --2- 172.21.15.84:0/753266521 >> v2:172.21.15.84:3300/0 conn(0x7f55c80cafb0 0x7f55c80cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.995+0000 7f55cd49a640 1 -- 172.21.15.84:0/753266521 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863209324 0 0) 0x7f55c80cb970 con 0x7f55c815c290 2023-10-11T06:14:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.995+0000 7f55cd49a640 1 -- 172.21.15.84:0/753266521 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b4003520 con 0x7f55c815c290 2023-10-11T06:14:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.995+0000 7f55cdc9b640 1 -- 172.21.15.84:0/753266521 >> v1:172.21.15.99:6789/0 conn(0x7f55c80cc410 legacy=0x7f55c80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.995+0000 7f55cdc9b640 1 -- 172.21.15.84:0/753266521 >> v1:172.21.15.84:6789/0 conn(0x7f55c815c290 legacy=0x7f55c80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.995+0000 7f55cdc9b640 1 -- 172.21.15.84:0/753266521 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c806ff30 con 0x7f55c80cafb0 2023-10-11T06:14:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.996+0000 7f55cdc9b640 1 --2- 172.21.15.84:0/753266521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c80cafb0 0x7f55c80cb430 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f55b80097a0 tx=0x7f55b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8cbb8210a30ca5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55cd49a640 1 -- 172.21.15.84:0/753266521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8031830 con 0x7f55c80cafb0 2023-10-11T06:14:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55cd49a640 1 -- 172.21.15.84:0/753266521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b8031990 con 0x7f55c80cafb0 2023-10-11T06:14:50.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55cd49a640 1 -- 172.21.15.84:0/753266521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8031d00 con 0x7f55c80cafb0 2023-10-11T06:14:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55d0700640 1 -- 172.21.15.84:0/753266521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c80cafb0 msgr2=0x7f55c80cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55d0700640 1 --2- 172.21.15.84:0/753266521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c80cafb0 0x7f55c80cb430 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f55b80097a0 tx=0x7f55b802b8d0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55d0700640 1 -- 172.21.15.84:0/753266521 shutdown_connections 2023-10-11T06:14:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55d0700640 1 --2- 172.21.15.84:0/753266521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c80cafb0 0x7f55c80cb430 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.997+0000 7f55d0700640 1 -- 172.21.15.84:0/753266521 >> 172.21.15.84:0/753266521 conn(0x7f55c8069c70 msgr2=0x7f55c806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.998+0000 7f55d0700640 1 -- 172.21.15.84:0/753266521 shutdown_connections 2023-10-11T06:14:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.998+0000 7f55d0700640 1 -- 172.21.15.84:0/753266521 wait complete. 2023-10-11T06:14:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.999+0000 7f55d0700640 1 Processor -- start 2023-10-11T06:14:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:49.999+0000 7f55d0700640 1 -- start start 2023-10-11T06:14:50.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.000+0000 7f55d0700640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55c80cafb0 0x7f55c80c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.000+0000 7f55d0700640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c80cc410 0x7f55c80b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.000+0000 7f55d0700640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c815c290 0x7f55c80b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.000+0000 7f55d0700640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80b2690 con 0x7f55c815c290 2023-10-11T06:14:50.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.000+0000 7f55d0700640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80b2970 con 0x7f55c80cc410 2023-10-11T06:14:50.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.000+0000 7f55d0700640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c815c660 con 0x7f55c80cafb0 2023-10-11T06:14:50.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c815c290 0x7f55c80b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c815c290 0x7f55c80b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41614/0 (socket says 172.21.15.84:41614) 2023-10-11T06:14:50.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 -- 172.21.15.84:0/1691628971 learned_addr learned my addr 172.21.15.84:0/1691628971 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:50.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cdc9b640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c80cc410 0x7f55c80b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 -- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55c80cafb0 msgr2=0x7f55c80c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55ce49c640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55c80cafb0 0x7f55c80c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55c80cafb0 0x7f55c80c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 -- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c80cc410 msgr2=0x7f55c80b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c80cc410 0x7f55c80b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.001+0000 7f55cec9d640 1 -- 172.21.15.84:0/1691628971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009400 con 0x7f55c815c290 2023-10-11T06:14:50.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.002+0000 7f55cec9d640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c815c290 0x7f55c80b1e40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f55c000c780 tx=0x7f55c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.002+0000 7f55beffd640 1 -- 172.21.15.84:0/1691628971 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0004b60 con 0x7f55c815c290 2023-10-11T06:14:50.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.002+0000 7f55beffd640 1 -- 172.21.15.84:0/1691628971 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c000ce90 con 0x7f55c815c290 2023-10-11T06:14:50.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.002+0000 7f55beffd640 1 -- 172.21.15.84:0/1691628971 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c000a850 con 0x7f55c815c290 2023-10-11T06:14:50.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.002+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c815c940 con 0x7f55c815c290 2023-10-11T06:14:50.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.002+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c80b2380 con 0x7f55c815c290 2023-10-11T06:14:50.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.003+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c8066fc0 con 0x7f55c815c290 2023-10-11T06:14:50.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.004+0000 7f55beffd640 1 -- 172.21.15.84:0/1691628971 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f55c000a9b0 con 0x7f55c815c290 2023-10-11T06:14:50.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.005+0000 7f55beffd640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f55ac06a830 0x7f55ac06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.005+0000 7f55beffd640 1 -- 172.21.15.84:0/1691628971 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f55c0088f80 con 0x7f55c815c290 2023-10-11T06:14:50.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.005+0000 7f55ce49c640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f55ac06a830 0x7f55ac06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.006+0000 7f55ce49c640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f55ac06a830 0x7f55ac06cce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f55c4001360 tx=0x7f55c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.009+0000 7f55beffd640 1 -- 172.21.15.84:0/1691628971 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f55c0054480 con 0x7f55c815c290 2023-10-11T06:14:50.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.200+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f55c80b2ae0 con 0x7f55ac06a830 2023-10-11T06:14:50.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.201+0000 7f55beffd640 1 -- 172.21.15.84:0/1691628971 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55c80b2ae0 con 0x7f55ac06a830 2023-10-11T06:14:50.203 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 5 to deep-scrub 2023-10-11T06:14:50.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f55ac06a830 msgr2=0x7f55ac06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f55ac06a830 0x7f55ac06cce0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f55c4001360 tx=0x7f55c4009040 comp rx=0 tx=0).stop 2023-10-11T06:14:50.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c815c290 msgr2=0x7f55c80b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c815c290 0x7f55c80b1e40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f55c000c780 tx=0x7f55c000cc40 comp rx=0 tx=0).stop 2023-10-11T06:14:50.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 shutdown_connections 2023-10-11T06:14:50.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f55ac06a830 0x7f55ac06cce0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55c815c290 0x7f55c80b1e40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c80cc410 0x7f55c80b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 --2- 172.21.15.84:0/1691628971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f55c80cafb0 0x7f55c80c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.203+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 >> 172.21.15.84:0/1691628971 conn(0x7f55c8069c70 msgr2=0x7f55c80ac680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:50.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.204+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 shutdown_connections 2023-10-11T06:14:50.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.204+0000 7f55d0700640 1 -- 172.21.15.84:0/1691628971 wait complete. 2023-10-11T06:14:50.225 INFO:tasks.ceph:Scrubbing osd.6 2023-10-11T06:14:50.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:50.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.439+0000 7ff62cc77640 1 Processor -- start 2023-10-11T06:14:50.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.440+0000 7ff62cc77640 1 -- start start 2023-10-11T06:14:50.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.441+0000 7ff62cc77640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6280d2ac0 0x7ff6280d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.441+0000 7ff62cc77640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280cf220 con 0x7ff6280d0940 2023-10-11T06:14:50.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.441+0000 7ff62cc77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280cf500 con 0x7ff62815f7e0 2023-10-11T06:14:50.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.441+0000 7ff62cc77640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6280d3480 con 0x7ff6280d2ac0 2023-10-11T06:14:50.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.441+0000 7ff626d9d640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff6280d0940 0x7ff6280d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60120/0 (socket says 172.21.15.84:60120) 2023-10-11T06:14:50.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff626d9d640 1 -- 172.21.15.84:0/488840534 learned_addr learned my addr 172.21.15.84:0/488840534 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:50.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff625d9b640 1 --2- 172.21.15.84:0/488840534 >> v2:172.21.15.99:3300/0 conn(0x7ff6280d2ac0 0x7ff6280d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980164832 0 0) 0x7ff6280cf220 con 0x7ff6280d0940 2023-10-11T06:14:50.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff614003520 con 0x7ff6280d0940 2023-10-11T06:14:50.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766057490 0 0) 0x7ff614003520 con 0x7ff6280d0940 2023-10-11T06:14:50.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 >> v2:172.21.15.99:3300/0 conn(0x7ff6280d2ac0 msgr2=0x7ff6280d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:50.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 --2- 172.21.15.84:0/488840534 >> v2:172.21.15.99:3300/0 conn(0x7ff6280d2ac0 0x7ff6280d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 >> v1:172.21.15.99:6789/0 conn(0x7ff62815f7e0 legacy=0x7ff6280cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280cf7e0 con 0x7ff6280d0940 2023-10-11T06:14:50.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.442+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 386800463 0 0) 0x7ff620001670 con 0x7ff6280d0940 2023-10-11T06:14:50.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 >> v1:172.21.15.84:6789/0 conn(0x7ff6280d0940 legacy=0x7ff6280d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62559a640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff614003970 0x7ff614003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62559a640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6140047c0 0x7ff614004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62559a640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff614005170 0x7ff61400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280d3480 con 0x7ff6140047c0 2023-10-11T06:14:50.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400bff0 con 0x7ff614005170 2023-10-11T06:14:50.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400c1b0 con 0x7ff614003970 2023-10-11T06:14:50.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff625d9b640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6140047c0 0x7ff614004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff62659c640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff614003970 0x7ff614003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff625d9b640 1 -- 172.21.15.84:0/488840534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff614003970 msgr2=0x7ff614003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:50.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff625d9b640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff614003970 0x7ff614003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff625d9b640 1 -- 172.21.15.84:0/488840534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff614005170 msgr2=0x7ff61400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff625d9b640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff614005170 0x7ff61400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff625d9b640 1 -- 172.21.15.84:0/488840534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614003520 con 0x7ff6140047c0 2023-10-11T06:14:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.443+0000 7ff626d9d640 1 -- 172.21.15.84:0/488840534 reap_dead start 2023-10-11T06:14:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.444+0000 7ff625d9b640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6140047c0 0x7ff614004c30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff610009ac0 tx=0x7ff6140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418beccb0c9f9d83 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002c6d0 con 0x7ff6140047c0 2023-10-11T06:14:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61002c830 con 0x7ff6140047c0 2023-10-11T06:14:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62559a640 1 -- 172.21.15.84:0/488840534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002cbc0 con 0x7ff6140047c0 2023-10-11T06:14:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62cc77640 1 -- 172.21.15.84:0/488840534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6140047c0 msgr2=0x7ff614004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62cc77640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6140047c0 0x7ff614004c30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff610009ac0 tx=0x7ff6140057f0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62cc77640 1 -- 172.21.15.84:0/488840534 shutdown_connections 2023-10-11T06:14:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62cc77640 1 --2- 172.21.15.84:0/488840534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6140047c0 0x7ff614004c30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.445+0000 7ff62cc77640 1 -- 172.21.15.84:0/488840534 >> 172.21.15.84:0/488840534 conn(0x7ff6280713a0 msgr2=0x7ff6280717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.446+0000 7ff62cc77640 1 -- 172.21.15.84:0/488840534 shutdown_connections 2023-10-11T06:14:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.446+0000 7ff62cc77640 1 -- 172.21.15.84:0/488840534 wait complete. 2023-10-11T06:14:50.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.447+0000 7ff62cc77640 1 Processor -- start 2023-10-11T06:14:50.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.448+0000 7ff62cc77640 1 -- start start 2023-10-11T06:14:50.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.448+0000 7ff62cc77640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6140047c0 0x7ff6280cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.448+0000 7ff62cc77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628154060 0x7ff62814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.448+0000 7ff62cc77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6280c8830 0x7ff6280c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.448+0000 7ff62cc77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280c91f0 con 0x7ff628154060 2023-10-11T06:14:50.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.448+0000 7ff62cc77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280cca30 con 0x7ff6280c8830 2023-10-11T06:14:50.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.448+0000 7ff62cc77640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280ccd10 con 0x7ff6140047c0 2023-10-11T06:14:50.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628154060 0x7ff62814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628154060 0x7ff62814ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41632/0 (socket says 172.21.15.84:41632) 2023-10-11T06:14:50.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 -- 172.21.15.84:0/2953924447 learned_addr learned my addr 172.21.15.84:0/2953924447 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:50.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 -- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6140047c0 msgr2=0x7ff6280cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6140047c0 0x7ff6280cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 -- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6280c8830 msgr2=0x7ff6280c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6280c8830 0x7ff6280c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 -- 172.21.15.84:0/2953924447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280c9c00 con 0x7ff628154060 2023-10-11T06:14:50.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.449+0000 7ff625d9b640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628154060 0x7ff62814ff30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff610009ac0 tx=0x7ff610004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.450+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610003810 con 0x7ff628154060 2023-10-11T06:14:50.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.450+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610003970 con 0x7ff628154060 2023-10-11T06:14:50.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.450+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610003d00 con 0x7ff628154060 2023-10-11T06:14:50.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.450+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6280c9ee0 con 0x7ff628154060 2023-10-11T06:14:50.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.450+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6280cc690 con 0x7ff628154060 2023-10-11T06:14:50.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.451+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6280cfda0 con 0x7ff628154060 2023-10-11T06:14:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.452+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff610040450 con 0x7ff628154060 2023-10-11T06:14:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.453+0000 7ff61effd640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff60806a860 0x7ff60806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.453+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff6100aac40 con 0x7ff628154060 2023-10-11T06:14:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.453+0000 7ff62659c640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff60806a860 0x7ff60806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.453+0000 7ff61effd640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7ff608070cd0 0x7ff6080730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.453+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 --> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff608073790 con 0x7ff608070cd0 2023-10-11T06:14:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.453+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6100af240 con 0x7ff628154060 2023-10-11T06:14:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.453+0000 7ff62659c640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff60806a860 0x7ff60806cd10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff618007fd0 tx=0x7ff618009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.454+0000 7ff626d9d640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7ff608070cd0 0x7ff6080730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.454+0000 7ff626d9d640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7ff608070cd0 0x7ff6080730e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.456+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== osd.6 v2:172.21.15.99:6820/88045 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff608073790 con 0x7ff608070cd0 2023-10-11T06:14:50.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.470+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 --> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff6280cfac0 con 0x7ff608070cd0 2023-10-11T06:14:50.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.471+0000 7ff61effd640 1 -- 172.21.15.84:0/2953924447 <== osd.6 v2:172.21.15.99:6820/88045 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff620001b20 con 0x7ff608070cd0 2023-10-11T06:14:50.474 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:50.474 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:50.474 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7ff608070cd0 msgr2=0x7ff6080730e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.99:6820/88045,v1:172.21.15.99:6822/88045] conn(0x7ff608070cd0 0x7ff6080730e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff60806a860 msgr2=0x7ff60806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff60806a860 0x7ff60806cd10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff618007fd0 tx=0x7ff618009040 comp rx=0 tx=0).stop 2023-10-11T06:14:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628154060 msgr2=0x7ff62814ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 --2- 172.21.15.84:0/2953924447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628154060 0x7ff62814ff30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff610009ac0 tx=0x7ff610004730 comp rx=0 tx=0).stop 2023-10-11T06:14:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff626d9d640 1 -- 172.21.15.84:0/2953924447 reap_dead start 2023-10-11T06:14:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 shutdown_connections 2023-10-11T06:14:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 >> 172.21.15.84:0/2953924447 conn(0x7ff6280713a0 msgr2=0x7ff6280d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 shutdown_connections 2023-10-11T06:14:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.472+0000 7ff62cc77640 1 -- 172.21.15.84:0/2953924447 wait complete. 2023-10-11T06:14:50.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-11T06:14:50.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.702+0000 7f51298a4640 1 Processor -- start 2023-10-11T06:14:50.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.702+0000 7f51298a4640 1 -- start start 2023-10-11T06:14:50.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f51298a4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51240ceb30 0x7f512415c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f51298a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51240caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f51298a4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512406b060 con 0x7f512415c4c0 2023-10-11T06:14:50.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f51298a4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51240d1490 con 0x7f51240ceb30 2023-10-11T06:14:50.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f51298a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51240d0810 con 0x7f512406c4b0 2023-10-11T06:14:50.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f5122ffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f512415c4c0 0x7f51240ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60136/0 (socket says 172.21.15.84:60136) 2023-10-11T06:14:50.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f5122ffd640 1 -- 172.21.15.84:0/3544540604 learned_addr learned my addr 172.21.15.84:0/3544540604 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:50.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f51227fc640 1 --2- 172.21.15.84:0/3544540604 >> v2:172.21.15.84:3300/0 conn(0x7f51240ceb30 0x7f512415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.704+0000 7f51237fe640 1 --2- 172.21.15.84:0/3544540604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51240caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.705+0000 7f51227fc640 1 -- 172.21.15.84:0/3544540604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 msgr2=0x7f51240caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:50.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.705+0000 7f51227fc640 1 --2- 172.21.15.84:0/3544540604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51240caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.705+0000 7f51227fc640 1 -- 172.21.15.84:0/3544540604 >> v1:172.21.15.84:6789/0 conn(0x7f512415c4c0 legacy=0x7f51240ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.705+0000 7f51227fc640 1 -- 172.21.15.84:0/3544540604 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240d0af0 con 0x7f51240ceb30 2023-10-11T06:14:50.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.705+0000 7f5121ffb640 1 -- 172.21.15.84:0/3544540604 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150399038 0 0) 0x7f512406b060 con 0x7f512415c4c0 2023-10-11T06:14:50.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.706+0000 7f51227fc640 1 --2- 172.21.15.84:0/3544540604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415c890 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f51180097a0 tx=0x7f511802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91f5db9d00b385 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.706+0000 7f5121ffb640 1 -- 172.21.15.84:0/3544540604 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511802e070 con 0x7f51240ceb30 2023-10-11T06:14:50.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.706+0000 7f5121ffb640 1 -- 172.21.15.84:0/3544540604 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180315a0 con 0x7f51240ceb30 2023-10-11T06:14:50.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.706+0000 7f5121ffb640 1 -- 172.21.15.84:0/3544540604 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118031930 con 0x7f51240ceb30 2023-10-11T06:14:50.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.706+0000 7f51298a4640 1 -- 172.21.15.84:0/3544540604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 msgr2=0x7f512415c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.706+0000 7f51298a4640 1 --2- 172.21.15.84:0/3544540604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415c890 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f51180097a0 tx=0x7f511802b840 comp rx=0 tx=0).stop 2023-10-11T06:14:50.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.707+0000 7f51298a4640 1 -- 172.21.15.84:0/3544540604 shutdown_connections 2023-10-11T06:14:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.707+0000 7f51298a4640 1 --2- 172.21.15.84:0/3544540604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51240caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.707+0000 7f51298a4640 1 --2- 172.21.15.84:0/3544540604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415c890 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.707+0000 7f51298a4640 1 -- 172.21.15.84:0/3544540604 >> 172.21.15.84:0/3544540604 conn(0x7f512406e2a0 msgr2=0x7f512406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:50.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.707+0000 7f51298a4640 1 -- 172.21.15.84:0/3544540604 shutdown_connections 2023-10-11T06:14:50.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.707+0000 7f51298a4640 1 -- 172.21.15.84:0/3544540604 wait complete. 2023-10-11T06:14:50.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.708+0000 7f51298a4640 1 Processor -- start 2023-10-11T06:14:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.709+0000 7f51298a4640 1 -- start start 2023-10-11T06:14:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.709+0000 7f51298a4640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51241f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.709+0000 7f51298a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.709+0000 7f51298a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512415c4c0 0x7f512415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.709+0000 7f51298a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51241f50f0 con 0x7f51240ceb30 2023-10-11T06:14:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.709+0000 7f51298a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240b3680 con 0x7f512415c4c0 2023-10-11T06:14:50.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51298a4640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51240b3960 con 0x7f512406c4b0 2023-10-11T06:14:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41648/0 (socket says 172.21.15.84:41648) 2023-10-11T06:14:50.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 -- 172.21.15.84:0/3928951098 learned_addr learned my addr 172.21.15.84:0/3928951098 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:50.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f5122ffd640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51241f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 -- 172.21.15.84:0/3928951098 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 msgr2=0x7f51241f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51241f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 -- 172.21.15.84:0/3928951098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512415c4c0 msgr2=0x7f512415aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512415c4c0 0x7f512415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 -- 172.21.15.84:0/3928951098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009400 con 0x7f51240ceb30 2023-10-11T06:14:50.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.710+0000 7f51227fc640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415a510 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5118034040 tx=0x7f51180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.711+0000 7f510bfff640 1 -- 172.21.15.84:0/3928951098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118032840 con 0x7f51240ceb30 2023-10-11T06:14:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.711+0000 7f510bfff640 1 -- 172.21.15.84:0/3928951098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180329c0 con 0x7f51240ceb30 2023-10-11T06:14:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.711+0000 7f510bfff640 1 -- 172.21.15.84:0/3928951098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118032d70 con 0x7f51240ceb30 2023-10-11T06:14:50.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.711+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512415f760 con 0x7f51240ceb30 2023-10-11T06:14:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.711+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512415fd20 con 0x7f51240ceb30 2023-10-11T06:14:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.712+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51240b3ad0 con 0x7f51240ceb30 2023-10-11T06:14:50.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.713+0000 7f510bfff640 1 -- 172.21.15.84:0/3928951098 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f511803b3f0 con 0x7f51240ceb30 2023-10-11T06:14:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.714+0000 7f510bfff640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f511006a830 0x7f511006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.714+0000 7f510bfff640 1 -- 172.21.15.84:0/3928951098 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f51180aad20 con 0x7f51240ceb30 2023-10-11T06:14:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.714+0000 7f5122ffd640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f511006a830 0x7f511006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:50.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.715+0000 7f5122ffd640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f511006a830 0x7f511006cce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5114000f60 tx=0x7f5114009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:50.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.718+0000 7f510bfff640 1 -- 172.21.15.84:0/3928951098 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f5118076220 con 0x7f51240ceb30 2023-10-11T06:14:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.903+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f51240d10b0 con 0x7f511006a830 2023-10-11T06:14:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.903+0000 7f510bfff640 1 -- 172.21.15.84:0/3928951098 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5114001c70 con 0x7f511006a830 2023-10-11T06:14:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 6 to deep-scrub 2023-10-11T06:14:50.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f511006a830 msgr2=0x7f511006cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f511006a830 0x7f511006cce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5114000f60 tx=0x7f5114009040 comp rx=0 tx=0).stop 2023-10-11T06:14:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 msgr2=0x7f512415a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415a510 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5118034040 tx=0x7f51180046c0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 shutdown_connections 2023-10-11T06:14:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f511006a830 0x7f511006cce0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512415c4c0 0x7f512415aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51240ceb30 0x7f512415a510 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 --2- 172.21.15.84:0/3928951098 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f512406c4b0 0x7f51241f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:50.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 >> 172.21.15.84:0/3928951098 conn(0x7f512406e2a0 msgr2=0x7f51241600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:50.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 shutdown_connections 2023-10-11T06:14:50.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:50.906+0000 7f51298a4640 1 -- 172.21.15.84:0/3928951098 wait complete. 2023-10-11T06:14:50.927 INFO:tasks.ceph:Scrubbing osd.7 2023-10-11T06:14:50.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-11T06:14:51.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.126+0000 7f7c85d1b640 1 Processor -- start 2023-10-11T06:14:51.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.127+0000 7f7c85d1b640 1 -- start start 2023-10-11T06:14:51.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.128+0000 7f7c85d1b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7c8015a0b0 0x7f7c800cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.128+0000 7f7c85d1b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800ced30 0x7f7c8006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.128+0000 7f7c85d1b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c80071380 con 0x7f7c800c8d00 2023-10-11T06:14:51.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.128+0000 7f7c85d1b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8006e0f0 con 0x7f7c8015a0b0 2023-10-11T06:14:51.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.128+0000 7f7c85d1b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800cbae0 con 0x7f7c800ced30 2023-10-11T06:14:51.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7c8015a0b0 0x7f7c800cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7effd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7c800c8d00 0x7f7c800ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60138/0 (socket says 172.21.15.84:60138) 2023-10-11T06:14:51.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7effd640 1 -- 172.21.15.84:0/665275260 learned_addr learned my addr 172.21.15.84:0/665275260 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:51.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7ffff640 1 --2- 172.21.15.84:0/665275260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800ced30 0x7f7c8006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7f7fe640 1 -- 172.21.15.84:0/665275260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800ced30 msgr2=0x7f7c8006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:51.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7f7fe640 1 --2- 172.21.15.84:0/665275260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800ced30 0x7f7c8006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7f7fe640 1 -- 172.21.15.84:0/665275260 >> v1:172.21.15.84:6789/0 conn(0x7f7c800c8d00 legacy=0x7f7c800ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7f7fe640 1 -- 172.21.15.84:0/665275260 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8006de10 con 0x7f7c8015a0b0 2023-10-11T06:14:51.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7ffff640 1 --2- 172.21.15.84:0/665275260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800ced30 0x7f7c8006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:51.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.129+0000 7f7c7e7fc640 1 -- 172.21.15.84:0/665275260 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933406327 0 0) 0x7f7c80071380 con 0x7f7c800c8d00 2023-10-11T06:14:51.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.130+0000 7f7c7f7fe640 1 --2- 172.21.15.84:0/665275260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c800cb5a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097d0 tx=0x7f7c7002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de420fad8d2206c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.130+0000 7f7c7e7fc640 1 -- 172.21.15.84:0/665275260 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700316f0 con 0x7f7c8015a0b0 2023-10-11T06:14:51.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.130+0000 7f7c7e7fc640 1 -- 172.21.15.84:0/665275260 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70031850 con 0x7f7c8015a0b0 2023-10-11T06:14:51.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.130+0000 7f7c7e7fc640 1 -- 172.21.15.84:0/665275260 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70031be0 con 0x7f7c8015a0b0 2023-10-11T06:14:51.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 -- 172.21.15.84:0/665275260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 msgr2=0x7f7c800cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 --2- 172.21.15.84:0/665275260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c800cb5a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097d0 tx=0x7f7c7002b790 comp rx=0 tx=0).stop 2023-10-11T06:14:51.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 -- 172.21.15.84:0/665275260 shutdown_connections 2023-10-11T06:14:51.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 --2- 172.21.15.84:0/665275260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800ced30 0x7f7c8006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 --2- 172.21.15.84:0/665275260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c800cb5a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 -- 172.21.15.84:0/665275260 >> 172.21.15.84:0/665275260 conn(0x7f7c80069c70 msgr2=0x7f7c8006a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:51.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 -- 172.21.15.84:0/665275260 shutdown_connections 2023-10-11T06:14:51.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.131+0000 7f7c85d1b640 1 -- 172.21.15.84:0/665275260 wait complete. 2023-10-11T06:14:51.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.133+0000 7f7c85d1b640 1 Processor -- start 2023-10-11T06:14:51.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.133+0000 7f7c85d1b640 1 -- start start 2023-10-11T06:14:51.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c85d1b640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800c8d00 0x7f7c800c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c85d1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c800ced30 0x7f7c8014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c85d1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c801474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c85d1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8014b130 con 0x7f7c8015a0b0 2023-10-11T06:14:51.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c85d1b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8014ac10 con 0x7f7c800ced30 2023-10-11T06:14:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c85d1b640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800c6050 con 0x7f7c800c8d00 2023-10-11T06:14:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c801474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c801474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41660/0 (socket says 172.21.15.84:41660) 2023-10-11T06:14:51.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 -- 172.21.15.84:0/2964035545 learned_addr learned my addr 172.21.15.84:0/2964035545 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7f7fe640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800c8d00 0x7f7c800c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7effd640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c800ced30 0x7f7c8014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 -- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800c8d00 msgr2=0x7f7c800c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c800c8d00 0x7f7c800c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 -- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c800ced30 msgr2=0x7f7c8014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c800ced30 0x7f7c8014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7ffff640 1 -- 172.21.15.84:0/2964035545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009400 con 0x7f7c8015a0b0 2023-10-11T06:14:51.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.134+0000 7f7c7effd640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c800ced30 0x7f7c8014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.135+0000 7f7c7ffff640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c801474f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800cef0 tx=0x7f7c7800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.135+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800d840 con 0x7f7c8015a0b0 2023-10-11T06:14:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.135+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7800d9c0 con 0x7f7c8015a0b0 2023-10-11T06:14:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.135+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7800dd70 con 0x7f7c8015a0b0 2023-10-11T06:14:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.135+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c80147a30 con 0x7f7c8015a0b0 2023-10-11T06:14:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.135+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8014ae50 con 0x7f7c8015a0b0 2023-10-11T06:14:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.136+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c800710a0 con 0x7f7c8015a0b0 2023-10-11T06:14:51.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.137+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f7c7801f440 con 0x7f7c8015a0b0 2023-10-11T06:14:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.138+0000 7f7c67fff640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c6c06a830 0x7f7c6c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.138+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7c78089800 con 0x7f7c8015a0b0 2023-10-11T06:14:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.138+0000 7f7c67fff640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7f7c6c070ca0 0x7f7c6c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.138+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 --> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c6c073760 con 0x7f7c6c070ca0 2023-10-11T06:14:51.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.138+0000 7f7c7f7fe640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c6c06a830 0x7f7c6c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.138+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c7808bde0 con 0x7f7c8015a0b0 2023-10-11T06:14:51.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.139+0000 7f7c7f7fe640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c6c06a830 0x7f7c6c06cce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7c70036000 tx=0x7f7c70008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.139+0000 7f7c7effd640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7f7c6c070ca0 0x7f7c6c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.140+0000 7f7c7effd640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7f7c6c070ca0 0x7f7c6c0730b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.142+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== osd.7 v2:172.21.15.99:6816/88032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7c6c073760 con 0x7f7c6c070ca0 2023-10-11T06:14:51.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.155+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 --> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7c80070dc0 con 0x7f7c6c070ca0 2023-10-11T06:14:51.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.156+0000 7f7c67fff640 1 -- 172.21.15.84:0/2964035545 <== osd.7 v2:172.21.15.99:6816/88032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7c740018d0 con 0x7f7c6c070ca0 2023-10-11T06:14:51.159 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-10-11T06:14:51.159 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-11T06:14:51.159 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-10-11T06:14:51.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7f7c6c070ca0 msgr2=0x7f7c6c0730b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.99:6816/88032,v1:172.21.15.99:6817/88032] conn(0x7f7c6c070ca0 0x7f7c6c0730b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c6c06a830 msgr2=0x7f7c6c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f7c6c06a830 0x7f7c6c06cce0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7c70036000 tx=0x7f7c70008030 comp rx=0 tx=0).stop 2023-10-11T06:14:51.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 msgr2=0x7f7c801474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 --2- 172.21.15.84:0/2964035545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c8015a0b0 0x7f7c801474f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800cef0 tx=0x7f7c7800a710 comp rx=0 tx=0).stop 2023-10-11T06:14:51.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c7ffff640 1 -- 172.21.15.84:0/2964035545 reap_dead start 2023-10-11T06:14:51.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 shutdown_connections 2023-10-11T06:14:51.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 >> 172.21.15.84:0/2964035545 conn(0x7f7c80069c70 msgr2=0x7f7c800c3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:51.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 shutdown_connections 2023-10-11T06:14:51.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.157+0000 7f7c85d1b640 1 -- 172.21.15.84:0/2964035545 wait complete. 2023-10-11T06:14:51.176 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-11T06:14:51.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.381+0000 7f2f9359e640 1 Processor -- start 2023-10-11T06:14:51.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.382+0000 7f2f9359e640 1 -- start start 2023-10-11T06:14:51.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.383+0000 7f2f9359e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2f9406c000 0x7f2f940c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.383+0000 7f2f9359e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.383+0000 7f2f9359e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f940adc40 con 0x7f2f9406c3d0 2023-10-11T06:14:51.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.383+0000 7f2f9359e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f940c97e0 con 0x7f2f9406c000 2023-10-11T06:14:51.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.383+0000 7f2f9359e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940c9ac0 con 0x7f2f94158e40 2023-10-11T06:14:51.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f9259c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2f9406c000 0x7f2f940c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f8bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2f9406c3d0 0x7f2f94158730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60146/0 (socket says 172.21.15.84:60146) 2023-10-11T06:14:51.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f8bfff640 1 -- 172.21.15.84:0/1173963458 learned_addr learned my addr 172.21.15.84:0/1173963458 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:14:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f92d9d640 1 --2- 172.21.15.84:0/1173963458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f9259c640 1 -- 172.21.15.84:0/1173963458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 msgr2=0x7f2f940c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f9259c640 1 --2- 172.21.15.84:0/1173963458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f9259c640 1 -- 172.21.15.84:0/1173963458 >> v1:172.21.15.84:6789/0 conn(0x7f2f9406c3d0 legacy=0x7f2f94158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f9259c640 1 -- 172.21.15.84:0/1173963458 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940c9da0 con 0x7f2f9406c000 2023-10-11T06:14:51.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f91d9b640 1 -- 172.21.15.84:0/1173963458 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244400592 0 0) 0x7f2f940adc40 con 0x7f2f9406c3d0 2023-10-11T06:14:51.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.384+0000 7f2f92d9d640 1 --2- 172.21.15.84:0/1173963458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:14:51.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.385+0000 7f2f9259c640 1 --2- 172.21.15.84:0/1173963458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c000 0x7f2f940c8c20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2f840097a0 tx=0x7f2f8402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d8c8bcdd6481d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.385+0000 7f2f91d9b640 1 -- 172.21.15.84:0/1173963458 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8402d040 con 0x7f2f9406c000 2023-10-11T06:14:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.385+0000 7f2f91d9b640 1 -- 172.21.15.84:0/1173963458 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f84031810 con 0x7f2f9406c000 2023-10-11T06:14:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.385+0000 7f2f91d9b640 1 -- 172.21.15.84:0/1173963458 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84031ba0 con 0x7f2f9406c000 2023-10-11T06:14:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 -- 172.21.15.84:0/1173963458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c000 msgr2=0x7f2f940c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 --2- 172.21.15.84:0/1173963458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c000 0x7f2f940c8c20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2f840097a0 tx=0x7f2f8402b8b0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 -- 172.21.15.84:0/1173963458 shutdown_connections 2023-10-11T06:14:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 --2- 172.21.15.84:0/1173963458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 --2- 172.21.15.84:0/1173963458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c000 0x7f2f940c8c20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 -- 172.21.15.84:0/1173963458 >> 172.21.15.84:0/1173963458 conn(0x7f2f94069c70 msgr2=0x7f2f9406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:51.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 -- 172.21.15.84:0/1173963458 shutdown_connections 2023-10-11T06:14:51.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.386+0000 7f2f9359e640 1 -- 172.21.15.84:0/1173963458 wait complete. 2023-10-11T06:14:51.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.388+0000 7f2f9359e640 1 Processor -- start 2023-10-11T06:14:51.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.388+0000 7f2f9359e640 1 -- start start 2023-10-11T06:14:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f9359e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f9406c000 0x7f2f941eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f9359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c3d0 0x7f2f94156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f9359e640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f9359e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940c5ee0 con 0x7f2f9406c3d0 2023-10-11T06:14:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f9359e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940c6960 con 0x7f2f9406c000 2023-10-11T06:14:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f9359e640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f941548d0 con 0x7f2f94158e40 2023-10-11T06:14:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c3d0 0x7f2f94156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c3d0 0x7f2f94156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41688/0 (socket says 172.21.15.84:41688) 2023-10-11T06:14:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.389+0000 7f2f8bfff640 1 -- 172.21.15.84:0/3517681527 learned_addr learned my addr 172.21.15.84:0/3517681527 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8bfff640 1 -- 172.21.15.84:0/3517681527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 msgr2=0x7f2f940c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8bfff640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8bfff640 1 -- 172.21.15.84:0/3517681527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f9406c000 msgr2=0x7f2f941eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8bfff640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f9406c000 0x7f2f941eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8bfff640 1 -- 172.21.15.84:0/3517681527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84009400 con 0x7f2f9406c3d0 2023-10-11T06:14:51.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8bfff640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c3d0 0x7f2f94156fa0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c4d0 tx=0x7f2f7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8affd640 1 -- 172.21.15.84:0/3517681527 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0074e0 con 0x7f2f9406c3d0 2023-10-11T06:14:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8affd640 1 -- 172.21.15.84:0/3517681527 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c007660 con 0x7f2f9406c3d0 2023-10-11T06:14:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f8affd640 1 -- 172.21.15.84:0/3517681527 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c004a20 con 0x7f2f9406c3d0 2023-10-11T06:14:51.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f940c60c0 con 0x7f2f9406c3d0 2023-10-11T06:14:51.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.390+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f940c6680 con 0x7f2f9406c3d0 2023-10-11T06:14:51.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.392+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9404e890 con 0x7f2f9406c3d0 2023-10-11T06:14:51.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.393+0000 7f2f8affd640 1 -- 172.21.15.84:0/3517681527 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f2f7c004b80 con 0x7f2f9406c3d0 2023-10-11T06:14:51.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.393+0000 7f2f8affd640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f7406a830 0x7f2f7406cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.394+0000 7f2f8affd640 1 -- 172.21.15.84:0/3517681527 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2f7c087df0 con 0x7f2f9406c3d0 2023-10-11T06:14:51.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.394+0000 7f2f9259c640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f7406a830 0x7f2f7406cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.394+0000 7f2f9259c640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f7406a830 0x7f2f7406cce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2f84036000 tx=0x7f2f84030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.398+0000 7f2f8affd640 1 -- 172.21.15.84:0/3517681527 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f2f7c053400 con 0x7f2f9406c3d0 2023-10-11T06:14:51.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.586+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f940ca080 con 0x7f2f7406a830 2023-10-11T06:14:51.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.587+0000 7f2f8affd640 1 -- 172.21.15.84:0/3517681527 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2f84031a00 con 0x7f2f7406a830 2023-10-11T06:14:51.590 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 7 to deep-scrub 2023-10-11T06:14:51.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f7406a830 msgr2=0x7f2f7406cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f7406a830 0x7f2f7406cce0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2f84036000 tx=0x7f2f84030000 comp rx=0 tx=0).stop 2023-10-11T06:14:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c3d0 msgr2=0x7f2f94156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c3d0 0x7f2f94156fa0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c4d0 tx=0x7f2f7c00c990 comp rx=0 tx=0).stop 2023-10-11T06:14:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 shutdown_connections 2023-10-11T06:14:51.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f2f7406a830 0x7f2f7406cce0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f94158e40 0x7f2f940c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f9406c3d0 0x7f2f94156fa0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 --2- 172.21.15.84:0/3517681527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f9406c000 0x7f2f941eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 >> 172.21.15.84:0/3517681527 conn(0x7f2f94069c70 msgr2=0x7f2f940cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 shutdown_connections 2023-10-11T06:14:51.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.590+0000 7f2f9359e640 1 -- 172.21.15.84:0/3517681527 wait complete. 2023-10-11T06:14:51.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:14:51.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.813+0000 7f048b2f7640 1 Processor -- start 2023-10-11T06:14:51.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.813+0000 7f048b2f7640 1 -- start start 2023-10-11T06:14:51.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.815+0000 7f048b2f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.815+0000 7f048b2f7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0484160850 0x7f04840b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.815+0000 7f048b2f7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048415c910 con 0x7f048415d4e0 2023-10-11T06:14:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.815+0000 7f048b2f7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0484160430 con 0x7f0484160850 2023-10-11T06:14:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.815+0000 7f048b2f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04840d0b00 con 0x7f04840cf330 2023-10-11T06:14:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.815+0000 7f0489894640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0484160850 0x7f04840b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.816+0000 7f0489894640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0484160850 0x7f04840b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41700/0 (socket says 172.21.15.84:41700) 2023-10-11T06:14:51.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.816+0000 7f0489894640 1 -- 172.21.15.84:0/4244639571 learned_addr learned my addr 172.21.15.84:0/4244639571 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:51.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.816+0000 7f0489093640 1 --2- 172.21.15.84:0/4244639571 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.816+0000 7f0489894640 1 -- 172.21.15.84:0/4244639571 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 msgr2=0x7f04840d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:51.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.816+0000 7f0489894640 1 --2- 172.21.15.84:0/4244639571 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.816+0000 7f0489894640 1 -- 172.21.15.84:0/4244639571 >> v1:172.21.15.99:6789/0 conn(0x7f048415d4e0 legacy=0x7f04840cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:14:51.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.816+0000 7f0489894640 1 -- 172.21.15.84:0/4244639571 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04840d0de0 con 0x7f0484160850 2023-10-11T06:14:51.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.817+0000 7f0489894640 1 --2- 172.21.15.84:0/4244639571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840b2280 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f047c009e50 tx=0x7f047c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc93cb04c3d4f7b server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.817+0000 7f047bfff640 1 -- 172.21.15.84:0/4244639571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c02cb00 con 0x7f0484160850 2023-10-11T06:14:51.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.817+0000 7f047bfff640 1 -- 172.21.15.84:0/4244639571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c02cc60 con 0x7f0484160850 2023-10-11T06:14:51.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.817+0000 7f047bfff640 1 -- 172.21.15.84:0/4244639571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0343f0 con 0x7f0484160850 2023-10-11T06:14:51.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 -- 172.21.15.84:0/4244639571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 msgr2=0x7f04840b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 --2- 172.21.15.84:0/4244639571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840b2280 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f047c009e50 tx=0x7f047c02bf60 comp rx=0 tx=0).stop 2023-10-11T06:14:51.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 -- 172.21.15.84:0/4244639571 shutdown_connections 2023-10-11T06:14:51.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 --2- 172.21.15.84:0/4244639571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840b2280 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 --2- 172.21.15.84:0/4244639571 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 -- 172.21.15.84:0/4244639571 >> 172.21.15.84:0/4244639571 conn(0x7f0484071cc0 msgr2=0x7f048409bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:51.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 -- 172.21.15.84:0/4244639571 shutdown_connections 2023-10-11T06:14:51.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.818+0000 7f048b2f7640 1 -- 172.21.15.84:0/4244639571 wait complete. 2023-10-11T06:14:51.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.820+0000 7f048b2f7640 1 Processor -- start 2023-10-11T06:14:51.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.820+0000 7f048b2f7640 1 -- start start 2023-10-11T06:14:51.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f048b2f7640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f048b2f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048415d4e0 0x7f04840c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f048b2f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f048b2f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0484158510 con 0x7f0484160850 2023-10-11T06:14:51.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f048b2f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04840b08e0 con 0x7f048415d4e0 2023-10-11T06:14:51.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f048b2f7640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04840c91e0 con 0x7f04840cf330 2023-10-11T06:14:51.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f0489894640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f0489894640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41702/0 (socket says 172.21.15.84:41702) 2023-10-11T06:14:51.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.821+0000 7f0489894640 1 -- 172.21.15.84:0/2868144963 learned_addr learned my addr 172.21.15.84:0/2868144963 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:14:51.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0489093640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0488892640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048415d4e0 0x7f04840c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0489894640 1 -- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 msgr2=0x7f04840af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0489894640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0489894640 1 -- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048415d4e0 msgr2=0x7f04840c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:51.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0489894640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048415d4e0 0x7f04840c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:51.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0489894640 1 -- 172.21.15.84:0/2868144963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c009b30 con 0x7f0484160850 2023-10-11T06:14:51.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0489894640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840c6f40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f047c036040 tx=0x7f047c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0479ffb640 1 -- 172.21.15.84:0/2868144963 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c040670 con 0x7f0484160850 2023-10-11T06:14:51.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0479ffb640 1 -- 172.21.15.84:0/2868144963 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c0407d0 con 0x7f0484160850 2023-10-11T06:14:51.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f0479ffb640 1 -- 172.21.15.84:0/2868144963 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c040b80 con 0x7f0484160850 2023-10-11T06:14:51.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04840afcd0 con 0x7f0484160850 2023-10-11T06:14:51.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.822+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04840b0290 con 0x7f0484160850 2023-10-11T06:14:51.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.824+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04840b0a70 con 0x7f0484160850 2023-10-11T06:14:51.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.825+0000 7f0479ffb640 1 -- 172.21.15.84:0/2868144963 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7f047c040ce0 con 0x7f0484160850 2023-10-11T06:14:51.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.825+0000 7f0479ffb640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f046806a830 0x7f046806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:14:51.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.825+0000 7f0479ffb640 1 -- 172.21.15.84:0/2868144963 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f047c0abde0 con 0x7f0484160850 2023-10-11T06:14:51.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.826+0000 7f0489093640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f046806a830 0x7f046806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:14:51.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.826+0000 7f0489093640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f046806a830 0x7f046806cce0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:14:51.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:51.830+0000 7f0479ffb640 1 -- 172.21.15.84:0/2868144963 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7f047c0773f0 con 0x7f0484160850 2023-10-11T06:14:52.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.018+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04840d10c0 con 0x7f046806a830 2023-10-11T06:14:52.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.026+0000 7f0479ffb640 1 -- 172.21.15.84:0/2868144963 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70758 (secure 0 0 0) 0x7f048000e4b0 con 0x7f046806a830 2023-10-11T06:14:52.029 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:14:52.029 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:14:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.028+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f046806a830 msgr2=0x7f046806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.028+0000 7f048b2f7640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f046806a830 0x7f046806cce0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480008040 comp rx=0 tx=0).stop 2023-10-11T06:14:52.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.028+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 msgr2=0x7f04840c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:14:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.028+0000 7f048b2f7640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840c6f40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f047c036040 tx=0x7f047c004a40 comp rx=0 tx=0).stop 2023-10-11T06:14:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 shutdown_connections 2023-10-11T06:14:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7f046806a830 0x7f046806cce0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:52.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0484160850 0x7f04840c6f40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:52.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048415d4e0 0x7f04840c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:52.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 --2- 172.21.15.84:0/2868144963 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04840cf330 0x7f04840af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:14:52.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 >> 172.21.15.84:0/2868144963 conn(0x7f0484071cc0 msgr2=0x7f04840b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:14:52.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 shutdown_connections 2023-10-11T06:14:52.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:14:52.029+0000 7f048b2f7640 1 -- 172.21.15.84:0/2868144963 wait complete. 2023-10-11T06:14:52.048 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2023-10-11T06:14:50.841333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":226248,"kb_used_data":6536,"kb_used_omap":0,"kb_used_meta":219648,"kb_avail":754748472,"statfs":{"total":773094113280,"available":772862435328,"internally_reserved":0,"allocated":6692864,"data_stored":3581032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":224919552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002611"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-11T06:14:40.010968+0000","last_change":"2023-10-11T06:11:08.135397+0000","last_active":"2023-10-11T06:14:40.010968+0000","last_peered":"2023-10-11T06:14:40.010968+0000","last_clean":"2023-10-11T06:14:40.010968+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:14:40.010968+0000","last_undegraded":"2023-10-11T06:14:40.010968+0000","last_fullsized":"2023-10-11T06:14:40.010968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:41.020797+0000","last_change":"2023-10-11T06:11:08.132096+0000","last_active":"2023-10-11T06:14:41.020797+0000","last_peered":"2023-10-11T06:14:41.020797+0000","last_clean":"2023-10-11T06:14:41.020797+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:14:41.020797+0000","last_undegraded":"2023-10-11T06:14:41.020797+0000","last_fullsized":"2023-10-11T06:14:41.020797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-11T06:14:41.196621+0000","last_change":"2023-10-11T06:11:08.136998+0000","last_active":"2023-10-11T06:14:41.196621+0000","last_peered":"2023-10-11T06:14:41.196621+0000","last_clean":"2023-10-11T06:14:41.196621+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:14:41.196621+0000","last_undegraded":"2023-10-11T06:14:41.196621+0000","last_fullsized":"2023-10-11T06:14:41.196621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T07:23:12.609934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:41.021269+0000","last_change":"2023-10-11T06:11:08.125494+0000","last_active":"2023-10-11T06:14:41.021269+0000","last_peered":"2023-10-11T06:14:41.021269+0000","last_clean":"2023-10-11T06:14:41.021269+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:14:41.021269+0000","last_undegraded":"2023-10-11T06:14:41.021269+0000","last_fullsized":"2023-10-11T06:14:41.021269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-11T06:14:40.668017+0000","last_change":"2023-10-11T06:11:08.129564+0000","last_active":"2023-10-11T06:14:40.668017+0000","last_peered":"2023-10-11T06:14:40.668017+0000","last_clean":"2023-10-11T06:14:40.668017+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:14:40.668017+0000","last_undegraded":"2023-10-11T06:14:40.668017+0000","last_fullsized":"2023-10-11T06:14:40.668017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:08:40.484176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:48.755868+0000","last_change":"2023-10-11T06:14:48.756109+0000","last_active":"2023-10-11T06:14:48.755868+0000","last_peered":"2023-10-11T06:14:48.755868+0000","last_clean":"2023-10-11T06:14:48.755868+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:14:48.755868+0000","last_undegraded":"2023-10-11T06:14:48.755868+0000","last_fullsized":"2023-10-11T06:14:48.755868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:48.755823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:48.755823+0000","last_clean_scrub_stamp":"2023-10-11T06:14:48.755823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:36.263200+0000","scrub_duration":0.0029438200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-11T06:14:41.193253+0000","last_change":"2023-10-11T06:11:08.132272+0000","last_active":"2023-10-11T06:14:41.193253+0000","last_peered":"2023-10-11T06:14:41.193253+0000","last_clean":"2023-10-11T06:14:41.193253+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:14:41.193253+0000","last_undegraded":"2023-10-11T06:14:41.193253+0000","last_fullsized":"2023-10-11T06:14:41.193253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:40:28.409733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-11T06:14:40.668788+0000","last_change":"2023-10-11T06:11:08.129344+0000","last_active":"2023-10-11T06:14:40.668788+0000","last_peered":"2023-10-11T06:14:40.668788+0000","last_clean":"2023-10-11T06:14:40.668788+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:14:40.668788+0000","last_undegraded":"2023-10-11T06:14:40.668788+0000","last_fullsized":"2023-10-11T06:14:40.668788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:07.093804+0000","last_clean_scrub_stamp":"2023-10-11T06:11:07.093804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T11:51:23.334715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-11T06:14:41.193494+0000","last_change":"2023-10-11T06:11:02.694575+0000","last_active":"2023-10-11T06:14:41.193494+0000","last_peered":"2023-10-11T06:14:41.193494+0000","last_clean":"2023-10-11T06:14:41.193494+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:14:41.193494+0000","last_undegraded":"2023-10-11T06:14:41.193494+0000","last_fullsized":"2023-10-11T06:14:41.193494+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:11:01.640186+0000","last_clean_scrub_stamp":"2023-10-11T06:11:01.640186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:27:15.783756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1282048,"data_stored":890405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.72599999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.4379999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.171,"15min":1.161},"min":{"1min":0.78700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5569999999999999,"5min":1.788,"15min":1.788},"last":1.3100000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.083,"15min":1.099},"min":{"1min":0.72499999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.383,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.876},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.091,"15min":1.1180000000000001},"min":{"1min":0.63400000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.256}]},{"osd":2,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.1040000000000001,"15min":1.1220000000000001},"min":{"1min":0.745,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4890000000000001,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.26,"15min":1.284},"min":{"1min":0.92000000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.427,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.1399999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.206,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.575,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.2350000000000001,"15min":1.256},"min":{"1min":0.83099999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.637,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.3580000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.371,"15min":1.4059999999999999},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.645,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.581},{"interface":"front","average":{"1min":1.27,"5min":1.3080000000000001,"15min":1.3},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.7250000000000001,"5min":1.913,"15min":1.913},"last":1.4330000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3759999999999999,"15min":1.3899999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6870000000000001,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.397},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.53,"15min":1.5760000000000001},"min":{"1min":0.86199999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.8240000000000001,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.4710000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4690000000000001,"15min":1.4730000000000001},"min":{"1min":0.747,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7929999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.5269999999999999,"15min":1.5529999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7829999999999999,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.619}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.88700000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.298},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2130000000000001,"15min":1.216},"min":{"1min":0.82899999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.8,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.1839999999999999}]},{"osd":1,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.038,"15min":1.038},"min":{"1min":0.69399999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.617,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.115},{"interface":"front","average":{"1min":1.167,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.76000000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.645,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.72799999999999998}]},{"osd":2,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0680000000000001},"min":{"1min":0.78000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.21,"15min":1.2150000000000001},"min":{"1min":0.98999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5369999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.4019999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.874,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.542,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":1.016},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.236,"15min":1.2490000000000001},"min":{"1min":0.70999999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6339999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.365}]},{"osd":4,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3400000000000001,"15min":1.373},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.667,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.681,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.345}]},{"osd":5,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.359,"15min":1.387},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.722,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.246},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4570000000000001,"15min":1.4670000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.748,"5min":1.992,"15min":1.992},"last":1.464}]},{"osd":7,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5609999999999999,"15min":1.589},"min":{"1min":0.81899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0979999999999999,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.51,"15min":1.538},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.792,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.4279999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4590000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.032,"15min":1.022},"min":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4279999999999999,"5min":1.621,"15min":1.621},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.046,"15min":1.056},"min":{"1min":0.69899999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.052,"15min":1.0529999999999999},"min":{"1min":0.76500000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4259999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.214,"15min":1.24},"min":{"1min":0.79000000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.49,"5min":2.508,"15min":2.508},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.161,"5min":1.2869999999999999,"15min":1.345},"min":{"1min":0.76900000000000002,"5min":0.752,"15min":0.752},"max":{"1min":1.7689999999999999,"5min":2.524,"15min":2.524},"last":1.1359999999999999}]},{"osd":3,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.82599999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5629999999999999,"5min":2.052,"15min":2.052},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.129,"5min":1.2470000000000001,"15min":1.2969999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.877,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.0309999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2669999999999999,"15min":1.2829999999999999},"min":{"1min":0.746,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.516,"5min":2.242,"15min":2.242},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.84899999999999998,"5min":0.747,"15min":0.747},"max":{"1min":1.6619999999999999,"5min":2.427,"15min":2.427},"last":1.395}]},{"osd":6,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3720000000000001,"15min":1.4219999999999999},"min":{"1min":0.81599999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.96,"5min":2.472,"15min":2.472},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4399999999999999,"15min":1.456},"min":{"1min":1.1599999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9770000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.4390000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:14:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4179999999999999,"15min":1.4530000000000001},"min":{"1min":0.876,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.736,"5min":2.617,"15min":2.617},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.534,"15min":1.5740000000000001},"min":{"1min":1.0249999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.04,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.2370000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.052,"15min":1.0740000000000001},"min":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.3979999999999999,"5min":1.841,"15min":1.841},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0009999999999999,"15min":0.96999999999999997},"min":{"1min":0.73499999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.98899999999999999}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.105,"15min":1.087},"min":{"1min":0.77600000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1040000000000001,"15min":1.127},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.282}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0589999999999999,"15min":1.0569999999999999},"min":{"1min":0.87,"5min":0.63,"15min":0.63},"max":{"1min":1.444,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.458},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1930000000000001,"15min":1.1970000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.107}]},{"osd":3,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.228,"15min":1.2569999999999999},"min":{"1min":0.79800000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.641,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.379},{"interface":"front","average":{"1min":1.165,"5min":1.194,"15min":1.2110000000000001},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.4390000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.0549999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3080000000000001,"15min":1.319},"min":{"1min":0.80400000000000005,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":1.625},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.371,"15min":1.407},"min":{"1min":0.98699999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.746,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.5640000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3220000000000001,"15min":1.339},"min":{"1min":0.97099999999999997,"5min":0.627,"15min":0.627},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4590000000000001},"min":{"1min":1.0529999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.6879999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5,"15min":1.548},"min":{"1min":1.0860000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.8400000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.716},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4610000000000001,"15min":1.4930000000000001},"min":{"1min":1.2390000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.2919999999999998,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.597}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.875,"15min":0.90100000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4910000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.431},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.93899999999999995,"15min":0.96199999999999997},"min":{"1min":0.47699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3420000000000001,"5min":2.097,"15min":2.097},"last":0.97399999999999998}]},{"osd":1,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.399,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.052,"15min":1.0960000000000001},"min":{"1min":0.53400000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3129999999999999,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.3740000000000001}]},{"osd":2,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.99199999999999999,"15min":1.0329999999999999},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.321,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.879,"5min":1.1160000000000001,"15min":1.1839999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.417,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":1.2789999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.222},"min":{"1min":0.79200000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.456,"5min":3.286,"15min":3.286},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.84999999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.629,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":1.204}]},{"osd":5,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3700000000000001,"15min":1.4119999999999999},"min":{"1min":0.87,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.792,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.2949999999999999,"15min":1.3320000000000001},"min":{"1min":0.78900000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.633,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.2310000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.357,"15min":1.3819999999999999},"min":{"1min":0.84799999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8400000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.468,"15min":1.552},"min":{"1min":0.80800000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8999999999999999,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":1.4790000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:14:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.3300000000000001,"15min":1.3740000000000001},"min":{"1min":0.79700000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.554,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3959999999999999,"15min":1.4119999999999999},"min":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6950000000000001,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.258}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79500000000000004,"15min":0.82699999999999996},"min":{"1min":0.373,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2649999999999999,"5min":1.494,"15min":1.494},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78100000000000003,"15min":0.79400000000000004},"min":{"1min":0.53000000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.86499999999999999},"min":{"1min":0.498,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.173,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.44700000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3420000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.89100000000000001,"15min":0.91900000000000004},"min":{"1min":0.47199999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.6759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.127,"15min":1.216},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3720000000000001,"5min":3.032,"15min":3.032},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2490000000000001,"15min":1.288},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.5029999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.079},{"interface":"front","average":{"1min":1.181,"5min":1.23,"15min":1.2589999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6000000000000001,"5min":3.069,"15min":3.069},"last":0.92000000000000004}]},{"osd":5,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.216,"15min":1.242},"min":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.5629999999999999,"5min":2.181,"15min":2.181},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.228,"5min":1.3240000000000001,"15min":1.385},"min":{"1min":0.84299999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6950000000000001,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3340000000000001,"15min":1.361},"min":{"1min":0.81299999999999994,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.913,"5min":2.633,"15min":2.633},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.302,"5min":1.2829999999999999,"15min":1.2909999999999999},"min":{"1min":0.72599999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.6719999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.252}]},{"osd":7,"last update":"Wed Oct 11 06:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.367,"15min":1.415},"min":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7430000000000001,"5min":2.794,"15min":2.794},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3120000000000001,"15min":1.335},"min":{"1min":0.95199999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5960000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.98499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.131,"5min":1.302,"15min":1.302},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.86199999999999999},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2190000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.498,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4530000000000001,"5min":1.768,"15min":1.768},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0009999999999999},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5960000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4990000000000001,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.64200000000000002}]},{"osd":4,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2609999999999999,"15min":1.2869999999999999},"min":{"1min":0.88,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8580000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.272,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.873,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0259999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.98099999999999998}]},{"osd":5,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2609999999999999,"15min":1.2929999999999999},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7689999999999999,"5min":3.319,"15min":3.319},"last":1.012},{"interface":"front","average":{"1min":1.224,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.82499999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.9930000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.292,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.931,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.284,"15min":1.2989999999999999},"min":{"1min":0.76200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7070000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.86899999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3600000000000001,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.575,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.401,"15min":1.425},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.3460000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.0900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1282048,"data_stored":890405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.46200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.98799999999999999,"15min":1.0600000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.329,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.874}]},{"osd":2,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.89000000000000001,"15min":0.90200000000000002},"min":{"1min":0.443,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3300000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.49099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.278,"5min":2.444,"15min":2.444},"last":0.65400000000000003}]},{"osd":3,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.84099999999999997,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.2010000000000001,"5min":1.474,"15min":1.474},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.083,"15min":1.1200000000000001},"min":{"1min":0.52600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.514,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.80100000000000005}]},{"osd":4,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.2969999999999999,"15min":1.3300000000000001},"min":{"1min":0.91300000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.55,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.262,"15min":1.2889999999999999},"min":{"1min":0.85899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5089999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.381}]},{"osd":5,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5820000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.266,"15min":1.29},"min":{"1min":0.78600000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5800000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.0860000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.327,"15min":1.3580000000000001},"min":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.605,"5min":2.552,"15min":2.552},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.325,"15min":1.373},"min":{"1min":0.874,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.716,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.3160000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:14:03 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.341,"15min":1.3879999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6559999999999999,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3560000000000001},"min":{"1min":0.85899999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.722,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.343}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:14:52.052 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-11T06:11:07.093804+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.053 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-11T06:11:07.093804+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.053 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-11T06:11:07.093804+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.054 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-11T06:11:07.093804+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.054 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-11T06:11:07.093804+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.054 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-11T06:11:07.093804+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.055 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-11T06:11:07.093804+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=7, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.055 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-11T06:11:01.640186+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=11, tm_sec=1, tm_wday=2, tm_yday=284, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=11, tm_hour=6, tm_min=14, tm_sec=44, tm_wday=2, tm_yday=284, tm_isdst=0) 2023-10-11T06:14:52.055 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-11T06:14:57.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:14:57.562 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:15:12.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-11T06:15:12.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.281+0000 7fee4bfff640 1 Processor -- start 2023-10-11T06:15:12.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.281+0000 7fee4bfff640 1 -- start start 2023-10-11T06:15:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.283+0000 7fee4bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee4c0ceae0 0x7fee4c15a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.283+0000 7fee4bfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c15da50 con 0x7fee4c0b3e70 2023-10-11T06:15:12.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.283+0000 7fee4bfff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c0cad10 con 0x7fee4c06fd80 2023-10-11T06:15:12.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.283+0000 7fee4bfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c15dca0 con 0x7fee4c0ceae0 2023-10-11T06:15:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.283+0000 7fee43fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee4c0ceae0 0x7fee4c15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.283+0000 7fee4affd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fee4c0b3e70 0x7fee4c0ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33404/0 (socket says 172.21.15.84:33404) 2023-10-11T06:15:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.283+0000 7fee4affd640 1 -- 172.21.15.84:0/1889875968 learned_addr learned my addr 172.21.15.84:0/1889875968 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:15:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.284+0000 7fee43fff640 1 -- 172.21.15.84:0/1889875968 >> v1:172.21.15.99:6789/0 conn(0x7fee4c06fd80 legacy=0x7fee4c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:15:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.284+0000 7fee43fff640 1 -- 172.21.15.84:0/1889875968 >> v1:172.21.15.84:6789/0 conn(0x7fee4c0b3e70 legacy=0x7fee4c0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.284+0000 7fee4a7fc640 1 -- 172.21.15.84:0/1889875968 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584240333 0 0) 0x7fee4c15da50 con 0x7fee4c0b3e70 2023-10-11T06:15:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.284+0000 7fee43fff640 1 -- 172.21.15.84:0/1889875968 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c15df80 con 0x7fee4c0ceae0 2023-10-11T06:15:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.285+0000 7fee43fff640 1 --2- 172.21.15.84:0/1889875968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0ceae0 0x7fee4c15a470 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2fb6354e0b70fa server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.285+0000 7fee4a7fc640 1 -- 172.21.15.84:0/1889875968 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34031830 con 0x7fee4c0ceae0 2023-10-11T06:15:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.285+0000 7fee4a7fc640 1 -- 172.21.15.84:0/1889875968 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34031990 con 0x7fee4c0ceae0 2023-10-11T06:15:12.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.285+0000 7fee4a7fc640 1 -- 172.21.15.84:0/1889875968 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34031d20 con 0x7fee4c0ceae0 2023-10-11T06:15:12.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.286+0000 7fee4bfff640 1 -- 172.21.15.84:0/1889875968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0ceae0 msgr2=0x7fee4c15a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.286+0000 7fee4bfff640 1 --2- 172.21.15.84:0/1889875968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0ceae0 0x7fee4c15a470 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee3402b8d0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.286+0000 7fee4bfff640 1 -- 172.21.15.84:0/1889875968 shutdown_connections 2023-10-11T06:15:12.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.286+0000 7fee4bfff640 1 --2- 172.21.15.84:0/1889875968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0ceae0 0x7fee4c15a470 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.286+0000 7fee4bfff640 1 -- 172.21.15.84:0/1889875968 >> 172.21.15.84:0/1889875968 conn(0x7fee4c069c70 msgr2=0x7fee4c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:15:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.286+0000 7fee4bfff640 1 -- 172.21.15.84:0/1889875968 shutdown_connections 2023-10-11T06:15:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.286+0000 7fee4bfff640 1 -- 172.21.15.84:0/1889875968 wait complete. 2023-10-11T06:15:12.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.287+0000 7fee4bfff640 1 Processor -- start 2023-10-11T06:15:12.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.288+0000 7fee4bfff640 1 -- start start 2023-10-11T06:15:12.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4bfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee4c06fd80 0x7fee4c1f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0b3e70 0x7fee4c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee4c0ceae0 0x7fee4c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c147200 con 0x7fee4c0b3e70 2023-10-11T06:15:12.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4bfff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0afd80 con 0x7fee4c0ceae0 2023-10-11T06:15:12.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4bfff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c146680 con 0x7fee4c06fd80 2023-10-11T06:15:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0b3e70 0x7fee4c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0b3e70 0x7fee4c15bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53974/0 (socket says 172.21.15.84:53974) 2023-10-11T06:15:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 -- 172.21.15.84:0/2953242003 learned_addr learned my addr 172.21.15.84:0/2953242003 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:15:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4b7fe640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee4c0ceae0 0x7fee4c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee4affd640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee4c06fd80 0x7fee4c1f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 -- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee4c06fd80 msgr2=0x7fee4c1f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee4c06fd80 0x7fee4c1f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 -- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee4c0ceae0 msgr2=0x7fee4c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee4c0ceae0 0x7fee4c158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.289+0000 7fee43fff640 1 -- 172.21.15.84:0/2953242003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34009400 con 0x7fee4c0b3e70 2023-10-11T06:15:12.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee4affd640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee4c06fd80 0x7fee4c1f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:15:12.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee4b7fe640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee4c0ceae0 0x7fee4c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-11T06:15:12.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee43fff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0b3e70 0x7fee4c15bdd0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:12.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee48ff9640 1 -- 172.21.15.84:0/2953242003 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340317d0 con 0x7fee4c0b3e70 2023-10-11T06:15:12.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee48ff9640 1 -- 172.21.15.84:0/2953242003 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34004030 con 0x7fee4c0b3e70 2023-10-11T06:15:12.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c146960 con 0x7fee4c0b3e70 2023-10-11T06:15:12.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee48ff9640 1 -- 172.21.15.84:0/2953242003 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340043e0 con 0x7fee4c0b3e70 2023-10-11T06:15:12.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.290+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c146f20 con 0x7fee4c0b3e70 2023-10-11T06:15:12.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.291+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c0aff40 con 0x7fee4c0b3e70 2023-10-11T06:15:12.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.292+0000 7fee48ff9640 1 -- 172.21.15.84:0/2953242003 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fee34043070 con 0x7fee4c0b3e70 2023-10-11T06:15:12.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.294+0000 7fee48ff9640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee2c06a830 0x7fee2c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.294+0000 7fee48ff9640 1 -- 172.21.15.84:0/2953242003 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fee340aaa90 con 0x7fee4c0b3e70 2023-10-11T06:15:12.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.294+0000 7fee4affd640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee2c06a830 0x7fee2c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.295+0000 7fee4affd640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee2c06a830 0x7fee2c06cce0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fee3c000f60 tx=0x7fee3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:12.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.299+0000 7fee48ff9640 1 -- 172.21.15.84:0/2953242003 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fee34076020 con 0x7fee4c0b3e70 2023-10-11T06:15:12.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.488+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 --> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee4c0cb2b0 con 0x7fee2c06a830 2023-10-11T06:15:12.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.491+0000 7fee48ff9640 1 -- 172.21.15.84:0/2953242003 <== mgr.4103 v2:172.21.15.84:6800/87595 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70694 (secure 0 0 0) 0x7fee3c0013d0 con 0x7fee2c06a830 2023-10-11T06:15:12.493 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:15:12.493 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-10-11T06:15:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee2c06a830 msgr2=0x7fee2c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee2c06a830 0x7fee2c06cce0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fee3c000f60 tx=0x7fee3c009040 comp rx=0 tx=0).stop 2023-10-11T06:15:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0b3e70 msgr2=0x7fee4c15bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0b3e70 0x7fee4c15bdd0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee340046c0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 shutdown_connections 2023-10-11T06:15:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fee2c06a830 0x7fee2c06cce0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee4c0ceae0 0x7fee4c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4c0b3e70 0x7fee4c15bdd0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 --2- 172.21.15.84:0/2953242003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee4c06fd80 0x7fee4c1f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.493+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 >> 172.21.15.84:0/2953242003 conn(0x7fee4c069c70 msgr2=0x7fee4c0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:15:12.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.494+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 shutdown_connections 2023-10-11T06:15:12.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.494+0000 7fee4bfff640 1 -- 172.21.15.84:0/2953242003 wait complete. 2023-10-11T06:15:12.513 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2023-10-11T06:15:10.847141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":230344,"kb_used_data":6536,"kb_used_omap":0,"kb_used_meta":223744,"kb_avail":754744376,"statfs":{"total":773094113280,"available":772858241024,"internally_reserved":0,"allocated":6692864,"data_stored":3581032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":229113856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003522"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":59,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:51.666264+0000","last_change":"2023-10-11T06:14:51.666506+0000","last_active":"2023-10-11T06:14:51.666264+0000","last_peered":"2023-10-11T06:14:51.666264+0000","last_clean":"2023-10-11T06:14:51.666264+0000","last_became_active":"2023-10-11T06:11:08.133742+0000","last_became_peered":"2023-10-11T06:11:08.133742+0000","last_unstale":"2023-10-11T06:14:51.666264+0000","last_undegraded":"2023-10-11T06:14:51.666264+0000","last_fullsized":"2023-10-11T06:14:51.666264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:51.666219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:51.666219+0000","last_clean_scrub_stamp":"2023-10-11T06:14:51.666219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T08:38:51.593932+0000","scrub_duration":0.0039737160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:50.181150+0000","last_change":"2023-10-11T06:14:50.181280+0000","last_active":"2023-10-11T06:14:50.181150+0000","last_peered":"2023-10-11T06:14:50.181150+0000","last_clean":"2023-10-11T06:14:50.181150+0000","last_became_active":"2023-10-11T06:11:08.131558+0000","last_became_peered":"2023-10-11T06:11:08.131558+0000","last_unstale":"2023-10-11T06:14:50.181150+0000","last_undegraded":"2023-10-11T06:14:50.181150+0000","last_fullsized":"2023-10-11T06:14:50.181150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:50.181115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:50.181115+0000","last_clean_scrub_stamp":"2023-10-11T06:14:50.181115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T10:08:59.507894+0000","scrub_duration":0.0031002600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:15:00.875217+0000","last_change":"2023-10-11T06:15:00.875434+0000","last_active":"2023-10-11T06:15:00.875217+0000","last_peered":"2023-10-11T06:15:00.875217+0000","last_clean":"2023-10-11T06:15:00.875217+0000","last_became_active":"2023-10-11T06:11:08.135403+0000","last_became_peered":"2023-10-11T06:11:08.135403+0000","last_unstale":"2023-10-11T06:15:00.875217+0000","last_undegraded":"2023-10-11T06:15:00.875217+0000","last_fullsized":"2023-10-11T06:15:00.875217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:15:00.875170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:15:00.875170+0000","last_clean_scrub_stamp":"2023-10-11T06:15:00.875170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:41:37.440067+0000","scrub_duration":0.0041149100000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:56.100196+0000","last_change":"2023-10-11T06:14:56.100414+0000","last_active":"2023-10-11T06:14:56.100196+0000","last_peered":"2023-10-11T06:14:56.100196+0000","last_clean":"2023-10-11T06:14:56.100196+0000","last_became_active":"2023-10-11T06:11:08.125137+0000","last_became_peered":"2023-10-11T06:11:08.125137+0000","last_unstale":"2023-10-11T06:14:56.100196+0000","last_undegraded":"2023-10-11T06:14:56.100196+0000","last_fullsized":"2023-10-11T06:14:56.100196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:56.100151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:56.100151+0000","last_clean_scrub_stamp":"2023-10-11T06:14:56.100151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T06:51:21.755124+0000","scrub_duration":0.0031548489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":55,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:52.707485+0000","last_change":"2023-10-11T06:14:52.707708+0000","last_active":"2023-10-11T06:14:52.707485+0000","last_peered":"2023-10-11T06:14:52.707485+0000","last_clean":"2023-10-11T06:14:52.707485+0000","last_became_active":"2023-10-11T06:11:08.128699+0000","last_became_peered":"2023-10-11T06:11:08.128699+0000","last_unstale":"2023-10-11T06:14:52.707485+0000","last_undegraded":"2023-10-11T06:14:52.707485+0000","last_fullsized":"2023-10-11T06:14:52.707485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:52.707434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:52.707434+0000","last_clean_scrub_stamp":"2023-10-11T06:14:52.707434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:00:01.919574+0000","scrub_duration":0.003697049,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:48.755868+0000","last_change":"2023-10-11T06:14:48.756109+0000","last_active":"2023-10-11T06:14:48.755868+0000","last_peered":"2023-10-11T06:14:48.755868+0000","last_clean":"2023-10-11T06:14:48.755868+0000","last_became_active":"2023-10-11T06:11:08.323097+0000","last_became_peered":"2023-10-11T06:11:08.323097+0000","last_unstale":"2023-10-11T06:14:48.755868+0000","last_undegraded":"2023-10-11T06:14:48.755868+0000","last_fullsized":"2023-10-11T06:14:48.755868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:48.755823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:48.755823+0000","last_clean_scrub_stamp":"2023-10-11T06:14:48.755823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T15:22:36.263200+0000","scrub_duration":0.0029438200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:55.967268+0000","last_change":"2023-10-11T06:14:55.967268+0000","last_active":"2023-10-11T06:14:55.967268+0000","last_peered":"2023-10-11T06:14:55.967268+0000","last_clean":"2023-10-11T06:14:55.967268+0000","last_became_active":"2023-10-11T06:11:08.131585+0000","last_became_peered":"2023-10-11T06:11:08.131585+0000","last_unstale":"2023-10-11T06:14:55.967268+0000","last_undegraded":"2023-10-11T06:14:55.967268+0000","last_fullsized":"2023-10-11T06:14:55.967268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:55.967073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:55.967073+0000","last_clean_scrub_stamp":"2023-10-11T06:14:55.967073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T14:00:05.179213+0000","scrub_duration":0.0039819199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":55,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:56.810105+0000","last_change":"2023-10-11T06:14:56.810318+0000","last_active":"2023-10-11T06:14:56.810105+0000","last_peered":"2023-10-11T06:14:56.810105+0000","last_clean":"2023-10-11T06:14:56.810105+0000","last_became_active":"2023-10-11T06:11:08.128175+0000","last_became_peered":"2023-10-11T06:11:08.128175+0000","last_unstale":"2023-10-11T06:14:56.810105+0000","last_undegraded":"2023-10-11T06:14:56.810105+0000","last_fullsized":"2023-10-11T06:14:56.810105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-11T06:14:56.810060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-11T06:14:56.810060+0000","last_clean_scrub_stamp":"2023-10-11T06:14:56.810060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T12:41:33.374957+0000","scrub_duration":0.0033682730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"57'119","reported_seq":210,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-11T06:14:54.932881+0000","last_change":"2023-10-11T06:14:54.932881+0000","last_active":"2023-10-11T06:14:54.932881+0000","last_peered":"2023-10-11T06:14:54.932881+0000","last_clean":"2023-10-11T06:14:54.932881+0000","last_became_active":"2023-10-11T06:11:02.693624+0000","last_became_peered":"2023-10-11T06:11:02.693624+0000","last_unstale":"2023-10-11T06:14:54.932881+0000","last_undegraded":"2023-10-11T06:14:54.932881+0000","last_fullsized":"2023-10-11T06:14:54.932881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'119","last_scrub_stamp":"2023-10-11T06:14:54.932565+0000","last_deep_scrub":"57'119","last_deep_scrub_stamp":"2023-10-11T06:14:54.932565+0000","last_clean_scrub_stamp":"2023-10-11T06:14:54.932565+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-12T09:13:54.636417+0000","scrub_duration":0.012174356000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1282048,"data_stored":890405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.0880000000000001,"15min":1.05},"min":{"1min":0.82299999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.6679999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.1899999999999999,"15min":1.1679999999999999},"min":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.532,"5min":1.788,"15min":1.788},"last":1.484}]},{"osd":1,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0780000000000001,"15min":1.097},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8839999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.347},{"interface":"front","average":{"1min":1.171,"5min":1.099,"15min":1.121},"min":{"1min":0.73299999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6259999999999999,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.133}]},{"osd":2,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.105,"15min":1.123},"min":{"1min":0.60399999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.462,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.2},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2470000000000001,"15min":1.2789999999999999},"min":{"1min":0.85299999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.839,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.395}]},{"osd":3,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.198,"15min":1.234},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6200000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.83199999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.591,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":1.5129999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3440000000000001,"15min":1.397},"min":{"1min":0.92900000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.77,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3120000000000001,"15min":1.3009999999999999},"min":{"1min":0.88800000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.651,"5min":1.913,"15min":1.913},"last":0.95799999999999996}]},{"osd":5,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.359,"15min":1.385},"min":{"1min":0.87,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9299999999999999,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.87},{"interface":"front","average":{"1min":1.401,"5min":1.4910000000000001,"15min":1.5629999999999999},"min":{"1min":0.995,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.6919999999999999,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":1.26}]},{"osd":6,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.466,"15min":1.472},"min":{"1min":1.032,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.02,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.466,"5min":1.5069999999999999,"15min":1.546},"min":{"1min":1.052,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.837,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.24}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.88700000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.274},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2130000000000001,"15min":1.216},"min":{"1min":0.82899999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.8,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.1240000000000001}]},{"osd":1,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.038,"15min":1.038},"min":{"1min":0.69399999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.617,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.071},{"interface":"front","average":{"1min":1.167,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.76000000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.645,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.413}]},{"osd":2,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0569999999999999,"15min":1.0680000000000001},"min":{"1min":0.78000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.21,"15min":1.2150000000000001},"min":{"1min":0.98999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.5369999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.284,"15min":1.3200000000000001},"min":{"1min":0.874,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.542,"5min":3.6000000000000001,"15min":3.6000000000000001},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.236,"15min":1.2490000000000001},"min":{"1min":0.70999999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6339999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3420000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3400000000000001,"15min":1.373},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.667,"5min":2.7170000000000001,"15min":2.7170000000000001},"last":1.373},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4219999999999999,"15min":1.4590000000000001},"min":{"1min":0.69499999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.681,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.359,"15min":1.387},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.722,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4570000000000001,"15min":1.4670000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.748,"5min":1.992,"15min":1.992},"last":1.4890000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:14:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5609999999999999,"15min":1.589},"min":{"1min":0.81899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0979999999999999,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.325},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.51,"15min":1.538},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.792,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.002,"15min":0.996},"min":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.9490000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.022,"15min":1.0189999999999999},"min":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4199999999999999,"5min":1.621,"15min":1.621},"last":1.3959999999999999}]},{"osd":1,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.04,"15min":1.054},"min":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.72099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.526,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.2969999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.194,"15min":1.234},"min":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.738,"5min":2.508,"15min":2.508},"last":1.738},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.256,"15min":1.3340000000000001},"min":{"1min":0.84699999999999998,"5min":0.752,"15min":0.752},"max":{"1min":1.649,"5min":2.524,"15min":2.524},"last":1.25}]},{"osd":3,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.169,"15min":1.1579999999999999},"min":{"1min":0.94699999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.839,"5min":2.052,"15min":2.052},"last":1.839},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2210000000000001,"15min":1.288},"min":{"1min":0.76400000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.835,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.0229999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.2829999999999999,"15min":1.2889999999999999},"min":{"1min":0.98599999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":2.419},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.95799999999999996,"5min":0.747,"15min":0.747},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":2.6240000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3480000000000001,"15min":1.4139999999999999},"min":{"1min":0.91600000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1859999999999999,"5min":2.472,"15min":2.472},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4430000000000001,"15min":1.4570000000000001},"min":{"1min":0.94199999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9710000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.6459999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.401,"15min":1.4470000000000001},"min":{"1min":0.94699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.0289999999999999,"5min":2.617,"15min":2.617},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.51,"15min":1.5660000000000001},"min":{"1min":1.1519999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0990000000000002,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":1.3500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0589999999999999,"15min":1.0760000000000001},"min":{"1min":0.749,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0449999999999999,"15min":0.98499999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.958,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.403}]},{"osd":1,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.109,"15min":1.089},"min":{"1min":0.77100000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4910000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.123,"15min":1.133},"min":{"1min":0.91800000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":2.0990000000000002}]},{"osd":2,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.081,"15min":1.0649999999999999},"min":{"1min":0.69599999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1799999999999999,"15min":1.1919999999999999},"min":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.5269999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.484}]},{"osd":3,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.236,"15min":1.26},"min":{"1min":0.80400000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.238,"5min":1.198,"15min":1.212},"min":{"1min":0.88200000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5329999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.355}]},{"osd":5,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3109999999999999,"15min":1.3200000000000001},"min":{"1min":0.93999999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0790000000000002,"5min":2.617,"15min":2.617},"last":1.988},{"interface":"front","average":{"1min":1.302,"5min":1.347,"15min":1.399},"min":{"1min":1,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.2730000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":2.2730000000000001}]},{"osd":6,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3109999999999999,"15min":1.335},"min":{"1min":0.879,"5min":0.627,"15min":0.627},"max":{"1min":1.615,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.462,"15min":1.46},"min":{"1min":1.2709999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.0539999999999998,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":2.0539999999999998}]},{"osd":7,"last update":"Wed Oct 11 06:15:08 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.468,"15min":1.5369999999999999},"min":{"1min":1.046,"5min":1.018,"15min":1.018},"max":{"1min":2.254,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":2.1829999999999998},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4470000000000001,"15min":1.4890000000000001},"min":{"1min":1.228,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.681,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.681}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.84999999999999998,"15min":0.89300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.5569999999999999,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.92400000000000004,"15min":0.95699999999999996},"min":{"1min":0.42399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5009999999999999,"5min":2.097,"15min":2.097},"last":1.327}]},{"osd":1,"last update":"Wed Oct 11 06:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.85799999999999998,"15min":0.89600000000000002},"min":{"1min":0.42499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1759999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0189999999999999,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.6739999999999999,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.292}]},{"osd":2,"last update":"Wed Oct 11 06:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":1.026},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.363,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":0.88},{"interface":"front","average":{"1min":1.022,"5min":1.077,"15min":1.171},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.7270000000000001,"5min":4.1470000000000002,"15min":4.1470000000000002},"last":1.0820000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:15:05 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1639999999999999,"15min":1.214},"min":{"1min":0.72099999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7749999999999999,"5min":3.286,"15min":3.286},"last":1.421},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2569999999999999,"15min":1.288},"min":{"1min":0.84899999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5129999999999999,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":1.272}]},{"osd":5,"last update":"Wed Oct 11 06:15:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.341,"15min":1.4019999999999999},"min":{"1min":0.80800000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.829,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.387},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.256,"15min":1.319},"min":{"1min":0.75600000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.609,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.1279999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:15:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3220000000000001,"15min":1.3700000000000001},"min":{"1min":0.83799999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5880000000000001,"5min":3.2570000000000001,"15min":3.2570000000000001},"last":1.177},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4199999999999999,"15min":1.536},"min":{"1min":0.83599999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.986,"5min":5.1100000000000003,"15min":5.1100000000000003},"last":1.2150000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:15:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3100000000000001,"15min":1.3680000000000001},"min":{"1min":0.89500000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.9550000000000001,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.238},{"interface":"front","average":{"1min":1.274,"5min":1.3660000000000001,"15min":1.4019999999999999},"min":{"1min":0.78800000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8049999999999999,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":1.351}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.78400000000000003,"15min":0.82299999999999995},"min":{"1min":0.502,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3600000000000001,"5min":1.494,"15min":1.494},"last":1.012},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.40899999999999997,"5min":0.31,"15min":0.31},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.054}]},{"osd":1,"last update":"Wed Oct 11 06:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.56699999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.93799999999999994,"15min":0.95599999999999996},"min":{"1min":0.55000000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.153}]},{"osd":3,"last update":"Wed Oct 11 06:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.88600000000000001,"15min":0.91800000000000004},"min":{"1min":0.56699999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.321,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.071,"5min":1.089,"15min":1.2030000000000001},"min":{"1min":0.60099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.506,"5min":3.032,"15min":3.032},"last":1.1919999999999999}]},{"osd":4,"last update":"Wed Oct 11 06:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.242,"15min":1.286},"min":{"1min":0.86299999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6839999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2290000000000001,"15min":1.2589999999999999},"min":{"1min":0.876,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.1949999999999998,"5min":3.069,"15min":3.069},"last":1.286}]},{"osd":5,"last update":"Wed Oct 11 06:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2050000000000001,"15min":1.2390000000000001},"min":{"1min":0.77700000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7569999999999999,"5min":2.181,"15min":2.181},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.323,"15min":1.385},"min":{"1min":0.86399999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.79,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.6879999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3280000000000001,"15min":1.359},"min":{"1min":0.91900000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.3010000000000002,"5min":2.633,"15min":2.633},"last":1.379},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.292,"15min":1.294},"min":{"1min":0.80300000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8540000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.8540000000000001}]},{"osd":7,"last update":"Wed Oct 11 06:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.345,"15min":1.4079999999999999},"min":{"1min":0.84999999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.254,"5min":2.794,"15min":2.794},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.3069999999999999,"15min":1.333},"min":{"1min":0.84299999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.893,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.6399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":688128,"data_stored":300037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.79100000000000004,"15min":0.80300000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.131,"5min":1.302,"15min":1.302},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.81999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":1.054}]},{"osd":2,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.86199999999999999},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2190000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.498,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4530000000000001,"5min":1.768,"15min":1.768},"last":0.89000000000000001}]},{"osd":3,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0009999999999999},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5960000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4990000000000001,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.97999999999999998}]},{"osd":4,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2609999999999999,"15min":1.2869999999999999},"min":{"1min":0.88,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8580000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.873,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0259999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.3120000000000001}]},{"osd":5,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2609999999999999,"15min":1.2929999999999999},"min":{"1min":0.83199999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7689999999999999,"5min":3.319,"15min":3.319},"last":1.23},{"interface":"front","average":{"1min":1.224,"5min":1.288,"15min":1.3200000000000001},"min":{"1min":0.82499999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.9930000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.202}]},{"osd":6,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.292,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.931,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.284,"15min":1.2989999999999999},"min":{"1min":0.76200000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7070000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.5109999999999999}]},{"osd":7,"last update":"Wed Oct 11 06:14:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3600000000000001,"15min":1.4119999999999999},"min":{"1min":0.83699999999999997,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.575,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.401,"15min":1.425},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.3460000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.4179999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1282048,"data_stored":890405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 11 06:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84799999999999998,"15min":0.86899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3879999999999999,"5min":1.427,"15min":1.427},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.96499999999999997,"15min":1.052},"min":{"1min":0.46999999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.538,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":1.1539999999999999}]},{"osd":2,"last update":"Wed Oct 11 06:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.88800000000000001,"15min":0.90200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.486,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.92100000000000004,"15min":0.98899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.536,"5min":2.444,"15min":2.444},"last":1.016}]},{"osd":3,"last update":"Wed Oct 11 06:15:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83699999999999997,"15min":0.86299999999999999},"min":{"1min":0.55400000000000005,"5min":0.31,"15min":0.31},"max":{"1min":1.466,"5min":1.474,"15min":1.474},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0669999999999999,"15min":1.115},"min":{"1min":0.59999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.593,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.2190000000000001}]},{"osd":4,"last update":"Wed Oct 11 06:15:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2789999999999999,"15min":1.3240000000000001},"min":{"1min":1,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.488,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":1.343},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2609999999999999,"15min":1.2889999999999999},"min":{"1min":0.84299999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0990000000000002,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed Oct 11 06:15:04 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0169999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.095},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.25,"15min":1.284},"min":{"1min":0.876,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.863,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.2809999999999999}]},{"osd":6,"last update":"Wed Oct 11 06:15:04 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3100000000000001,"15min":1.3520000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.9370000000000001,"5min":2.552,"15min":2.552},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3,"15min":1.3640000000000001},"min":{"1min":0.80900000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.974,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.427}]},{"osd":7,"last update":"Wed Oct 11 06:15:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.321,"15min":1.381},"min":{"1min":0.83699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.0470000000000002,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.3120000000000001,"15min":1.3540000000000001},"min":{"1min":0.871,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2280000000000002,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.1950000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-11T06:15:12.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-11T06:15:12.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.711+0000 7fc1bbfff640 1 Processor -- start 2023-10-11T06:15:12.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.711+0000 7fc1bbfff640 1 -- start start 2023-10-11T06:15:12.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.712+0000 7fc1bbfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1bc06a380 0x7fc1bc1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.712+0000 7fc1bbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1bc0ce850 0x7fc1bc06ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.712+0000 7fc1bbfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1bc06b2c0 con 0x7fc1bc06d980 2023-10-11T06:15:12.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.712+0000 7fc1bbfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc15c740 con 0x7fc1bc0ce850 2023-10-11T06:15:12.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.712+0000 7fc1bbfff640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc0cda90 con 0x7fc1bc06a380 2023-10-11T06:15:12.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.713+0000 7fc1bb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc1bc0ce850 0x7fc1bc06ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.713+0000 7fc1baffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc1bc06d980 0x7fc1bc069c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:33412/0 (socket says 172.21.15.84:33412) 2023-10-11T06:15:12.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.713+0000 7fc1baffd640 1 -- 172.21.15.84:0/578202567 learned_addr learned my addr 172.21.15.84:0/578202567 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-10-11T06:15:12.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.713+0000 7fc1b27fc640 1 --2- 172.21.15.84:0/578202567 >> v2:172.21.15.99:3300/0 conn(0x7fc1bc06a380 0x7fc1bc1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.714+0000 7fc1ba7fc640 1 -- 172.21.15.84:0/578202567 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629549732 0 0) 0x7fc1bc06b2c0 con 0x7fc1bc06d980 2023-10-11T06:15:12.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.714+0000 7fc1ba7fc640 1 -- 172.21.15.84:0/578202567 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a8003520 con 0x7fc1bc06d980 2023-10-11T06:15:12.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.714+0000 7fc1bb7fe640 1 -- 172.21.15.84:0/578202567 >> v2:172.21.15.99:3300/0 conn(0x7fc1bc06a380 msgr2=0x7fc1bc1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:15:12.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.714+0000 7fc1bb7fe640 1 --2- 172.21.15.84:0/578202567 >> v2:172.21.15.99:3300/0 conn(0x7fc1bc06a380 0x7fc1bc1608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.714+0000 7fc1bb7fe640 1 -- 172.21.15.84:0/578202567 >> v1:172.21.15.84:6789/0 conn(0x7fc1bc06d980 legacy=0x7fc1bc069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.714+0000 7fc1bb7fe640 1 -- 172.21.15.84:0/578202567 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0cdd70 con 0x7fc1bc0ce850 2023-10-11T06:15:12.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.715+0000 7fc1bb7fe640 1 --2- 172.21.15.84:0/578202567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc06ad80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc072320 tx=0x7fc1b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeba4423d2d9b1e server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:12.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.715+0000 7fc1ba7fc640 1 -- 172.21.15.84:0/578202567 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b402cc80 con 0x7fc1bc0ce850 2023-10-11T06:15:12.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.715+0000 7fc1ba7fc640 1 -- 172.21.15.84:0/578202567 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b402cde0 con 0x7fc1bc0ce850 2023-10-11T06:15:12.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.715+0000 7fc1ba7fc640 1 -- 172.21.15.84:0/578202567 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4034640 con 0x7fc1bc0ce850 2023-10-11T06:15:12.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 -- 172.21.15.84:0/578202567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 msgr2=0x7fc1bc06ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/578202567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc06ad80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc072320 tx=0x7fc1b402bed0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 -- 172.21.15.84:0/578202567 shutdown_connections 2023-10-11T06:15:12.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/578202567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc06ad80 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/578202567 >> v2:172.21.15.99:3300/0 conn(0x7fc1bc06a380 0x7fc1bc1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 -- 172.21.15.84:0/578202567 >> 172.21.15.84:0/578202567 conn(0x7fc1bc0cb7f0 msgr2=0x7fc1bc0cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:15:12.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 -- 172.21.15.84:0/578202567 shutdown_connections 2023-10-11T06:15:12.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.716+0000 7fc1bbfff640 1 -- 172.21.15.84:0/578202567 wait complete. 2023-10-11T06:15:12.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.718+0000 7fc1bbfff640 1 Processor -- start 2023-10-11T06:15:12.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.718+0000 7fc1bbfff640 1 -- start start 2023-10-11T06:15:12.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bbfff640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1bc06a380 0x7fc1bc0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc06d980 0x7fc1bc15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bbfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0b3220 con 0x7fc1bc0ce850 2023-10-11T06:15:12.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bbfff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc15bb00 con 0x7fc1bc06d980 2023-10-11T06:15:12.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bbfff640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0b2850 con 0x7fc1bc06a380 2023-10-11T06:15:12.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc15f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53994/0 (socket says 172.21.15.84:53994) 2023-10-11T06:15:12.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bb7fe640 1 -- 172.21.15.84:0/1332238182 learned_addr learned my addr 172.21.15.84:0/1332238182 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:15:12.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1b27fc640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc06d980 0x7fc1bc15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1baffd640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1bc06a380 0x7fc1bc0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bb7fe640 1 -- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1bc06a380 msgr2=0x7fc1bc0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bb7fe640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1bc06a380 0x7fc1bc0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.719+0000 7fc1bb7fe640 1 -- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc06d980 msgr2=0x7fc1bc15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1bb7fe640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc06d980 0x7fc1bc15b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1bb7fe640 1 -- 172.21.15.84:0/1332238182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009a70 con 0x7fc1bc0ce850 2023-10-11T06:15:12.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1bb7fe640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc15f8a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc1b402c4e0 tx=0x7fc1b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:12.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b403e610 con 0x7fc1bc0ce850 2023-10-11T06:15:12.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b403e770 con 0x7fc1bc0ce850 2023-10-11T06:15:12.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b403eb20 con 0x7fc1bc0ce850 2023-10-11T06:15:12.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc0b1c40 con 0x7fc1bc0ce850 2023-10-11T06:15:12.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.720+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc0b2200 con 0x7fc1bc0ce850 2023-10-11T06:15:12.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.722+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc0b29c0 con 0x7fc1bc0ce850 2023-10-11T06:15:12.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.722+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7fc1b403b050 con 0x7fc1bc0ce850 2023-10-11T06:15:12.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.723+0000 7fc1b8ff9640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc19c06a830 0x7fc19c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:12.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.723+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc1b40aae80 con 0x7fc1bc0ce850 2023-10-11T06:15:12.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.724+0000 7fc1baffd640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc19c06a830 0x7fc19c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:12.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.724+0000 7fc1baffd640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc19c06a830 0x7fc19c06cce0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000f60 tx=0x7fc1ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:12.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.728+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7fc1b4076400 con 0x7fc1bc0ce850 2023-10-11T06:15:12.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.920+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fc1bc04e890 con 0x7fc1bc0ce850 2023-10-11T06:15:12.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.934+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc1b4076400 con 0x7fc1bc0ce850 2023-10-11T06:15:12.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.934+0000 7fc1b8ff9640 1 -- 172.21.15.84:0/1332238182 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fc1b4009450 con 0x7fc1bc0ce850 2023-10-11T06:15:12.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc19c06a830 msgr2=0x7fc19c06cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc19c06a830 0x7fc19c06cce0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000f60 tx=0x7fc1ac009040 comp rx=0 tx=0).stop 2023-10-11T06:15:12.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 msgr2=0x7fc1bc15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:12.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc15f8a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc1b402c4e0 tx=0x7fc1b4004a40 comp rx=0 tx=0).stop 2023-10-11T06:15:12.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 shutdown_connections 2023-10-11T06:15:12.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7fc19c06a830 0x7fc19c06cce0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1bc0ce850 0x7fc1bc15f8a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc06d980 0x7fc1bc15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 --2- 172.21.15.84:0/1332238182 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1bc06a380 0x7fc1bc0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:12.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 >> 172.21.15.84:0/1332238182 conn(0x7fc1bc0cb7f0 msgr2=0x7fc1bc070610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:15:12.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 shutdown_connections 2023-10-11T06:15:12.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:12.938+0000 7fc1bbfff640 1 -- 172.21.15.84:0/1332238182 wait complete. 2023-10-11T06:15:12.957 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-11T06:15:12.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-11T06:15:13.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.152+0000 7ff441918640 1 Processor -- start 2023-10-11T06:15:13.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.152+0000 7ff441918640 1 -- start start 2023-10-11T06:15:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.153+0000 7ff441918640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff43c15c290 0x7ff43c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.153+0000 7ff441918640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c0d25e0 0x7ff43c0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.153+0000 7ff441918640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0d0eb0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.153+0000 7ff441918640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c0d2a30 con 0x7ff43c15c290 2023-10-11T06:15:13.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.153+0000 7ff441918640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c06cd10 con 0x7ff43c0d25e0 2023-10-11T06:15:13.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff43c15c290 0x7ff43c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:13.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff43c15c290 0x7ff43c0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54000/0 (socket says 172.21.15.84:54000) 2023-10-11T06:15:13.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43affd640 1 -- 172.21.15.84:0/3763988599 learned_addr learned my addr 172.21.15.84:0/3763988599 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:15:13.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43a7fc640 1 --2- 172.21.15.84:0/3763988599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c0d25e0 0x7ff43c0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:13.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43affd640 1 -- 172.21.15.84:0/3763988599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c0d25e0 msgr2=0x7ff43c0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:15:13.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43affd640 1 --2- 172.21.15.84:0/3763988599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c0d25e0 0x7ff43c0d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43affd640 1 -- 172.21.15.84:0/3763988599 >> v1:172.21.15.99:6789/0 conn(0x7ff43c0d0a50 legacy=0x7ff43c06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-11T06:15:13.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.154+0000 7ff43affd640 1 -- 172.21.15.84:0/3763988599 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0afb60 con 0x7ff43c15c290 2023-10-11T06:15:13.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.155+0000 7ff43affd640 1 --2- 172.21.15.84:0/3763988599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c15c290 0x7ff43c0af4d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009870 tx=0x7ff42c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59829fdbb7912c02 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:13.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff439ffb640 1 -- 172.21.15.84:0/3763988599 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c0317c0 con 0x7ff43c15c290 2023-10-11T06:15:13.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff439ffb640 1 -- 172.21.15.84:0/3763988599 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff42c031920 con 0x7ff43c15c290 2023-10-11T06:15:13.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff439ffb640 1 -- 172.21.15.84:0/3763988599 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c031cd0 con 0x7ff43c15c290 2023-10-11T06:15:13.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 -- 172.21.15.84:0/3763988599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c15c290 msgr2=0x7ff43c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:13.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 --2- 172.21.15.84:0/3763988599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c15c290 0x7ff43c0af4d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009870 tx=0x7ff42c02b860 comp rx=0 tx=0).stop 2023-10-11T06:15:13.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 -- 172.21.15.84:0/3763988599 shutdown_connections 2023-10-11T06:15:13.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 --2- 172.21.15.84:0/3763988599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c0d25e0 0x7ff43c0d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 --2- 172.21.15.84:0/3763988599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c15c290 0x7ff43c0af4d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 -- 172.21.15.84:0/3763988599 >> 172.21.15.84:0/3763988599 conn(0x7ff43c071cc0 msgr2=0x7ff43c09bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:15:13.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 -- 172.21.15.84:0/3763988599 shutdown_connections 2023-10-11T06:15:13.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.156+0000 7ff441918640 1 -- 172.21.15.84:0/3763988599 wait complete. 2023-10-11T06:15:13.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.158+0000 7ff441918640 1 Processor -- start 2023-10-11T06:15:13.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.158+0000 7ff441918640 1 -- start start 2023-10-11T06:15:13.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff441918640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c0d0a50 0x7ff43c1f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:13.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff441918640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c0d25e0 0x7ff43c09cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:13.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff441918640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c15c290 0x7ff43c1450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:13.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff441918640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c156f90 con 0x7ff43c0d0a50 2023-10-11T06:15:13.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff441918640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c1f1c20 con 0x7ff43c0d25e0 2023-10-11T06:15:13.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff441918640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c144030 con 0x7ff43c15c290 2023-10-11T06:15:13.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff43affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c0d0a50 0x7ff43c1f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:13.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff43affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c0d0a50 0x7ff43c1f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54004/0 (socket says 172.21.15.84:54004) 2023-10-11T06:15:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.159+0000 7ff43affd640 1 -- 172.21.15.84:0/2840997299 learned_addr learned my addr 172.21.15.84:0/2840997299 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-10-11T06:15:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff43affd640 1 -- 172.21.15.84:0/2840997299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c15c290 msgr2=0x7ff43c1450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff43affd640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c15c290 0x7ff43c1450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff43affd640 1 -- 172.21.15.84:0/2840997299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c0d25e0 msgr2=0x7ff43c09cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:13.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff43affd640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c0d25e0 0x7ff43c09cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff43affd640 1 -- 172.21.15.84:0/2840997299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0094d0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff43affd640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c0d0a50 0x7ff43c1f16e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff42c02b8a0 tx=0x7ff42c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:13.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff423fff640 1 -- 172.21.15.84:0/2840997299 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c032860 con 0x7ff43c0d0a50 2023-10-11T06:15:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff423fff640 1 -- 172.21.15.84:0/2840997299 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff42c0329e0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c144310 con 0x7ff43c0d0a50 2023-10-11T06:15:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.160+0000 7ff423fff640 1 -- 172.21.15.84:0/2840997299 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c032db0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.161+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c1448d0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.162+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c073ed0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.162+0000 7ff423fff640 1 -- 172.21.15.84:0/2840997299 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 88915+0+0 (secure 0 0 0) 0x7ff42c0453f0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.163+0000 7ff423fff640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff42806a830 0x7ff42806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-11T06:15:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.163+0000 7ff423fff640 1 -- 172.21.15.84:0/2840997299 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff42c0ab0c0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.164+0000 7ff43a7fc640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff42806a830 0x7ff42806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-11T06:15:13.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.164+0000 7ff43a7fc640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff42806a830 0x7ff42806cce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff430006fd0 tx=0x7ff430008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-11T06:15:13.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.167+0000 7ff423fff640 1 -- 172.21.15.84:0/2840997299 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179088 (secure 0 0 0) 0x7ff42c076530 con 0x7ff43c0d0a50 2023-10-11T06:15:13.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.396+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff43c06c9d0 con 0x7ff43c0d0a50 2023-10-11T06:15:13.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.397+0000 7ff423fff640 1 -- 172.21.15.84:0/2840997299 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3529 (secure 0 0 0) 0x7ff42c079b50 con 0x7ff43c0d0a50 2023-10-11T06:15:13.398 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-10-11T06:15:13.399 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1835716973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1835716973},{"type":"v1","addr":"172.21.15.99:6833","nonce":1835716973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1415879342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1415879342},{"type":"v1","addr":"172.21.15.99:6835","nonce":1415879342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2122307729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2122307729},{"type":"v1","addr":"172.21.15.84:6836","nonce":2122307729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6537,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3722209938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":3722209938},{"type":"v1","addr":"172.21.15.84:6837","nonce":3722209938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-10-11T06:15:13.399 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 26 2023-10-11T06:15:13.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff42806a830 msgr2=0x7ff42806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:13.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff42806a830 0x7ff42806cce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff430006fd0 tx=0x7ff430008040 comp rx=0 tx=0).stop 2023-10-11T06:15:13.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c0d0a50 msgr2=0x7ff43c1f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-11T06:15:13.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c0d0a50 0x7ff43c1f16e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff42c02b8a0 tx=0x7ff42c0046c0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 shutdown_connections 2023-10-11T06:15:13.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:6800/87595,v1:172.21.15.84:6801/87595] conn(0x7ff42806a830 0x7ff42806cce0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff43c15c290 0x7ff43c1450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c0d25e0 0x7ff43c09cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 --2- 172.21.15.84:0/2840997299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff43c0d0a50 0x7ff43c1f16e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-11T06:15:13.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 >> 172.21.15.84:0/2840997299 conn(0x7ff43c071cc0 msgr2=0x7ff43c0b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-11T06:15:13.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 shutdown_connections 2023-10-11T06:15:13.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-10-11T06:15:13.399+0000 7ff441918640 1 -- 172.21.15.84:0/2840997299 wait complete. 2023-10-11T06:15:13.418 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-11T06:15:13.418 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-11T06:15:13.419 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-11T06:15:13.419 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:13.434 INFO:tasks.ceph.mds.a:Stopped 2023-10-11T06:15:13.434 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-11T06:15:13.435 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:13.492 INFO:tasks.ceph.mds.c:Stopped 2023-10-11T06:15:13.493 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-11T06:15:13.493 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:13.513 INFO:tasks.ceph.mds.b:Stopped 2023-10-11T06:15:13.514 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-11T06:15:13.514 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:13.576 INFO:tasks.ceph.mds.d:Stopped 2023-10-11T06:15:13.577 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-11T06:15:13.577 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-11T06:15:13.577 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:13.634 INFO:tasks.ceph.osd.0:Stopped 2023-10-11T06:15:13.635 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-11T06:15:13.635 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:13.674 INFO:tasks.ceph.osd.1:Stopped 2023-10-11T06:15:13.675 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-11T06:15:13.675 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:14.787 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-10-11T06:15:19.777 INFO:tasks.ceph.osd.2:Stopped 2023-10-11T06:15:19.779 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-11T06:15:19.779 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:19.854 INFO:tasks.ceph.osd.3:Stopped 2023-10-11T06:15:19.855 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-11T06:15:19.855 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:19.915 INFO:tasks.ceph.osd.4:Stopped 2023-10-11T06:15:19.916 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-11T06:15:19.916 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:19.981 INFO:tasks.ceph.osd.5:Stopped 2023-10-11T06:15:19.981 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-11T06:15:19.982 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:20.041 INFO:tasks.ceph.osd.6:Stopped 2023-10-11T06:15:20.041 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-11T06:15:20.042 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:20.106 INFO:tasks.ceph.osd.7:Stopped 2023-10-11T06:15:20.107 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-11T06:15:20.107 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-11T06:15:20.107 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:20.175 INFO:tasks.ceph.mgr.y:Stopped 2023-10-11T06:15:20.176 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-11T06:15:20.176 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:20.268 INFO:tasks.ceph.mgr.x:Stopped 2023-10-11T06:15:20.269 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-11T06:15:20.269 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-11T06:15:20.270 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:20.311 INFO:tasks.ceph.mon.a:Stopped 2023-10-11T06:15:20.311 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-11T06:15:20.312 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:20.345 INFO:tasks.ceph.mon.b:Stopped 2023-10-11T06:15:20.346 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-11T06:15:20.346 INFO:teuthology.orchestra.run:waiting for 300 2023-10-11T06:15:20.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-11T06:15:20.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-10-11T06:15:20.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-10-11T06:15:20.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-10-11T06:15:20.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-11T06:15:20.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-10-11T06:15:20.418 INFO:tasks.ceph.mon.c:Stopped 2023-10-11T06:15:20.419 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-11T06:15:20.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-11T06:15:20.529 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:15:20.529 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-11T06:15:20.914 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:15:20.915 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-11T06:15:21.161 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:15:21.162 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-11T06:15:21.408 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:15:21.408 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-11T06:15:21.683 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:15:21.683 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-11T06:15:22.127 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:15:22.127 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-11T06:15:22.374 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:15:22.374 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-11T06:15:22.645 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:15:22.645 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-11T06:15:22.895 INFO:tasks.ceph:Archiving mon data... 2023-10-11T06:15:22.896 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/data/mon.a.tgz 2023-10-11T06:15:22.897 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2023-10-11T06:15:22.957 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.RFmapMmkxQ 2023-10-11T06:15:22.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RFmapMmkxQ 2023-10-11T06:15:23.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.RFmapMmkxQ 2023-10-11T06:15:23.320 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.RFmapMmkxQ is 368KB 2023-10-11T06:15:23.428 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.RFmapMmkxQ 2023-10-11T06:15:23.481 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/data/mon.b.tgz 2023-10-11T06:15:23.481 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-10-11T06:15:23.533 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.1wnpXNZeua 2023-10-11T06:15:23.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.1wnpXNZeua 2023-10-11T06:15:23.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.1wnpXNZeua 2023-10-11T06:15:23.867 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.1wnpXNZeua is 482KB 2023-10-11T06:15:23.967 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.1wnpXNZeua 2023-10-11T06:15:24.022 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/data/mon.c.tgz 2023-10-11T06:15:24.023 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2023-10-11T06:15:24.070 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.MTrcWWbU5Y 2023-10-11T06:15:24.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.MTrcWWbU5Y 2023-10-11T06:15:24.274 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.MTrcWWbU5Y 2023-10-11T06:15:24.421 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.MTrcWWbU5Y is 482KB 2023-10-11T06:15:24.529 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.MTrcWWbU5Y 2023-10-11T06:15:24.588 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-11T06:15:24.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-11T06:15:24.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-11T06:15:24.691 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-11T06:15:24.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-11T06:15:24.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-11T06:15:24.793 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-11T06:15:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-11T06:15:24.807 INFO:tasks.ceph:Archiving crash dumps... 2023-10-11T06:15:24.809 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/remote/smithi084/crash 2023-10-11T06:15:24.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-11T06:15:24.898 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/remote/smithi099/crash 2023-10-11T06:15:24.900 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-11T06:15:24.999 INFO:tasks.ceph:Shutting down logrotate 2023-10-11T06:15:25.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:15:25.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-11T06:15:25.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-11T06:15:25.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-11T06:15:25.349 INFO:tasks.ceph:Compressing logs... 2023-10-11T06:15:25.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T06:15:25.359 DEBUG:teuthology.orchestra.run.smithi099:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T06:15:25.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-11T06:15:25.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-10-11T06:15:25.414 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-10-11T06:15:25.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-10-11T06:15:25.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-11T06:15:26.834 INFO:tasks.ceph:Archiving logs... 2023-10-11T06:15:26.835 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/remote/smithi084/log 2023-10-11T06:15:26.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-11T06:15:27.361 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/remote/smithi099/log 2023-10-11T06:15:27.363 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-11T06:15:27.598 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-11T06:15:27.625 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-11T06:15:27.625 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-11T06:15:27.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-11T06:15:27.788 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-11T06:15:27.797 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-11T06:15:27.797 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T06:15:27.807 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-11T06:15:27.854 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2023-10-11T06:15:27.863 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2023-10-11T06:15:27.865 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-11T06:15:27.865 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2023-10-11T06:15:27.865 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 -833us[ -833us] +/- 99ms 2023-10-11T06:15:27.865 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -649us[ -648us] +/- 68ms 2023-10-11T06:15:27.866 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +1372us[+1372us] +/- 72ms 2023-10-11T06:15:27.866 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:15:27.872 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-11T06:15:27.873 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2023-10-11T06:15:27.873 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 -826us[ -808us] +/- 99ms 2023-10-11T06:15:27.873 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -641us[ -623us] +/- 68ms 2023-10-11T06:15:27.874 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +1378us[+1397us] +/- 72ms 2023-10-11T06:15:27.874 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-11T06:15:27.875 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-11T06:15:27.887 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-11T06:15:27.887 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-11T06:15:27.908 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-11T06:15:28.184 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-11T06:15:28.446 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-11T06:15:28.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:15:28.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-11T06:15:28.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:15:28.668 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-11T06:15:28.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-11T06:15:28.692 INFO:teuthology.task.internal:Duration was 1028.566746 seconds 2023-10-11T06:15:28.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-11T06:15:28.705 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-11T06:15:28.705 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-11T06:15:28.715 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-11T06:15:28.881 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-11T06:15:28.889 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-11T06:15:29.171 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-11T06:15:29.171 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2023-10-11T06:15:29.171 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-11T06:15:29.240 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2023-10-11T06:15:29.240 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-11T06:15:29.319 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-11T06:15:29.320 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T06:15:29.329 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-11T06:15:29.426 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-11T06:15:29.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-11T06:15:29.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-11T06:15:29.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-11T06:15:29.628 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-11T06:15:29.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-11T06:15:29.727 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-11T06:15:29.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-11T06:15:29.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-11T06:15:29.850 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-11T06:15:29.956 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2023-10-11T06:15:29.961 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2023-10-11T06:15:30.012 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-11T06:15:30.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:15:30.060 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-11T06:15:30.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-11T06:15:30.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-11T06:15:30.130 INFO:teuthology.task.internal:Transferring archived files... 2023-10-11T06:15:30.131 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/remote/smithi084 2023-10-11T06:15:30.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-11T06:15:30.245 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719/remote/smithi099 2023-10-11T06:15:30.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-11T06:15:30.354 INFO:teuthology.task.internal:Removing archive directory... 2023-10-11T06:15:30.355 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-11T06:15:30.365 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-11T06:15:30.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-11T06:15:30.436 INFO:teuthology.task.internal:Not uploading archives. 2023-10-11T06:15:30.437 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-11T06:15:30.446 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-11T06:15:30.447 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-11T06:15:30.456 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-11T06:15:30.506 INFO:teuthology.orchestra.run.smithi084.stdout: 393241 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 11 06:15 /home/ubuntu/cephtest 2023-10-11T06:15:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 11 06:15 /home/ubuntu/cephtest 2023-10-11T06:15:30.516 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-11T06:15:30.528 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-11T06:15:30.549 INFO:teuthology.nuke:Checking targets against current locks 2023-10-11T06:15:30.576 DEBUG:teuthology.nuke:shortname: smithi084 2023-10-11T06:15:30.577 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-11T06:15:30.603 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 05:49:11.621701', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIz07+vwo6enfYjYT3bgN9oAW/+WdhG0ul6F9y0AySoW1NTZ85wpDnIsN/oxucbA12mY5FlIxagz7oq58PLVr2s='} 2023-10-11T06:15:30.604 DEBUG:teuthology.nuke:shortname: smithi099 2023-10-11T06:15:30.605 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-11T06:15:30.631 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_16:48:27-fs-wip-vshankar-testing-reef-20231009.131610-testing-default-smithi/7418719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-11 05:49:11.619716', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO4Fkjn9MgszQq+O3mVjZjzL214/drNn9oJ1GIkUSVYEf036TMD8K0z7blUuOud3nbFZcubNP/Ios9UYGazh49c='} 2023-10-11T06:15:30.652 INFO:teuthology.orchestra.console.smithi084:Power off 2023-10-11T06:15:30.653 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-11T06:15:30.675 DEBUG:teuthology.orchestra.console.smithi084:power off output: Chassis Power Control: Down/Off 2023-10-11T06:15:30.675 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T06:15:30.692 INFO:teuthology.orchestra.console.smithi099:Power off 2023-10-11T06:15:30.692 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-11T06:15:30.701 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2023-10-11T06:15:30.716 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-10-11T06:15:30.716 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T06:15:30.738 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-10-11T06:15:34.703 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T06:15:34.739 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T06:15:34.814 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2023-10-11T06:15:34.850 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-10-11T06:15:38.818 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T06:15:38.851 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-11T06:15:38.928 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2023-10-11T06:15:38.960 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-10-11T06:15:39.029 INFO:teuthology.orchestra.console.smithi084:Power off completed 2023-10-11T06:15:39.062 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-10-11T06:15:39.167 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} duration: 1028.5667464733124 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c mon.b-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=43bdba28b8d24e76b4c5a7d20341f767 status: fail success: false 2023-10-11T06:15:39.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-11T06:15:39.264 INFO:teuthology.run:FAIL